Sustained by the underneath idea of model reduction, deep learning-based reduced order models (DL-ROMs) are characterized by their distinguishing dimensionality reduction core, usually founded on autoencoders, by which we represent the predominant characteristics of the physical system at hand through a few (latent) coordinates, ultimately aiming to realize the intrinsic dimension of the provided dataset. DL-ROMs can then be used to devise non-intrusive and efficient surrogates for parameterized physical systems. Unfortunately, the original finite-dimensional DL-ROMs' design is a limitation in many applications entailing, e.g., parameterized domains and/or multi-resolution datasets, which generally necessitate a more versatile framework. To overcome such drawback, within this talk we first discuss how to employ modern attention-based architectures to realize versatile and efficient surrogates combining concepts and methods of DL-ROMs and operator learning. The resulting framework, which has been recently considered in the literature, is founded on the Perceiver IO architecture. Within this context, we provide hints and insights for the analysis of such architecture in connection to classical DL-ROM's and operator learning's approximation theory, thus characterizing the complexity reduction aspects related to the Perceiver IO's peculiar design, with special regard to its cross-attention-based dimensionality reduction core.