The field of numerical methods for Partial Differential Equations (PDEs) has recently been invigorated by a new generation of deep learning-based approaches. These include both novel PDE solvers and surrogate modelling techniques for parametric PDEs. In this talk, I will illustrate recent theoretical advances in this area, focusing on Physics-Informed Neural Networks (PINNs) for the solution of high-dimensional diffusion-reaction PDEs and Deep Learning-based Reduced-Order Modeling (DL-ROM) for surrogate modeling of parametric elliptic PDEs. The results presented take the form of practical existence theorems. Establishing a direct connection between compressed sensing and deep learning, these theorems provide sufficient conditions on the neural network architecture and the number of training samples under which accurate approximations can be achieved. I will illustrate the main ideas underlying this approach and discuss its advantages and limitations.