In this talk, we introduce the dual-Fenchel variational formulation, a new continuous minimum residual principle for PDEs that naturally accommodates nonlinear approximation manifolds such as neural network classes. The method minimizes the dual norm of the residual and admits an equivalent saddle point form obtained from Fenchel conjugacy. This yields a stable and interpretable optimization framework in which the auxiliary field is the Riesz representation of the residual, providing a built-in error estimator. The Dual-Fenchel formulation unifies and generalizes several existing SciML and numerical PDE methods - PINNs, the Deep Ritz method, classical least squares, and DPG-type minimum residual schemes arise as special cases corresponding to particular test spaces and test inner products. However, a particularly attractive option arises when one chooses to approximate both the primal and auxiliary fields with neural networks, yielding the so-called deep dual-Fenchel method. In this talk, we first derive the dual-Fenchel variational formulation in the Hilbert space setting before exploring the deep dual-Fenchel structure that emerges when the primal and auxiliary fields are approximated with neural networks. We then present a practical alternating ascent-descent algorithm, discuss practical improvements to this algorithm, and demonstrate the method on elliptic, advection-diffusion, Helmholtz, and incompressible flow problems. We conclude by showing how the deep dual-Fenchel method can be extended to the Banach space setting.