Approximating solutions to time-dependent Partial Differential Equations (PDEs) remains a key challenge for neural PDE solvers. Physics-Informed Neural Networks (PINNs) provide a flexible framework for approximating solutions of PDEs, yet their impact is constrained by two fundamental difficulties: reliance on gradient-based iterative optimization over highly nonconvex loss landscapes, and the non-causal treatment of time as an additional spatial coordinate. In this talk, we introduce Frozen-PINNs, which are based on the principle of space-time separation, replacing gradient-based training with random feature methods while enforcing temporal causality by construction. This algorithmic shift eliminates the need for iterative gradient-descent-based training, reducing training time by several orders of magnitude while achieving higher accuracy than state-of-the-art PINNs. Frozen PINNs fundamentally challenge the dependence of PINNs on stochastic gradient descent and specialized hardware, signaling a paradigm shift in PINN development and offering a compelling benchmark for the field.