Differentiable Physics is a paradigm which allows learning algorithms to interact with gradients of classical physics solvers. This has proven effective across many domains, e.g., solving inverse problems, integrating physical constraints, and especially, creating hybrid models that blend classical numerical techniques with learned components. Traditionally, these methods prioritize achieving the highest possible accuracy in the physics solver. However, at the core of most classical physics solvers are iterative linear solvers, hence, differentiating through them introduces a severe computational burden as iterations grow large. We show that full accuracy of the network is achievable through solvers significantly coarser than fully converged solvers. We propose Progressively Refined Differentiable Physics (PRDP), an approach that identifies the level of solver refinement sufficient for full training accuracy. By beginning with coarse physics, adaptively refining it during training, and stopping refinement at the level adequate for training, it enables significant compute savings without sacrificing network accuracy. We validate its performance on a variety of learning scenarios involving differentiable physics solvers such as inverse problems, autoregressive neural emulators, and correction-based neural-hybrid solvers. In the challenging example of emulating the Navier-Stokes equations, we reduce training time by 62%.