On modern GPUs, dynamic optimization is currently limited by sensitivity computations that require information from full forward and adjoint wave fields. Since forward and adjoint solutions are computed in opposing time directions, the forward solution must be stored, requiring substantial memory even with checkpointing or data compression techniques. As a result, the problem is memory-bound rather than bound by wall clock time when working with modern GPU-based implementations that have limited memory capacity.\\ To overcome this limitation, we introduce a new approach [1] to approximating sensitivity computations based on the adjoint method (for self-adjoint problems, thus excluding attenuating effects) that relies on the principle of superposition. The approximation allows an incremental computation of the sensitivity, reducing the memory burden to that of the solution at a small number of time steps, i.e., to the number of degrees of freedom. This enables sensitivity computations for problems with billions of degrees of freedom on current GPUs. We demonstrate the approach on full waveform inversion and transient acoustic topology optimization problems, relying on a simple, yet highly efficient finite difference forward solver implemented in CUDA.