We propose Physics-Based Flow Matching (PBFM), a generative framework that jointly achieves high distributional fidelity and physical consistency for PDE-constrained data. PBFM enables simultaneous optimization on generative and physics objectives and avoids costly inference-time corrections by integrating three practical components into flow matching training: (i) conflict-free gradient composition (ConFIG), which resolves gradient interference between flow matching and physics-residual losses without manual loss-weight tuning; (ii) unrolled ODE integration during training, which produces more accurate reconstructions of the final, noise-free sample and mitigates Jensen's gap when nonlinear physics are evaluated on posterior means; and (iii) a stochastic sampler that improves distributional accuracy while preserving inference efficiency. Together, these design choices yield models that reduce physical residuals substantially with minimal or no loss in sample fidelity without requiring longer inference time. We demonstrate PBFM on steady Darcy flow, Kolmogorov flow, and an industrial dynamic stall aerodynamic case, showing improved Pareto frontiers between physical residual and distributional metrics compared to state-of-the-art baselines. Ablations confirm that ConFIG and unrolling are complementary: ConFIG aligns optimization directions, while unrolling improves final-sample fidelity and yields significantly lower residuals. PBFM is modular and readily integrated into existing flow matching pipelines, offering a practical tool for surrogate modeling, uncertainty quantification, and rapid scenario evaluation in scientific and engineering applications.