Controlling the ``temperature" of a stochastic system—its intrinsic level of randomness—is central to Langevin diffusion methods for unconstrained optimization. Formulating this control problem in continuous time leads to a Hamilton–Jacobi–Bellman (HJB) equation, or to its smoother and more tractable exploratory variant. While exploratory HJB equations have been demonstrated to be effective in simple one‑dimensional settings, extending these methods to higher‑dimensional and more realistic problems introduces significant challenges. Two obstacles dominate: determining principled bounds for the control variables and solving the exploratory HJB equations rapidly while maintaining a prescribed level of accuracy. We address both by developing reliable procedures for identifying appropriate control limits and by employing physics‑informed neural networks that embed the structure of the exploratory HJB equation directly into the learning process. Various examples, comparisons, and ablation studies demonstrate how these components interact and why the resulting approach scales effectively beyond low‑dimensional cases. For all examples, we have used tensor neural networks from our recent work to address high-dimensional approximations. References: X. Gao, Z. Q. Xu, and X. Y. Zhou. State-dependent temperature control for Langevin diffusions. SIAM Journal on Control and Optimization, 60(3):1250–1268, 2022. W. Tang, Y. P. Zhang, and X. Y. Zhou. Exploratory HJB equations and their convergence. SIAM Journal on Control and Optimization, 60(6):3191–3216, 2022. T. Wang, Z. Hu, K. Kawaguchi, Z. Zhang, and G. E. Karniadakis. Tensor neural networks for high-dimensional Fokker-Planck equations. Neural Networks, 185:Paper No. 107165, 2025.