The field of control engineering is undergoing a paradigm shift from conventional finite-dimensional modeling to data-driven, non-parametric representations, motivated by the demand for scalable and adaptable solutions in complex real-world systems. Addressing this challenge, we present an operator-theoretic framework for solving nonlinear stochastic optimal control problems by integrating convex duality, ergodic Hilbert–Sobolev space analysis, and data-driven learning. While traditional methods struggle with scalability and theoretical guarantees for infinite-horizon nonlinear optimal control problems, our technical contributions overcome these limitations through two key innovations: 1. Operator-Theoretic Dynamic Programming for Infinite-Dimensional Systems: We introduce a novel recursion for bounded linear operators, enabling efficient numerical solutions to Hamilton–Jacobi–Bellman (HJB) equations via Galerkin projection [1]. This method reduces computational complexity while preserving theoretical guarantees, operating directly in functional spaces without explicit parametric assumptions. It provides a scalable alternative to traditional discretization techniques, particularly for systems with complex dynamics. 2. Data-Driven Learning via RKHS Embeddings: By leveraging reproducing kernel Hilbert spaces (RKHS), we embed controlled Markov transitions into a non-parametric framework, capturing probabilistic uncertainties directly from empirical data [2]. This approach avoids the need for explicit system models, improving adaptability to real-world environments while maintaining computational tractability through operator-theoretic structures. The integration of RKHS embeddings with operator-theoretic dynamic programming enables robust solutions in the presence of model mismatches and measurement noise. Our framework unifies model-based stability analysis with data-driven learning, offering a theoretically rigorous and computationally efficient pathway for solving complex stochastic optimal control problems. Applications include robotics, energy systems, and autonomous vehicles, where high-dimensional nonlinear dynamics and uncertain environments are prevalent.