Data-driven Koopman-theoretic approaches have proven effective in output prediction, state estimation, and control of nonlinear dynamical systems. For control-affine systems, the Koopman generator’s affine input dependence enables finite-dimensional bilinear approximations. However, a significant challenge in constructing Koopman generators for actuated systems is its reliance on appropriate basis functions or observables, with no unified framework for their selection. Real-world applications often involve noisy, partially observed states, requiring Koopman observables to capture system behavior from input-output data. The challenge of identifying Koopman observables under partial observation is well studied, e.g., time-delayed observables offering viable solutions. However, the presence of actuation reduces the efficacy of these methods. To address this limitation, a recent data-driven method leverages a learned Koopman generator-based bilinear surrogate model with linear reconstruction, demonstrating promise for actuated nonlinear system identification. Further study is needed to assess its effectiveness in complex, partially observed nonlinear systems with actuation and sensitivity to initialization. To address this, we model the dynamics of a control-affine nonlinear system as a bilinear Hidden Markov Model (HMM) defined via Koopman generators with a nonlinear observation map (decoder) modeled using a multilayer perceptron (MLP). The parameters of the HMM and decoder are learned from noisy output data using a neural expectation maximization (EM) approach. In the EM method, the E-step employs an extended Kalman filter and smoother, while the M-step utilizes a least-squares approximation of the Koopman generators, combined with a gradient-based optimization for the decoder parameters. In addition, we present a model-predictive control (MPC)-based output regulation method using the learned HMM as a predictive model. We demonstrate the performance of our method on three nonlinear systems: (1) an actuated polynomial system with a slow manifold and partial observation, (2) a forced Duffing oscillator with partial observation, and (3) an unforced Kuramoto-Sivashinsky equation withmnoisy observation.