We present an RL based framework for parameter-varying control of complex dynamical systems, whose physical parameters (thermal coefficients, stiffness, friction) can vary during operation. Linear Parameter Varying techniques and Gain Scheduling, which interpolate a densely sampled family of local controllers and requires careful tuning, or Model Predictive Control, which needs a physics model and online computation are some of the ways to solve this problem. Deep Reinforcement Learning provides a model-free method but can fail in generalizing across wide parameter ranges. We aim to use few reference policies, each near-optimal at one parameter value, to obtain a global control law without explicit model knowledge. Formalized as limited-k specialist-to-generalist problem, with k between three and ten, we propose a mixture of frozen experts: the references are held fixed and only a lightweight gating network is trained with PPO, mapping the state and task parameter to a softmax weighting over the reference actions, with an optional correction term. We validate on four benchmarks: a bounded-actuation regulation task, the Van der Pol oscillator across a non-stiff to stiff transition, boundary control of a reaction–diffusion PDE with a two-dimensional parameter, and quadrotor navigation under wind. Over ten seeds, and against both nearest-neighbour selection among the references and PPO baselines that append the parameter to the state, the gated mixture performs best on every benchmark and degrades least when away from the reference policies.