Control of Parametric Distributed Systems with Hypernetwork-based Reinforcement Learning
Please login to view abstract download link
Reinforcement learning (RL) and multi-agent RL (MARL) have recently emerged as promising control strategies for complex dynamical systems governed by partial differential equations. These control problems, frequently arising in applied sciences and engineering, are typically high-dimensional in state and control variables and often have parametric dependencies, making the learning of the control policies extremely challenging. In this work, we explore the use of specialized neural network architectures suitable for RL and MARL to learn general policies for parametric control problems. In particular, we propose two hypernetwork-based architectures for RL and multi-agent RL that encode parametric dependencies into the policy weights and biases directly. Hypernetworks are special types of neural networks that learn weights and biases of another network. Throughout our numerical experiments, spanning from the stabilization of a chaotic system, the navigation in a gyre flow, to density and flow control, we show that hypernetworks (i) enable effective control of complex dynamical systems, outperforming state-of-the-art RL and MARL algorithms, (ii) can efficiently deal with parametric dependencies, and (iii) require minimal hyperparameter tuning.
