Theoretical Understanding of the Generalization of Neural Oscillators

  • Huang, Zifeng (Leibniz University Hannover)
  • Beer, Michael (Leibniz University Hannover)

Please login to view abstract download link

In this study, we derive two upper probably approximately correct (PAC) generalization bounds for a neural oscillator model composed of a second-order ordinary differential equation (ODE) followed with a multilayer perceptron (MLP). The first bound addresses the approximation of causal and uniformly continuous operators between continuous temporal function spaces, while the second pertains to the approximation of uniformly asymptotically incrementally stable second-order dynamical systems. Theoretical analysis shows that the estimation errors in both bounds grow polynomially with respect to the MLP size and time horizon, thereby avoiding the curse of parametric complexity. Furthermore, the derived error bounds reveal that regularizing the Lipschitz constants of the MLP via constraints on matrix and vector norms can enhance the generalization capability of the neural oscillator. A numerical study based on a Bouc-Wen nonlinear system under stochastic seismic excitation verifies the power-law behaviors of the estimation errors with respect to the sample size and time length, and demonstrates the effectiveness of Lipschitz-based regularization in improving the generalization of the neural oscillator under limited training data. Additionally, the numerical results show that the neural oscillator can directly learn the nonsmooth mapping from the seismic excitation to the extreme value response process, allowing the estimation of the extreme value distribution.