WCCM ECCOMAS 2026

Majorization Minimization for Neural Network Training

Baraldi, Robert J (Sandia National Laboratories)
Javeed, Aurya (Sandia National Laboratories)
Kouri, Drew P (Sandia National Laboratories)

In session: MS293A - Multilevel, Multiscale, and Hierarchical Machine Learning Methods for Scientific Machine Learning I

Please login to view abstract download link

We extend a majorization minimization method for training neural networks with piecewise affine activations. The method is provably convergent. It relaxes the multicomposite structure of neural networks by lifting the training problem to a higher-dimensional space. We extend the method to a broader class of losses that includes cross-entropy and to a proximal trust region method for the majorization minimization.