SPGD: Steepest Perturbed Gradient Descent Optimization
Please login to view abstract download link
Optimization algorithms are central to progress in many scientific and industrial domains, yet they frequently face challenges such as entrapment in local minima, saddle points, and plateaus (flat regions), which significantly hinder convergence to near-optimal solutions. We present Steepest Perturbed Gradient Descent (SPGD), a novel optimization algorithm with proven convergence guarantees that combines classical gradient descent with periodic uniform perturbation sampling to effectively overcome these challenges. SPGD is uniquely designed to generate a set of candidate solutions and select the one that yields the steepest decrease in the objective function relative to the current iterate. By incorporating a structured exploration mechanism, SPGD enhances traditional gradient descent and substantially improves the ability to escape suboptimal local minima and traverse complex optimization landscapes. The proposed approach preserves the directional efficiency of gradient descent while exploiting the exploratory advantages of stochastic perturbations, enabling a more thorough search for global optima across diverse problem settings. We demonstrate the effectiveness of SPGD on the 3D component packing problem, an NP-hard optimization task. Preliminary results indicate significant performance improvements over six established methods, particularly on highly nonconvex response surfaces and in multidimensional continuous optimization problems. Furthermore, comparative evaluations on standard 2D benchmark functions using 30 randomized initializations highlight SPGD’s robustness and reliability in nonconvex optimization.
