Parallel Optimized Particle Swarm Optimization: An Efficient Parallel Approach for Neural Network Training
- 1Department of Informatics and Telecommunications, University of Ioannina, 47150 Kostaki Artas, Greece
Abstract
Global optimization has become a cornerstone of modern machine learning, particularly for the challenging task of training high-dimensional neural network architectures. Despite the prevalence of established metaheuristics, traditional approaches frequently struggle with the irregular and multimodal error surfaces inherent in complex training landscapes. This paper proposes the parallel optimized particle swarm optimization (Parallel OPSO), a multi-population parallel framework specifically engineered to enhance the training efficiency and predictive accuracy of neural networks across classification and regression tasks. At its core, OPSO integrates several advanced mechanisms: a simulated annealing-based acceptance criterion to escape local minima, adaptive inertia weights that dynamically respond to stagnation by injecting momentum, and a hybrid Broyden–Fletcher–Goldfarb–Shanno (BFGS) local search phase for fine-grained convergence. While OPSO defines this underlying algorithmic model, Parallel OPSO represents its execution across parallel multi-population architectures. By partitioning the swarm into interacting subpopulations and leveraging modern multi-core architectures, Parallel OPSO significantly reduces training time, with prominent performance gains primarily manifesting on highly challenging and multimodal optimization tasks. Experimental evaluations demonstrate that while standard algorithms may perform adequately on simpler loss landscapes, Parallel OPSO achieves superior convergence robustness and accuracy when navigating demanding and highly irregular error surfaces. These results underscore Parallel OPSO as a highly efficient and scalable framework for optimizing neural network parameters in complex search spaces.
Related
- Parallel Optimization: Theory, Algorithms, and Applications,
- Query-Efficient Rejection Sampling for Parallel Masked Diffusion
- Randomized Parallel Complexity of Weakly Convex Optimization
- Improved Regret Analysis for Parallel Gaussian Process Bandit Optimization
- A Parallel Adaptive Proximal ADMM for Linearly Constrained Separable Composite Optimization Problems
BibTeX
@article{barkas2026parallel,
title = {{Parallel Optimized Particle Swarm Optimization: An Efficient Parallel Approach for Neural Network Training}},
author = {Konstantinos G. Barkas and Ioannis G. Tsoulos and Vasileios Charilogis and Glykeria Kyrou},
journal = {Mathematics},
year = {2026},
month = oct,
doi = {10.3390/math14193599},
url = {https://www.mdpi.com/2227-7390/14/19/3599},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - Parallel Optimized Particle Swarm Optimization: An Efficient Parallel Approach for Neural Network Training
AU - Konstantinos G. Barkas
AU - Ioannis G. Tsoulos
AU - Vasileios Charilogis
AU - Glykeria Kyrou
PY - 2026
DA - 2026/10/03
JO - Mathematics
DO - 10.3390/math14193599
UR - https://www.mdpi.com/2227-7390/14/19/3599
AB - Global optimization has become a cornerstone of modern machine learning, particularly for the challenging task of training high-dimensional neural network architectures. Despite the prevalence of established metaheuristics, traditional approaches frequently struggle with the irregular and multimodal error surfaces inherent in complex training landscapes. This paper proposes the parallel optimized particle swarm optimization (Parallel OPSO), a multi-population parallel framework specifically engineered to enhance the training efficiency and predictive accuracy of neural networks across classification and regression tasks. At its core, OPSO integrates several advanced mechanisms: a simulated annealing-based acceptance criterion to escape local minima, adaptive inertia weights that dynamically respond to stagnation by injecting momentum, and a hybrid Broyden–Fletcher–Goldfarb–Shanno (BFGS) local search phase for fine-grained convergence. While OPSO defines this underlying algorithmic model, Parallel OPSO represents its execution across parallel multi-population architectures. By partitioning the swarm into interacting subpopulations and leveraging modern multi-core architectures, Parallel OPSO significantly reduces training time, with prominent performance gains primarily manifesting on highly challenging and multimodal optimization tasks. Experimental evaluations demonstrate that while standard algorithms may perform adequately on simpler loss landscapes, Parallel OPSO achieves superior convergence robustness and accuracy when navigating demanding and highly irregular error surfaces. These results underscore Parallel OPSO as a highly efficient and scalable framework for optimizing neural network parameters in complex search spaces.
SN - 2227-7390
ER -