Large-Scale Comparison of Metaheuristic Algorithms via Behavioral Verification of Large Language Model-Generated Codes
- 1Department of Information Technology, Faculty of Computers and Information Technology, University of Tabuk, Tabuk 71491, Saudi Arabia
Abstract
The performance and reproducibility of metaheuristic algorithms remain unresolved because prior studies have examined only limited subsets, as most algorithms are published as mathematical descriptions without source code. A large language model (LLM) was used to implement 393 published metaheuristic algorithms. The LLM-generated codes were verified by running them on two optimization problems with three seeds while an automated script checked for crashes and improvement. These reports were returned to the LLM as repair feedback. First drafts passed at a rate of 53.4%, which increased to 78.4% after eight rounds of feedback, while 32 algorithms failed in all three replicates, 24 of them with the same failure signature. The common practice of using LLMs to review LLM-generated code was evaluated: at least one of two independent LLM reviewers approved 92.3% of broken codes, indicating that textual review provides no information about execution-level validity. Fifty verified codes were compared to reference codes from an established library: 49 differed significantly in both directions, and only the most completely specified algorithm showed no detected divergence. The 360 verified codes were compared on the Congress on Evolutionary Computation (CEC) 2017 and CEC 2022 benchmarks with 51 seeded runs. The Differential Evolution implementation ranked second on CEC 2017 and first on CEC 2022, outperforming the evaluated implementations of hundreds of recent metaphor-based algorithms.
Related
- Performance Evaluation of Metaheuristic Algorithms for Fractional-Order PID Controller Tuning in Non-Linear Systems
- Soft Computing Optimization of Industrial Robot Inverse Kinematics for Intelligent Manufacturing: A Statistical Benchmark of Metaheuristic Algorithms
- Design-Based Validation Method of EHR Algorithms to Address Partial Verification Bias
- Recursive Multilevel Decomposition of Pairwise Comparison Matrices: Algorithms and Inconsistency Diagnostics
- Comparison of Multisine Peak Factor Minimization Algorithms for Aircraft System Identification
BibTeX
@article{alharbi2026largescale,
title = {{Large-Scale Comparison of Metaheuristic Algorithms via Behavioral Verification of Large Language Model-Generated Codes}},
author = {Emad Alharbi},
journal = {Mathematics},
year = {2026},
month = oct,
doi = {10.3390/math14213976},
url = {https://www.mdpi.com/2227-7390/14/21/3976},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - Large-Scale Comparison of Metaheuristic Algorithms via Behavioral Verification of Large Language Model-Generated Codes
AU - Emad Alharbi
PY - 2026
DA - 2026/10/31
JO - Mathematics
DO - 10.3390/math14213976
UR - https://www.mdpi.com/2227-7390/14/21/3976
AB - The performance and reproducibility of metaheuristic algorithms remain unresolved because prior studies have examined only limited subsets, as most algorithms are published as mathematical descriptions without source code. A large language model (LLM) was used to implement 393 published metaheuristic algorithms. The LLM-generated codes were verified by running them on two optimization problems with three seeds while an automated script checked for crashes and improvement. These reports were returned to the LLM as repair feedback. First drafts passed at a rate of 53.4%, which increased to 78.4% after eight rounds of feedback, while 32 algorithms failed in all three replicates, 24 of them with the same failure signature. The common practice of using LLMs to review LLM-generated code was evaluated: at least one of two independent LLM reviewers approved 92.3% of broken codes, indicating that textual review provides no information about execution-level validity. Fifty verified codes were compared to reference codes from an established library: 49 differed significantly in both directions, and only the most completely specified algorithm showed no detected divergence. The 360 verified codes were compared on the Congress on Evolutionary Computation (CEC) 2017 and CEC 2022 benchmarks with 51 seeded runs. The Differential Evolution implementation ranked second on CEC 2017 and first on CEC 2022, outperforming the evaluated implementations of hundreds of recent metaphor-based algorithms.
SN - 2227-7390
ER -