Information-Geometric Selection for Validation-Guided Ensembles of Extreme Learning Machines: The LEA-EELM Framework
- 1Department of Computer Engineering, Ma.C., Islamic Azad University, Mashhad 9187147578, Iran
- 2Department of Artificial Intelligence and Data Science, Intelligent Financial Innovation Research Center, Ma.C., Islamic Azad University, Mashhad 9187147578, Iran
- 3Applied Data Science and Artificial Intelligence, SRH University Heidelberg, 69123 Heidelberg, Germany
Abstract
Extreme learning machines (ELMs) solve output weights in closed form, so an optimized population of ELMs leaves open which members should form the ensemble. We report a controlled comparison of five rules for this selection step inLEA-EELM, where a Lotus Effect Algorithm (LEA) variant generates fuzzy-activation ELM candidates and M=10 members are chosen from a shared top-2M validation shortlist. Two rules are derived from the Fisher information I(β)=H⊤H/σ2 of a candidate’s linear-Gaussian output stage. Because H⊤H is only positive semidefinite and candidates have different hidden layers, F-complementary compares candidates by the Kullback–Leibler divergence between their predictive distributions, not by a Fisher–Rao distance. For F-norm, we prove that, with least-squares output weights, ∥Hβ∥2=∥T∥2−∥T−Hβ∥2, so the rule selects the worst training fits; it is significantly worse than β-norm on three of four regression benchmarks. F-complementary performs like greedy validation selection and is never significantly better. The information-geometric rules therefore add no measured benefit; the gains come from validation-guided greedy selection, which lowers test RMSE on three of four regression datasets and raises accuracy only on Satellite and Letter, but not on Letter after duplicate rows are removed. All findings are conditional on the candidate pool of a non-canonical (subtractive-sign) LEA variant.
Related
- Information-Geometric Grammar Principle: A Statistical-Geometric Framework for Adaptive Systems
- Evaluation of Panel and Site Selection for a Solar Power Plant Using Geographic Information Systems and an Integrated Multi-Criteria Decision-Making Framework
- An Information-Theoretic Principle for Optimal Quantum Encoding: Tight Frames and Equiangular Ensembles
- Maximum Entropy Probability Distributions on Spheres with Fixed Mean Busemann Function and Holomorphic-Information-Geometric Model of Cognition
- Information-Geometric Blind Spots of Conformal Test Martingales
BibTeX
@article{sabzevari2026informationgeometric,
title = {{Information-Geometric Selection for Validation-Guided Ensembles of Extreme Learning Machines: The LEA-EELM Framework}},
author = {Amir Abbas Sabzevari and Hamid Tabatabaee and Mehrdad Jalali},
journal = {Mathematics},
year = {2026},
month = nov,
doi = {10.3390/math14213995},
url = {https://www.mdpi.com/2227-7390/14/21/3995},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - Information-Geometric Selection for Validation-Guided Ensembles of Extreme Learning Machines: The LEA-EELM Framework
AU - Amir Abbas Sabzevari
AU - Hamid Tabatabaee
AU - Mehrdad Jalali
PY - 2026
DA - 2026/11/02
JO - Mathematics
DO - 10.3390/math14213995
UR - https://www.mdpi.com/2227-7390/14/21/3995
AB - Extreme learning machines (ELMs) solve output weights in closed form, so an optimized population of ELMs leaves open which members should form the ensemble. We report a controlled comparison of five rules for this selection step inLEA-EELM, where a Lotus Effect Algorithm (LEA) variant generates fuzzy-activation ELM candidates and M=10 members are chosen from a shared top-2M validation shortlist. Two rules are derived from the Fisher information I(β)=H⊤H/σ2 of a candidate’s linear-Gaussian output stage. Because H⊤H is only positive semidefinite and candidates have different hidden layers, F-complementary compares candidates by the Kullback–Leibler divergence between their predictive distributions, not by a Fisher–Rao distance. For F-norm, we prove that, with least-squares output weights, ∥Hβ∥2=∥T∥2−∥T−Hβ∥2, so the rule selects the worst training fits; it is significantly worse than β-norm on three of four regression benchmarks. F-complementary performs like greedy validation selection and is never significantly better. The information-geometric rules therefore add no measured benefit; the gains come from validation-guided greedy selection, which lowers test RMSE on three of four regression datasets and raises accuracy only on Satellite and Letter, but not on Letter after duplicate rows are removed. All findings are conditional on the candidate pool of a non-canonical (subtractive-sign) LEA variant.
SN - 2227-7390
ER -