An Explainable Multi-Criteria Decision-Making Framework for Evaluating Malware Detection Models Across Heterogeneous Datasets
4 affiliations
- 1Department of Cybersecurity Engineering, College of Engineering, Al-Karkh University of Science, Baghdad 10081, Iraq
- 2Cybersecurity Department, College of Computer Science and Information Technology, Wasit University, Al Kut 52001, Iraq
- 3Medical Instruments Techniques Engineering Department, Technical College of Engineering, Al-Bayan University, Baghdad 10001, Iraq
- 4Cybersecurity Research Centre, Universiti Sains Malaysia, Gelugor 11800, Penang, Malaysia
Abstract
The diversity of today’s malware and the conflicting criteria for predictive performance, computational efficiency, dependability, and interpretability have made the choice of a suitable malware detection model more complicated. Existing research focuses predominantly on predictive performance, while the multidimensional decision process required for practical model selection remains insufficiently addressed. To address this gap, this study provides an explainability-aware hybrid multi-criteria decision-making (MCDM) framework that systematically evaluates and ranks malware detection models across heterogeneous malware datasets. The methodology incorporates predictive performance, computational efficiency, false positive rate, and a composite Explainability Index into a single decision procedure. The Explainability Index integrates explanation stability, sparsity, and expert relevance, enabling interpretability to be explicitly considered in the model-selection process. The hybrid criteria weights are obtained by combining the Analytic Hierarchy Process (AHP) with the entropy weighting method, thereby integrating expert-driven criterion importance with data-driven variability. The final ranking is obtained using the Technique for Order Preference by Similarity to the Ideal Solution (TOPSIS). Four candidate detection models, including Random Forest, XGBoost, Convolutional Neural Network, and Long Short-Term Memory, are independently used to validate the framework on the CIC-MalMem-2022 and CICMalDroid2020 datasets. Under a harmonized evaluation protocol without assuming direct cross-dataset predictive transfer, the experimental results rank XGBoost first with a TOPSIS closeness score of 0.670, followed by Random Forest with 0.624. Sensitivity and ablation analyses further show that the model ranking remains stable while changes in criterion weighting and framework components produce measurable variations in the multi-criteria preference structure. Overall, the framework provides a transparent and multidimensional alternative to conventional performance-centered malware model evaluation.
Related
- Evaluating IMMPACT Criteria for the Clinical Meaningfulness of Between-Group Differences: Interpreting Randomized Trial Results within a Context of Existing Chronic Pain Analgesics.
- Evaluating multi-season occupancy models with autocorrelation fitted to heterogeneous datasets
- Heterogeneous survivor average causal effects beyond monotonicity: Applications to a clinical trial evaluating mechanical ventilation strategies
- Explainable Outlier Detection for Multivariate Functional Data
- Amortized Bayesian Disease Mapping and Boundary Detection on Heterogeneous Spatial Graphs
BibTeX
@article{mohammed2026explainable,
title = {{An Explainable Multi-Criteria Decision-Making Framework for Evaluating Malware Detection Models Across Heterogeneous Datasets}},
author = {Husam Jasim Mohammed and Riyadh Rahef Nuiaa Alogaili and Mohanad Sameer Jabbar and Selvakumar Manickam},
journal = {Mathematical and Computational Applications},
year = {2026},
month = sep,
doi = {10.3390/mca31050184},
url = {https://doi.org/10.3390/mca31050184},
note = {CC BY},
}
RIS
TY - JOUR
TI - An Explainable Multi-Criteria Decision-Making Framework for Evaluating Malware Detection Models Across Heterogeneous Datasets
AU - Husam Jasim Mohammed
AU - Riyadh Rahef Nuiaa Alogaili
AU - Mohanad Sameer Jabbar
AU - Selvakumar Manickam
PY - 2026
DA - 2026/09/01
JO - Mathematical and Computational Applications
DO - 10.3390/mca31050184
UR - https://doi.org/10.3390/mca31050184
AB - The diversity of today’s malware and the conflicting criteria for predictive performance, computational efficiency, dependability, and interpretability have made the choice of a suitable malware detection model more complicated. Existing research focuses predominantly on predictive performance, while the multidimensional decision process required for practical model selection remains insufficiently addressed. To address this gap, this study provides an explainability-aware hybrid multi-criteria decision-making (MCDM) framework that systematically evaluates and ranks malware detection models across heterogeneous malware datasets. The methodology incorporates predictive performance, computational efficiency, false positive rate, and a composite Explainability Index into a single decision procedure. The Explainability Index integrates explanation stability, sparsity, and expert relevance, enabling interpretability to be explicitly considered in the model-selection process. The hybrid criteria weights are obtained by combining the Analytic Hierarchy Process (AHP) with the entropy weighting method, thereby integrating expert-driven criterion importance with data-driven variability. The final ranking is obtained using the Technique for Order Preference by Similarity to the Ideal Solution (TOPSIS). Four candidate detection models, including Random Forest, XGBoost, Convolutional Neural Network, and Long Short-Term Memory, are independently used to validate the framework on the CIC-MalMem-2022 and CICMalDroid2020 datasets. Under a harmonized evaluation protocol without assuming direct cross-dataset predictive transfer, the experimental results rank XGBoost first with a TOPSIS closeness score of 0.670, followed by Random Forest with 0.624. Sensitivity and ablation analyses further show that the model ranking remains stable while changes in criterion weighting and framework components produce measurable variations in the multi-criteria preference structure. Overall, the framework provides a transparent and multidimensional alternative to conventional performance-centered malware model evaluation.
SN - 2297-8747
SN - 1300-686X
ER -