On the possibilities and limitations of LLM-based multi-agent systems for multi-criteria decision-making in healthcare operations management
Abstract
Multi-criteria models for healthcare operations need weights that express how stakeholders trade off patient waiting time, provider workload, and operating cost, but these weights are difficult to quantify. Teams of large language model (LLM) agents, each prompted to represent a stakeholder, have been used to supply them. We propose a framework for assessing the reliability of LLM-generated preferences as inputs to multi-objective optimization and quantifying how their uncertainty affects decisions. We study explicit preferences by asking agents to rate the importance of the criteria in an outpatient appointment scheduling problem, and implicit preferences by inferring trade-offs from the agents' pairwise choices between emergency department configurations. The computational experiments show that the explicit preferences respond to the clinic's operating conditions and differ across roles, but deliberation among the agents can erase these differences, whereas simple averaging preserves them. The pairwise choices used to reveal the agents' implicit preferences depend largely on which option is presented first, and we find limited evidence that a statistical model using contextual information alongside presentation order predicts the agents' choices on new comparison pairs better than a model using presentation order alone. We then formulate robust weighted-sum optimization models to account for preference uncertainty. In most cases, these models select different schedules or configurations from those obtained using average preference weights. These results indicate that LLM-generated preferences need an audit of this kind before they are used as optimization weights in healthcare decision support.
Related
- Locally unital C*-algebras and limitations of seminorm criteria for compact operators
- An additional possibilities of the standard method of inverting the Radon transform
- Quantitative possibilities for Baxter's theorem in nonlinear Fourier analysis
- Pattern Intelligence: Unsupervised, LLM-Interpreted Discovery of Systematic Underpayment and Denial Patterns in Healthcare Revenue Cycle Management
- Strong isometric path complexity of graphs: Asymptotic minors, restricted holes, and graph operations
BibTeX
@misc{boresta2026possibilities,
title = {{On the possibilities and limitations of LLM-based multi-agent systems for multi-criteria decision-making in healthcare operations management}},
author = {Marco Boresta and Tommaso Giovannelli and Sepehr Namirad and Giulia Riccardi},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.23673},
eprint = {2610.23673},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.23673},
}
RIS
TY - UNPB
TI - On the possibilities and limitations of LLM-based multi-agent systems for multi-criteria decision-making in healthcare operations management
AU - Marco Boresta
AU - Tommaso Giovannelli
AU - Sepehr Namirad
AU - Giulia Riccardi
PY - 2026
DA - 2026/10/16
PB - arXiv
DO - 10.48550/arxiv.2610.23673
UR - https://arxiv.org/abs/2610.23673
AB - Multi-criteria models for healthcare operations need weights that express how stakeholders trade off patient waiting time, provider workload, and operating cost, but these weights are difficult to quantify. Teams of large language model (LLM) agents, each prompted to represent a stakeholder, have been used to supply them. We propose a framework for assessing the reliability of LLM-generated preferences as inputs to multi-objective optimization and quantifying how their uncertainty affects decisions. We study explicit preferences by asking agents to rate the importance of the criteria in an outpatient appointment scheduling problem, and implicit preferences by inferring trade-offs from the agents' pairwise choices between emergency department configurations. The computational experiments show that the explicit preferences respond to the clinic's operating conditions and differ across roles, but deliberation among the agents can erase these differences, whereas simple averaging preserves them. The pairwise choices used to reveal the agents' implicit preferences depend largely on which option is presented first, and we find limited evidence that a statistical model using contextual information alongside presentation order predicts the agents' choices on new comparison pairs better than a model using presentation order alone. We then formulate robust weighted-sum optimization models to account for preference uncertainty. In most cases, these models select different schedules or configurations from those obtained using average preference weights. These results indicate that LLM-generated preferences need an audit of this kind before they are used as optimization weights in healthcare decision support.
ER -