Beyond Verified Answers: Solver-Informed Self-Distillation for Bootstrapping Operations Research Language Models
Abstract
Modern large language models (LLMs) can translate natural-language descriptions into operations research (OR) formulations. Post-training techniques including reinforcement learning and on-policy self-distillation have further improved this capability. However, three limitations remain in training LLMs for OR formulations. First, training commonly relies on synthetic formulations validated by human experts or stronger models, constraining scalable supervision. Second, credit assignment is either coarse or costly: outcome rewards score an entire trajectory without locating the responsible modeling decision, whereas process-level supervision requires an additional evaluator. Third, privileged self-distillation can induce style mismatch by using solver context unavailable at deployment. We find that a model can improve from solver-artifact feedback generated by its own rollouts, making self-distillation a practical, evaluator-free source of dense supervision. Therefore, we propose SOLID: Solver-Informed On-Policy LearnIng through Self-Distillation, a novel framework for self-improving OR language models without verified answers or external evaluators. SOLID executes candidate programs from multiple rollouts, clusters their objectives, and selects a majority-group artifact as a pseudo-reference. The model then performs updates using group-relative advantages and dense self-supervision signals. Across multiple OR benchmarks, SOLID improves solution accuracy for both general-purpose and OR-tuned models over outcome-only group-relative training. These results show that solver artifacts can support scalable self-improvement without trusted answers.
Related
- Cognitive Warfare, Hybrid Pressure, and Sovereign Resilience: An Operations Research Framework Applied to the Iranian Case (2017--2026)
- The First Step of Decision Making: Alignment under Semantic Uncertainty for Automated Operations Research
- Exact Error Bootstrapping and Comb Bootstrapping in FHEW/TFHE
- Bootstrapping autoregressive duration models
- Bootstrapping Mutual Attestation with Kleene's Second Recursion Theorem
BibTeX
@misc{zhu2026beyond,
title = {{Beyond Verified Answers: Solver-Informed Self-Distillation for Bootstrapping Operations Research Language Models}},
author = {Rui Zhu and Minglong Cao and Chenyu Zhou and Jianghao Lin and Dongdong Ge},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.09957},
eprint = {2609.09957},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.09957},
note = {CC BY-SA 4.0},
}
RIS
TY - UNPB
TI - Beyond Verified Answers: Solver-Informed Self-Distillation for Bootstrapping Operations Research Language Models
AU - Rui Zhu
AU - Minglong Cao
AU - Chenyu Zhou
AU - Jianghao Lin
AU - Dongdong Ge
PY - 2026
DA - 2026/09/09
PB - arXiv
DO - 10.48550/arxiv.2609.09957
UR - https://arxiv.org/abs/2609.09957
AB - Modern large language models (LLMs) can translate natural-language descriptions into operations research (OR) formulations. Post-training techniques including reinforcement learning and on-policy self-distillation have further improved this capability. However, three limitations remain in training LLMs for OR formulations. First, training commonly relies on synthetic formulations validated by human experts or stronger models, constraining scalable supervision. Second, credit assignment is either coarse or costly: outcome rewards score an entire trajectory without locating the responsible modeling decision, whereas process-level supervision requires an additional evaluator. Third, privileged self-distillation can induce style mismatch by using solver context unavailable at deployment. We find that a model can improve from solver-artifact feedback generated by its own rollouts, making self-distillation a practical, evaluator-free source of dense supervision. Therefore, we propose SOLID: Solver-Informed On-Policy LearnIng through Self-Distillation, a novel framework for self-improving OR language models without verified answers or external evaluators. SOLID executes candidate programs from multiple rollouts, clusters their objectives, and selects a majority-group artifact as a pseudo-reference. The model then performs updates using group-relative advantages and dense self-supervision signals. Across multiple OR benchmarks, SOLID improves solution accuracy for both general-purpose and OR-tuned models over outcome-only group-relative training. These results show that solver artifacts can support scalable self-improvement without trusted answers.
ER -