Where the LLM Ends and Reliable Decisions Begin
Abstract
Systems that turn natural-language descriptions of optimization problems into solver-ready code generally use a language model at every stage, including the final translation from a mathematical formulation into executable model-building code. We propose the ANVIL compiler architecture, where we separate these concerns. A language model is called only once, assisted by constraint guidance based on problem type, to produce a LaTeX formulation. A deterministic compiler then translates that LaTeX into code with no language model involvement. We describe the deterministic compiler (a normalizer, a recursive-descent parser producing a typed intermediate representation, analysis passes that bind symbols to a dataset schema, and a code emitter) and evaluate it on the 354 easy and hard problems of the NLP4LP benchmark. The compiler produced code for 329 of 354 formulations (92.9%), taking the deterministic path in every one of those cases and never falling back to model-generated code. Median compile time was below the 10ms resolution of our timer. Overall, our formulations achieved an accuracy of 98.9% over easy problems and 91.1% for hard problems. The gap between these compilation and accuracy figures is a key point of analysis, and we analyze it: formulations that failed to compile, problems that returned as infeasible, problems raising errors at runtime, and problems returning a wrong objective. Almost all of these errors trace back to the formulation rather than to the translation. ANVIL performs exceptionally well on leading benchmarks by using language models purely where they are effective, rather than as a catch-all tool.
Related
- Reliable AI-Assisted Decisions via Selective Auditing
- Inequalities for 4-Manifolds with Ends using Fibered boundary metric and Seiberg-Witten Equations
- Number of ends of conjugacy diagrams
- Minimal Surfaces in ℝ4 with Three Planar Ends II: Symmetry and Nonexistence
- Mass-Capacity Rigidity for Asymptotically Flat Manifolds with Arbitrary Ends
BibTeX
@misc{patil2026where,
title = {{Where the LLM Ends and Reliable Decisions Begin}},
author = {Priyadarshan Patil and Abhishek Basu and Vikas Reddy and Abhinav Gupta},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.19545},
eprint = {2609.19545},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.19545},
note = {CC BY-NC-ND 4.0},
}
RIS
TY - UNPB
TI - Where the LLM Ends and Reliable Decisions Begin
AU - Priyadarshan Patil
AU - Abhishek Basu
AU - Vikas Reddy
AU - Abhinav Gupta
PY - 2026
DA - 2026/09/17
PB - arXiv
DO - 10.48550/arxiv.2609.19545
UR - https://arxiv.org/abs/2609.19545
AB - Systems that turn natural-language descriptions of optimization problems into solver-ready code generally use a language model at every stage, including the final translation from a mathematical formulation into executable model-building code. We propose the ANVIL compiler architecture, where we separate these concerns. A language model is called only once, assisted by constraint guidance based on problem type, to produce a LaTeX formulation. A deterministic compiler then translates that LaTeX into code with no language model involvement. We describe the deterministic compiler (a normalizer, a recursive-descent parser producing a typed intermediate representation, analysis passes that bind symbols to a dataset schema, and a code emitter) and evaluate it on the 354 easy and hard problems of the NLP4LP benchmark. The compiler produced code for 329 of 354 formulations (92.9%), taking the deterministic path in every one of those cases and never falling back to model-generated code. Median compile time was below the 10ms resolution of our timer. Overall, our formulations achieved an accuracy of 98.9% over easy problems and 91.1% for hard problems. The gap between these compilation and accuracy figures is a key point of analysis, and we analyze it: formulations that failed to compile, problems that returned as infeasible, problems raising errors at runtime, and problems returning a wrong objective. Almost all of these errors trace back to the formulation rather than to the translation. ANVIL performs exceptionally well on leading benchmarks by using language models purely where they are effective, rather than as a catch-all tool.
ER -