Ask Before You Optimize: Dynamic Pre-Formulation Clarification for Interactive Optimization
Abstract
Large language models (LLMs) are increasingly used to formulate optimization models from natural-language problem descriptions, yet realistic operations research (OR) requests are often incomplete: missing objectives, constraints, or business rules can change the resulting mathematical program. Existing evaluations largely assume a complete specification and therefore overlook whether an agent knows when clarification is needed before modeling. We introduce OR-Clarify, a benchmark for pre-formulation clarification. Each task presents a partial public problem description, withholds structured hidden slots, and evaluates agents through bounded interaction with a simulated user. The benchmark supports both openended and choice-based clarification, and measures slot recovery, stopping behavior, silent assumptions, and interaction cost. We further propose Interactive Optimization (InterOPT), a two-stage framework that identifies unresolved formulation-critical gaps and uses them to guide whether to ask the next question or to stop. In our choice-based experiments, InterOPT substantially outperforms all baselines in exact slot recovery; in the open-ended setting, it remains competitive with strong prior methods. Together, OR-Clarify and InterOPT reframe OR assistance as a selective completeness decision: clarify when needed, stop when ready, and quantify what remains missing.
Related
- ParetoChat: An LLM-Based Interface for Interactive Multi-Objective Optimization
- Fundamental Limits of Dynamic Tracking and Communication: A Stationary Policy Formulation
- A Theoretical Analysis of a Bilevel Optimization Formulation for Combined Local-Global Optical Flow Models
- Generic Machine Learning-Based Sparsification for Arc Flow Formulation of Combinatorial Optimization Problems
- A clarification of Fermi's formula for the maximum of partial sums
BibTeX
@misc{ge2026before,
title = {{Ask Before You Optimize: Dynamic Pre-Formulation Clarification for Interactive Optimization}},
author = {Sihan Ge and Yichen Lin and Chenyu Zhou and Jianghao Lin and Tao Yao and Dongdong Ge},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.05258},
eprint = {2609.05258},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.05258},
}
RIS
TY - UNPB
TI - Ask Before You Optimize: Dynamic Pre-Formulation Clarification for Interactive Optimization
AU - Sihan Ge
AU - Yichen Lin
AU - Chenyu Zhou
AU - Jianghao Lin
AU - Tao Yao
AU - Dongdong Ge
PY - 2026
DA - 2026/09/08
PB - arXiv
DO - 10.48550/arxiv.2609.05258
UR - https://arxiv.org/abs/2609.05258
AB - Large language models (LLMs) are increasingly used to formulate optimization models from natural-language problem descriptions, yet realistic operations research (OR) requests are often incomplete: missing objectives, constraints, or business rules can change the resulting mathematical program. Existing evaluations largely assume a complete specification and therefore overlook whether an agent knows when clarification is needed before modeling. We introduce OR-Clarify, a benchmark for pre-formulation clarification. Each task presents a partial public problem description, withholds structured hidden slots, and evaluates agents through bounded interaction with a simulated user. The benchmark supports both openended and choice-based clarification, and measures slot recovery, stopping behavior, silent assumptions, and interaction cost. We further propose Interactive Optimization (InterOPT), a two-stage framework that identifies unresolved formulation-critical gaps and uses them to guide whether to ask the next question or to stop. In our choice-based experiments, InterOPT substantially outperforms all baselines in exact slot recovery; in the open-ended setting, it remains competitive with strong prior methods. Together, OR-Clarify and InterOPT reframe OR assistance as a selective completeness decision: clarify when needed, stop when ready, and quantify what remains missing.
ER -