Think Short, Defer Smart, Act, and Repeat: Calibrated Reasoning and Uncertainty-Aware Deferral for Edge LLM Agents
Abstract
LLM agents following the ReAct paradigm are promising enablers of complex multi-step tasks, including multi-hop question answering, code generation, and control of physical AI systems. Yet, when deployed at the edge, they must tightly manage their reasoning budget while remaining reliable and deferring to a cloud-side model only when local uncertainty is too high to act safely. We propose Think Short, Defer Smart (TSDS), a framework that synergistically integrates a lightweight convergence probe, which halts on-device reasoning once the intended action has stabilized, with a perplexity-based deferral rule that escalates uncertain actions to a cloud-side model. Both mechanisms are jointly calibrated on end-to-end episode trajectories via a multi-objective Learn-Then-Test (LTT) procedure, providing simultaneous finite-sample guarantees on expected episode reward and cloud-call rate. We evaluate TSDS on four ReAct benchmarks spanning arithmetic reasoning (GSM8K), multi-hop question answering (HotpotQA), code generation (MBPP), and multi-step embodied planning (household robot), and compare against thought-calibration-only and calibrated-deferral-only standalone baselines. TSDS reduces per-episode thinking compute by 43%-65% over deferral-only baselines across HotpotQA, MBPP, and the household robot task, while maintaining certified reward and cloud-call rate guarantees.
Related
- Bagged Martingale Posteriors: Calibrated Uncertainty Quantification for Predictive Resampling
- No-Harm Physics-Informed Inverse Learning with Residual-Calibrated Uncertainty
- Calibrated for Which Policy? Conformal Uncertainty for Fixed World Models Under Intervention
- Towards Intelligent Investment Risk Modeling: Probabilistic Reasoning and Adaptive Decision Systems Under Uncertainty
- llmClassificR: Text Classification with Large Language Models, Network Psychometrics, and Calibrated Uncertainty in R
BibTeX
@misc{farzaneh2026think,
title = {{Think Short, Defer Smart, Act, and Repeat: Calibrated Reasoning and Uncertainty-Aware Deferral for Edge LLM Agents}},
author = {Amirmohammad Farzaneh and Osvaldo Simeone},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2607.26865},
eprint = {2607.26865},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2607.26865},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Think Short, Defer Smart, Act, and Repeat: Calibrated Reasoning and Uncertainty-Aware Deferral for Edge LLM Agents
AU - Amirmohammad Farzaneh
AU - Osvaldo Simeone
PY - 2026
DA - 2026/09/25
PB - arXiv
DO - 10.48550/arxiv.2607.26865
UR - https://arxiv.org/abs/2607.26865
AB - LLM agents following the ReAct paradigm are promising enablers of complex multi-step tasks, including multi-hop question answering, code generation, and control of physical AI systems. Yet, when deployed at the edge, they must tightly manage their reasoning budget while remaining reliable and deferring to a cloud-side model only when local uncertainty is too high to act safely. We propose Think Short, Defer Smart (TSDS), a framework that synergistically integrates a lightweight convergence probe, which halts on-device reasoning once the intended action has stabilized, with a perplexity-based deferral rule that escalates uncertain actions to a cloud-side model. Both mechanisms are jointly calibrated on end-to-end episode trajectories via a multi-objective Learn-Then-Test (LTT) procedure, providing simultaneous finite-sample guarantees on expected episode reward and cloud-call rate. We evaluate TSDS on four ReAct benchmarks spanning arithmetic reasoning (GSM8K), multi-hop question answering (HotpotQA), code generation (MBPP), and multi-step embodied planning (household robot), and compare against thought-calibration-only and calibrated-deferral-only standalone baselines. TSDS reduces per-episode thinking compute by 43%-65% over deferral-only baselines across HotpotQA, MBPP, and the household robot task, while maintaining certified reward and cloud-call rate guarantees.
ER -