What Limits Lithium-Ion Battery State of Charge Estimation with a Hybrid Large Language Model? Layer Type Aware Adaptation and the Initial Condition Bottleneck
- 1ReCharge Lab, Moscow Center for Advanced Studies, 123592 Moscow, Russia
Abstract
State of charge (SoC) is the unobserved quantity that a battery management system must reconstruct from terminal voltage, current and temperature. Recent estimators adapt frozen large language models by prompt-tuning the key–value cache of a dense decoder, an interface that does not exist in the linear-attention majority of current hybrid models. We adapt a frozen four-billion-parameter hybrid decoder by matching the injection to each layer type: learned key–value prefixes in the full-attention minority, and a trainable initial recurrent state (S0) in the gated-delta majority. Training only 22.3 million parameters (0.55%), this layer-type-aware adaptation reduces mean absolute error (MAE) by a factor of 2.3 relative to prefixes alone on a preliminary single-cell diagnostic. On the final four-billion-parameter configuration, the same comparison yields a factor of 1.86: pooled mean absolute error decreases from 2.18% for key–value prefixes alone to 1.17% for the full adapter. The nickel-based drive-cycle pool is 1.17% MAE on 18,712 windows. That pool does not include LFP. The LFP value on shared files is a fit of 1.22%. Held-out files of the same cell reach 2.53% in a separate run, and the nickel-based pool of that run is 1.24%. This represents parity with dedicated dense-LLM estimators without claiming superiority over compact conventional observers. On the same drive-cycle split, a gated recurrent unit with 0.2 million parameters reaches 1.18% pooled MAE, and the adapter reaches 1.17%. Instead, the primary contribution is mechanistic. We demonstrate that 98.4% of the residual error is a constant per-window offset on that diagnostic. On this configuration, a constant per-window offset accounts for 0.98 of the residual energy. The estimate therefore behaves as a near-exact Coulomb integration after an initial-condition inverse problem at t=0. Compact models trained on the same windows show the same offset-dominated residual, so this pattern is a property of the short-window task as well as of the backbone. This factorization is consistent with the observed failure regimes—concentration of error below 0 °C, and a zero-shot LFP error of 12% MAE on the reference range of about 0–47% state of charge, about ten times the nickel-based pool. Held-out files of an LFP cell partly seen in training reach 2.53% in a separate run. Consequently, we argue that the engineering target for LLM-based battery estimation should transition from expanding backbone capacity to explicitly conditioning the initial state.
Related
- Critical Evaluation of Electrochemical Impedance Spectroscopy for State-of-Health Estimation of Lithium-Ion Cells Against Battery Management System Baselines
- RxnScout: Deploying a Vision-Language Agent for Nickel-Catalyzed C-Glycosylation Condition Optimization
- Multi-Stage State-of-Health Estimation for Lithium-Ion Batteries Based on an Adaptive SHAP-Guided Dual-Attention LSTM
- Real Electric Vehicle Lithium-Ion Battery Capacity Estimation Method Based on the Fusion Model of Discrete Arrhenius and Double Extended Kalman Filter
- State of Charge Estimation for Lithium-Ion Batteries Under Wide Temperature Ranges Using Error Feedforward of Sparrow-Optimized Back Propagation Neural Network
BibTeX
@article{druzhinin2026what,
title = {{What Limits Lithium-Ion Battery State of Charge Estimation with a Hybrid Large Language Model? Layer Type Aware Adaptation and the Initial Condition Bottleneck}},
author = {Alexey A. Druzhinin},
journal = {Batteries},
year = {2026},
month = oct,
doi = {10.3390/batteries12100427},
url = {https://www.mdpi.com/2313-0105/12/10/427},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - What Limits Lithium-Ion Battery State of Charge Estimation with a Hybrid Large Language Model? Layer Type Aware Adaptation and the Initial Condition Bottleneck
AU - Alexey A. Druzhinin
PY - 2026
DA - 2026/10/15
JO - Batteries
DO - 10.3390/batteries12100427
UR - https://www.mdpi.com/2313-0105/12/10/427
AB - State of charge (SoC) is the unobserved quantity that a battery management system must reconstruct from terminal voltage, current and temperature. Recent estimators adapt frozen large language models by prompt-tuning the key–value cache of a dense decoder, an interface that does not exist in the linear-attention majority of current hybrid models. We adapt a frozen four-billion-parameter hybrid decoder by matching the injection to each layer type: learned key–value prefixes in the full-attention minority, and a trainable initial recurrent state (S0) in the gated-delta majority. Training only 22.3 million parameters (0.55%), this layer-type-aware adaptation reduces mean absolute error (MAE) by a factor of 2.3 relative to prefixes alone on a preliminary single-cell diagnostic. On the final four-billion-parameter configuration, the same comparison yields a factor of 1.86: pooled mean absolute error decreases from 2.18% for key–value prefixes alone to 1.17% for the full adapter. The nickel-based drive-cycle pool is 1.17% MAE on 18,712 windows. That pool does not include LFP. The LFP value on shared files is a fit of 1.22%. Held-out files of the same cell reach 2.53% in a separate run, and the nickel-based pool of that run is 1.24%. This represents parity with dedicated dense-LLM estimators without claiming superiority over compact conventional observers. On the same drive-cycle split, a gated recurrent unit with 0.2 million parameters reaches 1.18% pooled MAE, and the adapter reaches 1.17%. Instead, the primary contribution is mechanistic. We demonstrate that 98.4% of the residual error is a constant per-window offset on that diagnostic. On this configuration, a constant per-window offset accounts for 0.98 of the residual energy. The estimate therefore behaves as a near-exact Coulomb integration after an initial-condition inverse problem at t=0. Compact models trained on the same windows show the same offset-dominated residual, so this pattern is a property of the short-window task as well as of the backbone. This factorization is consistent with the observed failure regimes—concentration of error below 0 °C, and a zero-shot LFP error of 12% MAE on the reference range of about 0–47% state of charge, about ten times the nickel-based pool. Held-out files of an LFP cell partly seen in training reach 2.53% in a separate run. Consequently, we argue that the engineering target for LLM-based battery estimation should transition from expanding backbone capacity to explicitly conditioning the initial state.
SN - 2313-0105
ER -