The Functional Structure of Post-Compression Recovery in Low-Rank LLMs
Abstract
Different low-rank compression methods can produce compressed LLMs that respond differently to the same post-compression recovery procedure, and relative advantages observed between methods at the compression endpoint may shrink, grow, or even reverse after recovery. We ask whether this recovery heterogeneity reflects functional structure beyond scalar loss evolution, and how that structure evolves throughout recovery. Our results establish that this heterogeneity reflects a reproducible compression-induced functional structure, which we formalize as recovery pressure. To characterize this structure consistently throughout recovery, we develop a standardized functional characterization within each backbone that is applicable across heterogeneous low-rank methods. The primary backward characterization reveals reproducible module-wise structure across independent probes, while a complementary forward-only characterization recovers related structure without loss or backpropagation. We further find that recovery pressure measured at the endpoint is associated with subsequent recovery response; during recovery, its module-wise structure is reorganized non-uniformly, and localized updates induce distributed responses beyond directly updated modules. Further evidence indicates that tracking this evolving structure provides a complementary functional view of recovery progress alongside scalar loss.
Related
- Singleton-Optimal Rank-Metric CSS Codes : Equality Structure and Exact Projected-Recovery Radii
- The reverse Ritz inclusion: computable bounds on the pseudospectra of an operator by those of a finite-rank compression
- Optimal low-rank compression of quantum dynamics
- Hierarchical low-rank compression exploiting symmetric boundary integral equations in transmission scattering
- Accelerated S-NFC for Million-Chaff RCS Computation Using Low-Rank Compression of Concatenated Block Rows
BibTeX
@misc{shao2026functional,
title = {{The Functional Structure of Post-Compression Recovery in Low-Rank LLMs}},
author = {Zishan Shao and Liang Tian and Georgiy Zemlevskiy and Kangning Cui and Lixun Zhang and Yixiao Wang and Ting Jiang and Jinhee Kim and Yixuan Chen and Rui-Feng Wang and Fan Yang and Hai Li and Yiran Chen},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.05504},
eprint = {2610.05504},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.05504},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - The Functional Structure of Post-Compression Recovery in Low-Rank LLMs
AU - Zishan Shao
AU - Liang Tian
AU - Georgiy Zemlevskiy
AU - Kangning Cui
AU - Lixun Zhang
AU - Yixiao Wang
AU - Ting Jiang
AU - Jinhee Kim
AU - Yixuan Chen
AU - Rui-Feng Wang
AU - Fan Yang
AU - Hai Li
AU - Yiran Chen
PY - 2026
DA - 2026/10/04
PB - arXiv
DO - 10.48550/arxiv.2610.05504
UR - https://arxiv.org/abs/2610.05504
AB - Different low-rank compression methods can produce compressed LLMs that respond differently to the same post-compression recovery procedure, and relative advantages observed between methods at the compression endpoint may shrink, grow, or even reverse after recovery. We ask whether this recovery heterogeneity reflects functional structure beyond scalar loss evolution, and how that structure evolves throughout recovery. Our results establish that this heterogeneity reflects a reproducible compression-induced functional structure, which we formalize as recovery pressure. To characterize this structure consistently throughout recovery, we develop a standardized functional characterization within each backbone that is applicable across heterogeneous low-rank methods. The primary backward characterization reveals reproducible module-wise structure across independent probes, while a complementary forward-only characterization recovers related structure without loss or backpropagation. We further find that recovery pressure measured at the endpoint is associated with subsequent recovery response; during recovery, its module-wise structure is reorganized non-uniformly, and localized updates induce distributed responses beyond directly updated modules. Further evidence indicates that tracking this evolving structure provides a complementary functional view of recovery progress alongside scalar loss.
ER -