PreprintarXiv licencearXiv
IrekoGPT: Turning Structured Pruning into Post-Hoc Slimmable LLMs
Abstract
We introduce IrekoGPT, a post-hoc method for converting pretrained LLMs into slimmable models whose width can be adjusted at inference time. Building on SliceGPT, we retain its projection matrices without pruning them, allowing a single model to expose nested subnetworks at different widths. We improve robustness by calibrating each layer across multiple compression ratios, and correct downstream linear layers through gradient-free ridge regression. Across Llama and Qwen models, preliminary results show improvements over naive PCA-based slimming, with the largest gains at high compression. Code is available at https://github.com/aimagelab/IrekoGPT
Related
- Structured Pruning of Feedback Delay Networks
- Association of a structured prevention algorithm with nurse knowledge and device-related pressure injury severity in critical care: A pre‑post study
- Estimation of heat flux and temperature in a tool during turning
- PRIME: Frequency-Aware Unified Pruning and Merging for MoE Compression
- PMR-Aided Shifted-Pruning SCL-Flip Decoding for CRC-Aided Polar Codes
BibTeX
@misc{moriello2026irekogpt,
title = {{IrekoGPT: Turning Structured Pruning into Post-Hoc Slimmable LLMs}},
author = {Pietro Moriello and Pietro Buzzega and Angelo Porrello and Simone Calderara},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.00426},
eprint = {2610.00426},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.00426},
}
RIS
TY - UNPB
TI - IrekoGPT: Turning Structured Pruning into Post-Hoc Slimmable LLMs
AU - Pietro Moriello
AU - Pietro Buzzega
AU - Angelo Porrello
AU - Simone Calderara
PY - 2026
DA - 2026/10/01
PB - arXiv
DO - 10.48550/arxiv.2610.00426
UR - https://arxiv.org/abs/2610.00426
AB - We introduce IrekoGPT, a post-hoc method for converting pretrained LLMs into slimmable models whose width can be adjusted at inference time. Building on SliceGPT, we retain its projection matrices without pruning them, allowing a single model to expose nested subnetworks at different widths. We improve robustness by calibrating each layer across multiple compression ratios, and correct downstream linear layers through gradient-free ridge regression. Across Llama and Qwen models, preliminary results show improvements over naive PCA-based slimming, with the largest gains at high compression. Code is available at https://github.com/aimagelab/IrekoGPT
ER -