Enhancing Large Language Models Through Post-Training Selective Amnesia
Abstract
Large language models (LLMs) are typically improved through additional training, fine-tuning, preference optimization, or the incorporation of new knowledge and capabilities. These approaches generally attempt to improve performance by adding, reinforcing, or reorganizing learned behaviors. However, some limitations may arise not only from insufficient learning but also from the persistence of information, representations, associations, or behavioral patterns that interfere with desired performance. This paper proposes Post-Training Selective Amnesia (PTSA), a conceptual framework for investigating whether selectively weakening or removing particular learned influences after training can improve an LLM's target capabilities. In this framework, amnesia denotes an operational property of a computational model rather than a clinical condition or a claim about human-like memory. Unlike conventional machine unlearning, which commonly focuses on removing specified data influences or knowledge while preserving general utility, PTSA selects candidate forgetting interventions according to their experimentally estimated marginal contribution to target-capability improvement. The framework combines candidate identification, counterfactual evaluation, selective intervention, adaptive intervention strength, protected-capability constraints, and post-intervention recovery testing. Its central principle is that a component should not be selected for forgetting merely because it can be forgotten; it should be selected only when controlled evidence indicates that intervening on it provides a beneficial trade-off. This paper formalizes the framework, distinguishes it from related research, proposes an algorithm and testable hypotheses, and outlines an experimental protocol. As a conceptual and methodological proposal, PTSA does not claim that selective amnesia necessarily improves LLM performance. Its effectiveness, novelty relative to existing methods, and practical value require empirical validation.
Related
- Enhancing Large Language Models Through Pre-Training, In-Training, and Post-Training Deletion
- Deletion Learning for Multi-Agent Systems (DL-MAS): A General Framework for Enhancing Collective Intelligence Through Pre-Training, In-Training, and Post-Training Deletion
- Enhancing Large Language Models Through the Suppression of Other Intelligence: A General Framework
- Enhancing pollutant removal from contaminated soils using yield stress fluids as selective blocking agents
- Enhancing deep sleep with auditory stimulation to boost cognitive training responsiveness in people with Parkinson's disease and mild cognitive impairment: Rationale and design of the double-blind randomized controlled trial PD-CogT-Sleep
BibTeX
@misc{zhang2026enhancing,
title = {{Enhancing Large Language Models Through Post-Training Selective Amnesia}},
author = {Jincheng Zhang},
howpublished = {Zenodo},
year = {2026},
month = oct,
doi = {10.5281/zenodo.23452541},
url = {https://zenodo.org/records/23452541},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Enhancing Large Language Models Through Post-Training Selective Amnesia
AU - Jincheng Zhang
PY - 2026
DA - 2026/10/18
PB - Zenodo
DO - 10.5281/zenodo.23452541
UR - https://zenodo.org/records/23452541
AB - Large language models (LLMs) are typically improved through additional training, fine-tuning, preference optimization, or the incorporation of new knowledge and capabilities. These approaches generally attempt to improve performance by adding, reinforcing, or reorganizing learned behaviors. However, some limitations may arise not only from insufficient learning but also from the persistence of information, representations, associations, or behavioral patterns that interfere with desired performance. This paper proposes Post-Training Selective Amnesia (PTSA), a conceptual framework for investigating whether selectively weakening or removing particular learned influences after training can improve an LLM's target capabilities. In this framework, amnesia denotes an operational property of a computational model rather than a clinical condition or a claim about human-like memory. Unlike conventional machine unlearning, which commonly focuses on removing specified data influences or knowledge while preserving general utility, PTSA selects candidate forgetting interventions according to their experimentally estimated marginal contribution to target-capability improvement. The framework combines candidate identification, counterfactual evaluation, selective intervention, adaptive intervention strength, protected-capability constraints, and post-intervention recovery testing. Its central principle is that a component should not be selected for forgetting merely because it can be forgotten; it should be selected only when controlled evidence indicates that intervening on it provides a beneficial trade-off. This paper formalizes the framework, distinguishes it from related research, proposes an algorithm and testable hypotheses, and outlines an experimental protocol. As a conceptual and methodological proposal, PTSA does not claim that selective amnesia necessarily improves LLM performance. Its effectiveness, novelty relative to existing methods, and practical value require empirical validation.
ER -