Enhancing Large Language Models Through Pre-Training, In-Training, and Post-Training Deletion
Abstract
Large language models (LLMs) are commonly enhanced through additional data, parameter optimization, architectural modifications, preference optimization, and increased computational resources. However, improvement does not necessarily require adding information, parameters, components, or computational operations. In some circumstances, the removal of selected objects may improve a model by reducing redundancy, eliminating interference, simplifying computation, or changing the distribution of learning signals. This paper proposes a general conceptual framework for enhancing large language models through deletion performed before training, during training, and after training. The framework treats deletion as a broad class of operations rather than restricting it to the removal of training examples, previously learned knowledge, or any single established technique. Potential deletion targets include data, information, parameters, architectural components, connections, internal representations, behavioral patterns, computational pathways, training objectives, auxiliary mechanisms, memory structures, and other removable objects associated with model development and operation. The framework organizes deletion along three dimensions: lifecycle stage, target category, and deletion strategy. It further introduces a formal representation of deletion operations, a target-selection principle based on expected marginal contribution, and an evaluation framework that considers capability, efficiency, reliability, generalization, and the preservation of useful functionality. A central proposition is that appropriately selected deletion operations may improve selected aspects of model performance when the benefits of removing an object exceed the costs of its removal. The paper develops a research agenda for investigating this proposition through controlled experiments, cross-stage comparisons, interaction analysis, and multi-objective evaluation. The proposed framework does not assume that deletion is inherently beneficial or that all deletion targets are technically removable. Instead, it establishes deletion as a general design dimension for systematic investigation across the lifecycle of large language models.
Related
- Enhancing Large Language Models Through Post-Training Selective Amnesia
- Deletion Learning for Multi-Agent Systems (DL-MAS): A General Framework for Enhancing Collective Intelligence Through Pre-Training, In-Training, and Post-Training Deletion
- Enhancing Large Language Models Through the Suppression of Other Intelligence: A General Framework
- Deletion Learning (DL): A General Framework for Enhancing Learning Systems Through Pre-Learning, In-Learning, and Post-Learning Deletion
- Enhancing network science with large language models and federated artificial intelligence
BibTeX
@misc{zhang2026enhancing,
title = {{Enhancing Large Language Models Through Pre-Training, In-Training, and Post-Training Deletion}},
author = {Jincheng Zhang},
howpublished = {Zenodo},
year = {2026},
month = oct,
doi = {10.5281/zenodo.23464143},
url = {https://zenodo.org/records/23464143},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Enhancing Large Language Models Through Pre-Training, In-Training, and Post-Training Deletion
AU - Jincheng Zhang
PY - 2026
DA - 2026/10/19
PB - Zenodo
DO - 10.5281/zenodo.23464143
UR - https://zenodo.org/records/23464143
AB - Large language models (LLMs) are commonly enhanced through additional data, parameter optimization, architectural modifications, preference optimization, and increased computational resources. However, improvement does not necessarily require adding information, parameters, components, or computational operations. In some circumstances, the removal of selected objects may improve a model by reducing redundancy, eliminating interference, simplifying computation, or changing the distribution of learning signals. This paper proposes a general conceptual framework for enhancing large language models through deletion performed before training, during training, and after training. The framework treats deletion as a broad class of operations rather than restricting it to the removal of training examples, previously learned knowledge, or any single established technique. Potential deletion targets include data, information, parameters, architectural components, connections, internal representations, behavioral patterns, computational pathways, training objectives, auxiliary mechanisms, memory structures, and other removable objects associated with model development and operation. The framework organizes deletion along three dimensions: lifecycle stage, target category, and deletion strategy. It further introduces a formal representation of deletion operations, a target-selection principle based on expected marginal contribution, and an evaluation framework that considers capability, efficiency, reliability, generalization, and the preservation of useful functionality. A central proposition is that appropriately selected deletion operations may improve selected aspects of model performance when the benefits of removing an object exceed the costs of its removal. The paper develops a research agenda for investigating this proposition through controlled experiments, cross-stage comparisons, interaction analysis, and multi-objective evaluation. The proposed framework does not assume that deletion is inherently beneficial or that all deletion targets are technically removable. Instead, it establishes deletion as a general design dimension for systematic investigation across the lifecycle of large language models.
ER -