{
  "abstract": "Continual learning of large language models requires incorporating new knowledge while limiting catastrophic forgetting; however, many low-rank adaptation (LoRA)-based methods retain task-specific modules or separate learning spaces that grow with the task sequence. We propose PRISM-LoRA, a replay-free framework that repeatedly trains a single LoRA module, merges its update into the backbone, and reinitializes it for the next task. Unlike LoRA-based continual-learning approaches that retain task-specific modules or allocate separate task-specific learning spaces, PRISM-LoRA uses the directional structure of the accumulated backbone weight change to guide both subsequent training and consolidation while reusing a single LoRA module. Before each task, PRISM-LoRA decomposes the accumulated backbone weight change using singular value decomposition and uses the squared singular values as directional spectral energies. These energies are treated as empirical weight-space indicators of directions associated with retained performance, rather than as direct evidence of stored knowledge. Principal-direction regularization penalizes overlap with high-energy principal directions during training, whereas dynamic merge applies direction-wise scaling to the learned weight change before merging it into the backbone. On T5-Large, PRISM-LoRA achieved Avg OP scores of 79.4±0.2% on Standard CL and 73.3±0.3% on the 15-task Long Sequence benchmark across three random seeds. The Standard CL result was comparable to CLoRA, whereas the Long Sequence result was 1.6%p higher than CSF. On LLaMA2-7B, PRISM-LoRA achieved Avg OP scores of 80.4 ± 0.1% and 75.9 ± 0.1% on the Standard CL and Long Sequence benchmarks, respectively. Ablation and performance-recovery analyses further showed that both directional protection stages contributed to retention and that a small number of high-energy directions were associated with a substantial portion of the retained performance.",
  "abstract_shown": true,
  "affiliations": [
    "Department of Computing, Gachon University, Seongnam-si 13120, Republic of Korea"
  ],
  "article_type": null,
  "arxiv": null,
  "author_total": 2,
  "authors": [
    {
      "affs": [
        0
      ],
      "name": "Taehyeong Kwon",
      "orcid": "0009-0006-0452-9085"
    },
    {
      "affs": [
        0
      ],
      "name": "Okran Jeong",
      "orcid": "0000-0003-2182-1679"
    }
  ],
  "bibtex": "https://research.draconitic.com/paper/9cgrg3y6ba501.bib",
  "cited_by": 0,
  "doi": "10.3390/math14193616",
  "fulltext_url": null,
  "id": "9cgrg3y6ba501",
  "issns": [
    "2227-7390"
  ],
  "landing_url": "https://www.mdpi.com/2227-7390/14/19/3616",
  "language": null,
  "licence": "CC-BY-4.0",
  "licence_label": "CC BY 4.0",
  "licence_source": "article",
  "licence_url": "https://creativecommons.org/licenses/by/4.0/",
  "metadata_licence": null,
  "native_subjects": [
    [
      "openalex",
      "1702|Domain Adaptation and Few-Shot Learning"
    ]
  ],
  "openalex": "W7220711338",
  "pdf_url": null,
  "pmcid": null,
  "pmid": null,
  "published": "2026-10-06",
  "publisher": "MDPI AG",
  "ris": "https://research.draconitic.com/paper/9cgrg3y6ba501.ris",
  "sources": [
    {
      "first_seen": "2026-10-30T12:18:29Z",
      "native_id": "10.3390/math14193616",
      "source": "crossref",
      "url": "https://www.mdpi.com/2227-7390/14/19/3616"
    }
  ],
  "subjects": [
    {
      "field": "mathematics",
      "origin": "issn:2227-7390",
      "subfield": "general"
    }
  ],
  "title": "PRISM-LoRA: Principal-Direction-Guided Single-LoRA Merging Framework for Continual Learning of Large Language Models",
  "title_text": "PRISM-LoRA: Principal-Direction-Guided Single-LoRA Merging Framework for Continual Learning of Large Language Models",
  "type": "peer-reviewed",
  "updated_at": "2026-10-30T12:18:29Z",
  "url": "https://research.draconitic.com/paper/9cgrg3y6ba501",
  "venue": "Mathematics",
  "versions": []
}