Interpretable Symptom Vectors for Depression in a Large Language Model
Abstract
Patients with depression present with diverse symptom profiles, yet clinical practice routinely reduces this variation to a single severity score. Large language models (LLMs) can potentially capture various symptoms and their severity from patient speech. However, how depressive symptoms are represented inside LLMs remains poorly understood, limiting clinical trust. To examine whether internal model activations match clinician judgment, we analyzed the residual stream of Gemma-3-27B-PT using mechanistic interpretability techniques. Recording activations across symptom descriptions drawn from validated clinical instruments, we found that symptom groups geometrically separated the most at layer 21 across multiple distance metrics. Using Semantic Projection, we then projected held-out naturalistic text onto Symptom Vectors constructed from these instruments. The resulting per-symptom coefficients preserved clinician-annotated rank ordering across mood, somatic, and suicidality axes. Furthermore, a single depression vector in Layer 21 separates held-out depressive from non-depressive text (AUC = 0.789), which can be used as an emotional valence gate that restricts symptom projection to depressive speech. These results reveal a decorrelated, clinician-aligned symptom signal readable directly from internal activations, offering a mechanistic foundation for interpretable depression-assessment tools.
Related
- Large Language Models Predict Human Social Behavior via Interpretable Mechanisms
- SAIL: Sparse Autoencoders for Interpretable Alignment of Human Vision and Multimodal Large Language Models
- Gut bacterial composition and predicted functional profiles associated with depressive symptom burden in Japanese adults recruited independently of depression diagnosis
- Sex differences in symptom-specific brain abnormalities across disease stages in depression.
- Head-level lesion-symptom mapping of picture naming in vision-language models
BibTeX
@misc{zhu2026interpretable,
title = {{Interpretable Symptom Vectors for Depression in a Large Language Model}},
author = {Fangyi Zhu and Ajay Subramanian and Allison Constant and Camille Wang and Ravish Gupta and Corey J. Keller},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.01832},
eprint = {2609.01832},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.01832},
}
RIS
TY - UNPB
TI - Interpretable Symptom Vectors for Depression in a Large Language Model
AU - Fangyi Zhu
AU - Ajay Subramanian
AU - Allison Constant
AU - Camille Wang
AU - Ravish Gupta
AU - Corey J. Keller
PY - 2026
DA - 2026/09/01
PB - arXiv
DO - 10.48550/arxiv.2609.01832
UR - https://arxiv.org/abs/2609.01832
AB - Patients with depression present with diverse symptom profiles, yet clinical practice routinely reduces this variation to a single severity score. Large language models (LLMs) can potentially capture various symptoms and their severity from patient speech. However, how depressive symptoms are represented inside LLMs remains poorly understood, limiting clinical trust. To examine whether internal model activations match clinician judgment, we analyzed the residual stream of Gemma-3-27B-PT using mechanistic interpretability techniques. Recording activations across symptom descriptions drawn from validated clinical instruments, we found that symptom groups geometrically separated the most at layer 21 across multiple distance metrics. Using Semantic Projection, we then projected held-out naturalistic text onto Symptom Vectors constructed from these instruments. The resulting per-symptom coefficients preserved clinician-annotated rank ordering across mood, somatic, and suicidality axes. Furthermore, a single depression vector in Layer 21 separates held-out depressive from non-depressive text (AUC = 0.789), which can be used as an emotional valence gate that restricts symptom projection to depressive speech. These results reveal a decorrelated, clinician-aligned symptom signal readable directly from internal activations, offering a mechanistic foundation for interpretable depression-assessment tools.
ER -