Why shared attention vectors fail: a case for outcome-indexed tuning
Abstract
Dimensional attention in learning is often implemented as a globally shared attention vector, where each stimulus dimension corresponds to a single scalar. These scalars are learned by models through gradient-descent on error, where predictive features acquire more salience. We show that under multi-outcome learning, where models predict more than one outcome, this shared vector becomes unstable; it collapses to its bounds and prevents the models from learning meaningful attentional tunings for learning and generalization. We address this by introducing an outcome-indexed attentional matrix that converts globally shared attentional tuning into an outcome-indexed representation. We present an analysis of the unstable shared vectors and derive the conditions under which it holds. Empirically, three synthetic experiments benchmark the proposed attention matrices and show that they converge to meaningful representations, something shared attention vectors fail to do. These results suggest that outcome-indexed attentional matrices are a general fix for gradient-based attentional processes, which improves models of learning under multi-outcome conditions.
Related
- GBZ-base and GAF-base: Indexed pangenome file formats
- Non-standard memory models with indexed retrieval
- Altered GABAB receptor sensitivity in heavy drinkers indexed using a TMS-based biomarker: a randomized controlled experimental medicine trial
- Agroforestry Spread of Xylella fastidiosa in Minorca: Vectors, Hosts, and Dynamics in Wild Olive Woods and Pastures
- Contrasting Effects of Human, Canine, and Hybrid Adenovirus Vectors on the Phenotypical and Functional Maturation of Human Dendritic Cells: Implications for Clinical Efficacy
BibTeX
@misc{dome2026shared,
title = {{Why shared attention vectors fail: a case for outcome-indexed tuning}},
author = {Lenard Dome},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.08615},
eprint = {2609.08615},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.08615},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Why shared attention vectors fail: a case for outcome-indexed tuning
AU - Lenard Dome
PY - 2026
DA - 2026/09/08
PB - arXiv
DO - 10.48550/arxiv.2609.08615
UR - https://arxiv.org/abs/2609.08615
AB - Dimensional attention in learning is often implemented as a globally shared attention vector, where each stimulus dimension corresponds to a single scalar. These scalars are learned by models through gradient-descent on error, where predictive features acquire more salience. We show that under multi-outcome learning, where models predict more than one outcome, this shared vector becomes unstable; it collapses to its bounds and prevents the models from learning meaningful attentional tunings for learning and generalization. We address this by introducing an outcome-indexed attentional matrix that converts globally shared attentional tuning into an outcome-indexed representation. We present an analysis of the unstable shared vectors and derive the conditions under which it holds. Empirically, three synthetic experiments benchmark the proposed attention matrices and show that they converge to meaningful representations, something shared attention vectors fail to do. These results suggest that outcome-indexed attentional matrices are a general fix for gradient-based attentional processes, which improves models of learning under multi-outcome conditions.
ER -