Discovering compositional, interpretable structure in social behavior
- 1Johns Hopkins University
Abstract
Quantitative assessment of behavior is vital for rigorous analysis of how animals interact in the world and how behavior is generated from brain activity. Recent advances in automated pose detection have made possible data-driven discovery of movement action sequences. While many algorithms have been developed for this purpose, their categorical nature, i.e., the assumption that behavior is one action at a time, prohibits expressive quantification of social behavior. Such behavior is combinatorial and compositional, with multiple animals performing actions that may be either independent or coordinated over time. Here we show how developments in decomposed linear dynamical systems (dLDS) can instead represent action-generating dynamics as a signed, graded mixture of shared linear operators acting directly on keypoint positions. We demonstrate our approach across both single-mouse and two-mouse datasets, comparing against a state-of-the-art switching model, keypoint-MoSeq. In single-mouse data, we find that a composition of three operators can capture the full repertoire of 21 keypoint-MoSeq syllables, showing that even for a single animal, the action space is more parsimoniously described by a continuous subspace than clustered motifs. In a two-mouse resident--intruder assay, we identify, without behavioral labels, separate primitives for self-action, inter-animal-geometry, and social-action. The latter couples an animal's action to its partner's relative bearing, enabling the action to incorporate the social geometry in which it occurs. dLDS further decodes human-labeled social behaviors (i.e., "attack", "investigation", "mount", and "other") more accurately than behavior-selective keypoint-MoSeq syllable sets using fewer units. Together, these results demonstrate that compositional, time-varying linear dynamics can provide a path towards flexible and interpretable modeling of action-level behavior for both individual and social behavior.
Related
- Large Language Models Predict Human Social Behavior via Interpretable Mechanisms
- Discovering interpretable low-dimensional dynamics using maximum entropy
- Learning interpretable compositional features associated with continuous covariates
- Host Identity and Urban Space Structure Phyllosphere Bacterial Communities Through Compositional Turnover in a Tropical Andean City
- Language-aligned models and structured scene descriptions reveal sensitivity to compositional scene structure in the high-level visual cortex
BibTeX
@misc{wan2026discovering,
title = {{Discovering compositional, interpretable structure in social behavior}},
author = {Y. Wan and S. Koukuntla and A. El Hady and M. C. Aoi and A. Charles},
howpublished = {bioRxiv},
year = {2026},
month = nov,
doi = {10.64898/2026.11.03.764108},
url = {https://www.biorxiv.org/content/10.64898/2026.11.03.764108v1},
note = {CC BY-NC-ND},
}
RIS
TY - UNPB
TI - Discovering compositional, interpretable structure in social behavior
AU - Y. Wan
AU - S. Koukuntla
AU - A. El Hady
AU - M. C. Aoi
AU - A. Charles
PY - 2026
DA - 2026/11/03
PB - bioRxiv
DO - 10.64898/2026.11.03.764108
UR - https://www.biorxiv.org/content/10.64898/2026.11.03.764108v1
AB - Quantitative assessment of behavior is vital for rigorous analysis of how animals interact in the world and how behavior is generated from brain activity. Recent advances in automated pose detection have made possible data-driven discovery of movement action sequences. While many algorithms have been developed for this purpose, their categorical nature, i.e., the assumption that behavior is one action at a time, prohibits expressive quantification of social behavior. Such behavior is combinatorial and compositional, with multiple animals performing actions that may be either independent or coordinated over time. Here we show how developments in decomposed linear dynamical systems (dLDS) can instead represent action-generating dynamics as a signed, graded mixture of shared linear operators acting directly on keypoint positions. We demonstrate our approach across both single-mouse and two-mouse datasets, comparing against a state-of-the-art switching model, keypoint-MoSeq. In single-mouse data, we find that a composition of three operators can capture the full repertoire of 21 keypoint-MoSeq syllables, showing that even for a single animal, the action space is more parsimoniously described by a continuous subspace than clustered motifs. In a two-mouse resident--intruder assay, we identify, without behavioral labels, separate primitives for self-action, inter-animal-geometry, and social-action. The latter couples an animal's action to its partner's relative bearing, enabling the action to incorporate the social geometry in which it occurs. dLDS further decodes human-labeled social behaviors (i.e., "attack", "investigation", "mount", and "other") more accurately than behavior-selective keypoint-MoSeq syllable sets using fewer units. Together, these results demonstrate that compositional, time-varying linear dynamics can provide a path towards flexible and interpretable modeling of action-level behavior for both individual and social behavior.
ER -