AMADEUS: Annotation-free multi-animal direction estimation using self-supervised learning
- 1The University of Tokyo
Abstract
Markerless multi-animal tracking has advanced rapidly, yet measuring social interactions among visually similar animals remains challenging during occlusion and crowding. Here we present AMADEUS (Annotation-free Multi-Animal Direction Estimation Using Self-supervised learning), a tracking system that requires neither manual training annotation nor physical marking. AMADEUS extracts masks representing single animals, uses movement direction to assign head direction, and then synthesizes interaction images using copy-paste augmentation. A detector trained on the synthetic dataset estimates oriented bounding boxes and head direction, from which center, front and rear keypoints are derived. For a video containing instances of almost complete occlusion, AMADEUS additionally uses contrastive learning for identity verification. Across benchmark videos of 10 mice, 10 fish, 32 beetles, 32 ants with one cricket, 60 flies, and 368 ants, AMADEUS outperformed state-of-the-art methods in identity tracking and keypoint localization. AMADEUS enables scalable quantification of interactions across diverse species.
Related
- Beyond profiles: supervised repeat annotation using protein embeddings
- A canonical neural circuit for motion direction estimation
- Learning from Error Predictions Rather Than Prediction Errors: A Perspective on Self-Supervised Learning in the Brain
- Exploring noise in TCR specificity data through supervised learning
- Metabolite-centric Knowledge Graph Unveils Predictive and Mechanistic Insights through Self-supervised Multimodal Learning
BibTeX
@misc{notomi2026amadeus,
title = {{AMADEUS: Annotation-free multi-animal direction estimation using self-supervised learning}},
author = {Y. Notomi and K. Matsumura and S. Dobata},
howpublished = {bioRxiv},
year = {2026},
month = sep,
doi = {10.64898/2026.09.19.752854},
url = {https://www.biorxiv.org/content/10.64898/2026.09.19.752854v1},
note = {CC BY-NC},
}
RIS
TY - UNPB
TI - AMADEUS: Annotation-free multi-animal direction estimation using self-supervised learning
AU - Y. Notomi
AU - K. Matsumura
AU - S. Dobata
PY - 2026
DA - 2026/09/19
PB - bioRxiv
DO - 10.64898/2026.09.19.752854
UR - https://www.biorxiv.org/content/10.64898/2026.09.19.752854v1
AB - Markerless multi-animal tracking has advanced rapidly, yet measuring social interactions among visually similar animals remains challenging during occlusion and crowding. Here we present AMADEUS (Annotation-free Multi-Animal Direction Estimation Using Self-supervised learning), a tracking system that requires neither manual training annotation nor physical marking. AMADEUS extracts masks representing single animals, uses movement direction to assign head direction, and then synthesizes interaction images using copy-paste augmentation. A detector trained on the synthetic dataset estimates oriented bounding boxes and head direction, from which center, front and rear keypoints are derived. For a video containing instances of almost complete occlusion, AMADEUS additionally uses contrastive learning for identity verification. Across benchmark videos of 10 mice, 10 fish, 32 beetles, 32 ants with one cricket, 60 flies, and 368 ants, AMADEUS outperformed state-of-the-art methods in identity tracking and keypoint localization. AMADEUS enables scalable quantification of interactions across diverse species.
ER -