FLiTrak3D: Improved deep-learning-based 3D insect flightkinematics tracking using spatial and temporal encoding
- 1Wagningen University
Abstract
Quantitative measurements of insect flight behaviour are essential for understanding the biomechanics, control, and ecology of flight, yet obtaining such measurements under free-flight conditions remains challenging. Small body size, rapid wing motion, visual symmetry, and frequent occlusions complicate three-dimensional pose estimation, often requiring restrictive experimental setups or substantial manual annotation. We present FLiTrak3D, an open-source Python package for estimating insect flight kinematics from multi-view videography. It combines machine-learning-based markerless tracking with biomechanical modelling to reconstruct and parametrise insect body and wing motion. The workflow integrates image preprocessing, including dynamic image cropping and enhancement, two-dimensional bodypart localisation, three-dimensional reconstruction, and optimisation-based skeletal fitting. A key innovation is the use of spatio-temporal encoding across synchronised camera views and adjacent frames to improve neural-network awareness of spatial and temporal context during bodypart localisation. Multi-view image stitching allows the network to use cross-view spatial relationships, reducing left-right bodypart misidentifications, while temporal encoding stacks consecutive greyscale frames into RGB images provide short-term motion information. A species-specific skeleton is then fitted to reconstructed keypoints to enforce kinematic constraints, and estimate body and wing orientations. We demonstrate the approach using free-flying Aedes aegypti mosquitoes, achieving bodypart localisation errors close to human labelling, and realistic flight kinematics.
Related
- Maturational constraints on tracking of temporal attention in infant language acquisition.
- Pedigree-tracking improved population monitoring of recessive disease-causing alleles in pedigreed canine populations
- Improved passive catheter tracking with positive contrast for CMR-guided cardiac catheterization using partial saturation (pSAT)
- Auditory tracking magnitude, temporal alignment, and anatomical distribution during wakefulness and propofol-based anesthesia in human sEEG
- Source-Decomposed Multimodal Graph Connectivity: A Temporal Mixture-Manifold Pipeline for Tracking Ischemic Injury and Treatment
BibTeX
@misc{cribellier2026flitrak3d,
title = {{FLiTrak3D: Improved deep-learning-based 3D insect flightkinematics tracking using spatial and temporal encoding}},
author = {A. Cribellier and A.-J. Buchner},
howpublished = {bioRxiv},
year = {2026},
month = sep,
doi = {10.64898/2026.09.07.749842},
url = {https://www.biorxiv.org/content/10.64898/2026.09.07.749842v1},
note = {CC BY-NC},
}
RIS
TY - UNPB
TI - FLiTrak3D: Improved deep-learning-based 3D insect flightkinematics tracking using spatial and temporal encoding
AU - A. Cribellier
AU - A.-J. Buchner
PY - 2026
DA - 2026/09/07
PB - bioRxiv
DO - 10.64898/2026.09.07.749842
UR - https://www.biorxiv.org/content/10.64898/2026.09.07.749842v1
AB - Quantitative measurements of insect flight behaviour are essential for understanding the biomechanics, control, and ecology of flight, yet obtaining such measurements under free-flight conditions remains challenging. Small body size, rapid wing motion, visual symmetry, and frequent occlusions complicate three-dimensional pose estimation, often requiring restrictive experimental setups or substantial manual annotation. We present FLiTrak3D, an open-source Python package for estimating insect flight kinematics from multi-view videography. It combines machine-learning-based markerless tracking with biomechanical modelling to reconstruct and parametrise insect body and wing motion. The workflow integrates image preprocessing, including dynamic image cropping and enhancement, two-dimensional bodypart localisation, three-dimensional reconstruction, and optimisation-based skeletal fitting. A key innovation is the use of spatio-temporal encoding across synchronised camera views and adjacent frames to improve neural-network awareness of spatial and temporal context during bodypart localisation. Multi-view image stitching allows the network to use cross-view spatial relationships, reducing left-right bodypart misidentifications, while temporal encoding stacks consecutive greyscale frames into RGB images provide short-term motion information. A species-specific skeleton is then fitted to reconstructed keypoints to enforce kinematic constraints, and estimate body and wing orientations. We demonstrate the approach using free-flying Aedes aegypti mosquitoes, achieving bodypart localisation errors close to human labelling, and realistic flight kinematics.
ER -