Comparing Temporal Averaging and Transformer Encoders for Video-Based Cattle Behavior Classification Under Severe Class Imbalance.
4 affiliations
- 1Department of Smart Farm, Gyeongsang National University, Jinju 52828, Republic of Korea.
- 2Department of Bio-Systems Engineering, Gyeongsang National University, Jinju 52828, Republic of Korea.
- 3Department of Computer Science, Bahria University, Shangrila Road, Sector E-8, Islamabad 44000, Pakistan.
- 4Department of Bio-Industrial Machinery Engineering, Institute of Smart Space Agriculture (ISSA), Gyeongsang National University, Jinju 52828, Republic of Korea.
Abstract
Automated cattle behavior monitoring from video is challenging due to severe class imbalance and the need for efficient temporal modeling. We compared four architectures on 542 videos containing standing, lying down, and rumination; rumination appeared only six times, while two dataset classes were absent. ResNet-50 and EfficientNet-B3 backbones were paired with either a Transformer encoder or temporal averaging and trained with three-fold stratified cross-validation and weighted cross-entropy. EfficientNet-B3 with temporal averaging (Eff-B3 + Avg) achieved the highest out-of-fold accuracy (85.61%) and best calibration (ECE = 0.074), while requiring 41 MB and 28 ms per video. Its Transformer counterpart scored 84.13% despite four times more parameters. ResNet-50 + Transformer reached 84.69% but was poorly calibrated and storage-intensive. Frame ablation identified 16 frames per video as the best accuracy-speed trade-off. No model correctly classified any rumination example, underscoring that rumination was not learnable under the specific weighted-cross-entropy and stratified-sampling strategies evaluated here; this finding does not establish a general impossibility of learning extremely underrepresented behaviors under alternative training strategies such as focal loss, oversampling, anomaly detection, or self-supervised pre-training. Rare-class analysis, treating support ≤106 as rare, showed high-confidence rumination errors: mean confidence was 0.891, but only 1.01% of probability mass was assigned to the true class, yielding a class-wise expected calibration error of 0.891. These findings indicate that calibration post-processing cannot rescue a class with zero true positives.
Related
- Comparing ARG inference methods under transmission of reproductive success: Tree imbalance matters
- Contextual Biomedical Language Models for Imbalance-Aware Drug-Food Interaction Classification
- A lightweight privacy-aware federated vision transformer for tomato leaf disease classification
- MangoSwin-attention: a swin transformer with attention pooling for generalizable mango leaf disease classification
- A hybrid EfficientNet-B0-CBAM-Vision Transformer framework for accurate cherry leaf disease classification
BibTeX
@article{maratovich2026comparing,
title = {{Comparing Temporal Averaging and Transformer Encoders for Video-Based Cattle Behavior Classification Under Severe Class Imbalance.}},
author = {Daniyar Pirlepesov Maratovich and Dae-Yeong Kang and Mohammad Ismail and Usama Hanif and Seong-Hun Kim and Dasanayaka Lekamalage Vindya Sathsarani Dasanayaka and Reginald Gbenga Azuatalam and Muhammad Ismail Khan and Hyeon-Tae Kim},
journal = {Veterinary sciences},
year = {2026},
month = oct,
doi = {10.3390/vetsci13101068},
url = {https://europepmc.org/article/MED/42944163},
note = {CC BY},
}
RIS
TY - JOUR
TI - Comparing Temporal Averaging and Transformer Encoders for Video-Based Cattle Behavior Classification Under Severe Class Imbalance.
AU - Daniyar Pirlepesov Maratovich
AU - Dae-Yeong Kang
AU - Mohammad Ismail
AU - Usama Hanif
AU - Seong-Hun Kim
AU - Dasanayaka Lekamalage Vindya Sathsarani Dasanayaka
AU - Reginald Gbenga Azuatalam
AU - Muhammad Ismail Khan
AU - Hyeon-Tae Kim
PY - 2026
DA - 2026/10/10
JO - Veterinary sciences
DO - 10.3390/vetsci13101068
UR - https://europepmc.org/article/MED/42944163
AB - Automated cattle behavior monitoring from video is challenging due to severe class imbalance and the need for efficient temporal modeling. We compared four architectures on 542 videos containing standing, lying down, and rumination; rumination appeared only six times, while two dataset classes were absent. ResNet-50 and EfficientNet-B3 backbones were paired with either a Transformer encoder or temporal averaging and trained with three-fold stratified cross-validation and weighted cross-entropy. EfficientNet-B3 with temporal averaging (Eff-B3 + Avg) achieved the highest out-of-fold accuracy (85.61%) and best calibration (ECE = 0.074), while requiring 41 MB and 28 ms per video. Its Transformer counterpart scored 84.13% despite four times more parameters. ResNet-50 + Transformer reached 84.69% but was poorly calibrated and storage-intensive. Frame ablation identified 16 frames per video as the best accuracy-speed trade-off. No model correctly classified any rumination example, underscoring that rumination was not learnable under the specific weighted-cross-entropy and stratified-sampling strategies evaluated here; this finding does not establish a general impossibility of learning extremely underrepresented behaviors under alternative training strategies such as focal loss, oversampling, anomaly detection, or self-supervised pre-training. Rare-class analysis, treating support ≤106 as rare, showed high-confidence rumination errors: mean confidence was 0.891, but only 1.01% of probability mass was assigned to the true class, yielding a class-wise expected calibration error of 0.891. These findings indicate that calibration post-processing cannot rescue a class with zero true positives.
SN - 2306-7381
ER -