A Framework for Clustering and Zero-Shot Labeling of Ecological Images Using Foundation Models
- 1Aarhus University, Department of Electrical and Computer Engineering, Aarhus N, 8200, Denmark
Abstract
The increasing automation of biodiversity monitoring is generating large volumes of ecological images requiring taxonomic identification, creating a substantial annotation bottleneck. Assigning species-level labels requires considerable time and expert knowledge, limiting scalability. We propose BioCluster, a framework for unsupervised clustering and zero-shot labeling of ecological images using foundation models. The framework extracts image embeddings from pretrained models and groups images into taxonomic clusters without task-specific training. Species labels are assigned through zero-shot text matching against the taxa expected in the dataset. We investigate dimensionality reduction and six clustering algorithms for organizing fine-grained imagery. A per-image uncertainty score combines clustering and labeling information to sort images into three confidence tiers, so experts can focus on uncertain and unlabeled cases. We evaluate it on two moth and two bird datasets, with one annotated camera-trap collection per taxonomic group. For these camera-trap datasets, per-image species-level accuracy reached 84% on DK-Moth and 83% on HUGO-Bench Aves. An interactive report presents clusters and zero-shot labels by confidence tier and prioritizes cases requiring expert review. The framework provides a training-free approach for organizing and labeling ecological image collections, reducing manual annotation while retaining expert oversight and demonstrating the potential of foundation models for scalable, semi-automated annotation.
Related
- Spatially guided translation from histology images to transcriptomic profiles using foundation model-driven contrastive learning
- Stoichiometry-Dependent Specificity in Biotin Enrichment: A Benchmarking Framework for Proximity-Labeling Proteomics.
- Beyond the tangled bank: An evolving meta‐ecosystems framework to integrate ecological and evolutionary perspectives
- CellPatch: a flexible and efficient framework for single-cell foundation models empowered by heuristic gene patching
- Reframing Mangroves as Socio-Ecological Coastal Infrastructure: Development and Application of a Six-Pillar Socio-Ecological Infrastructure Framework
BibTeX
@misc{ahmed2026framework,
title = {{A Framework for Clustering and Zero-Shot Labeling of Ecological Images Using Foundation Models}},
author = {S. Ahmed and K. Bjerge and T. T. Hoye and C. N. S. Pedersen},
howpublished = {bioRxiv},
year = {2026},
month = oct,
doi = {10.64898/2026.10.15.759340},
url = {https://www.biorxiv.org/content/10.64898/2026.10.15.759340v1},
note = {CC BY-NC},
}
RIS
TY - UNPB
TI - A Framework for Clustering and Zero-Shot Labeling of Ecological Images Using Foundation Models
AU - S. Ahmed
AU - K. Bjerge
AU - T. T. Hoye
AU - C. N. S. Pedersen
PY - 2026
DA - 2026/10/15
PB - bioRxiv
DO - 10.64898/2026.10.15.759340
UR - https://www.biorxiv.org/content/10.64898/2026.10.15.759340v1
AB - The increasing automation of biodiversity monitoring is generating large volumes of ecological images requiring taxonomic identification, creating a substantial annotation bottleneck. Assigning species-level labels requires considerable time and expert knowledge, limiting scalability. We propose BioCluster, a framework for unsupervised clustering and zero-shot labeling of ecological images using foundation models. The framework extracts image embeddings from pretrained models and groups images into taxonomic clusters without task-specific training. Species labels are assigned through zero-shot text matching against the taxa expected in the dataset. We investigate dimensionality reduction and six clustering algorithms for organizing fine-grained imagery. A per-image uncertainty score combines clustering and labeling information to sort images into three confidence tiers, so experts can focus on uncertain and unlabeled cases. We evaluate it on two moth and two bird datasets, with one annotated camera-trap collection per taxonomic group. For these camera-trap datasets, per-image species-level accuracy reached 84% on DK-Moth and 83% on HUGO-Bench Aves. An interactive report presents clusters and zero-shot labels by confidence tier and prioritizes cases requiring expert review. The framework provides a training-free approach for organizing and labeling ecological image collections, reducing manual annotation while retaining expert oversight and demonstrating the potential of foundation models for scalable, semi-automated annotation.
ER -