Language models annotate mass cytometry clusters within the range where human experts disagree
- 1Luxembourg Institute of Health
Abstract
Cell-type identification of clusters from unsupervised cytometry pipelines (e.g. FlowSOM, GigaSOM) is a time-consuming, expert-dependent bottleneck, typically requiring an expert to read a heatmap of median marker intensities and assign a label to each cluster. Large language models (LLMs) can annotate single-cell RNA-seq clusters from marker-gene lists, but cytometry cluster data differ in both dimensionality and data type. We present a direct procedure in which cluster medians are scaled per marker and grouped by similarity into a few dozen metaclusters. The continuous marker profiles are then passed to the LLM, which returns a cell-type label and confidence level for each metacluster in a single call, without a gating hierarchy, manually defined marker-specific thresholds, or task-specific training data. Across four human and mouse datasets spanning blood, bone marrow and gut, a single well-chosen model reproduces the reference annotation at the major-lineage level with 85-98% metacluster-level concordance (cloud Claude Sonnet 4.6 85-98%, best local model 83-96%), substantially exceeding majority-class baselines; chance-corrected agreement is Cohen's k = 0.67-0.83. Local open-weight models perform competitively with the cloud model on the independent benchmarks, enabling application to clinical or patient-derived data subject to data-protection restrictions. Reliability is model-dependent: no local model is best everywhere, and the frontier model Claude Opus 5 was no more reliable than Claude Sonnet 4.6, the most consistent in concordance across datasets. Because no annotation is absolute ground truth, we obtained two additional expert annotations of the same continuous heatmaps, blinded to the reference and to all model output, to establish a method-matched agreement ceiling. The experts agreed on ~99% of major-lineage assignments but only ~73% of T-cell maturation states; on that axis the best models fall within the expert disagreement range. LLM-based annotation is therefore a practical, training-free alternative to rule-based annotation, provided its output is checked against the intended panel and species.
Related
- MI-PEFT: Mixture-of-Experts Integrated Parameter-Efficient Fine-Tuning Protein Language Models Improves Acidophilic Proteins Classification
- Can LLMs Reliably Annotate Bioassay Metadata to Improve Data Readiness?
- Why variant effect predictors and multiplexed assays agree and disagree
- When dates disagree: radiocarbon and sighting-based models reveal range-wide variability in great auk (Pinguinus impennis) extinction estimates
- 47 AI-assisted cytometry panel design for cancer immunotherapy
BibTeX
@misc{hunewald2026language,
title = {{Language models annotate mass cytometry clusters within the range where human experts disagree}},
author = {O. Hunewald and E. T. Grant and T.-M. Boehm and A. Kuehn and V. Despotovic and M. S. Desai and P. V. Nazarov},
howpublished = {bioRxiv},
year = {2026},
month = oct,
doi = {10.64898/2026.10.20.760379},
url = {https://www.biorxiv.org/content/10.64898/2026.10.20.760379v1},
note = {CC BY},
}
RIS
TY - UNPB
TI - Language models annotate mass cytometry clusters within the range where human experts disagree
AU - O. Hunewald
AU - E. T. Grant
AU - T.-M. Boehm
AU - A. Kuehn
AU - V. Despotovic
AU - M. S. Desai
AU - P. V. Nazarov
PY - 2026
DA - 2026/10/20
PB - bioRxiv
DO - 10.64898/2026.10.20.760379
UR - https://www.biorxiv.org/content/10.64898/2026.10.20.760379v1
AB - Cell-type identification of clusters from unsupervised cytometry pipelines (e.g. FlowSOM, GigaSOM) is a time-consuming, expert-dependent bottleneck, typically requiring an expert to read a heatmap of median marker intensities and assign a label to each cluster. Large language models (LLMs) can annotate single-cell RNA-seq clusters from marker-gene lists, but cytometry cluster data differ in both dimensionality and data type. We present a direct procedure in which cluster medians are scaled per marker and grouped by similarity into a few dozen metaclusters. The continuous marker profiles are then passed to the LLM, which returns a cell-type label and confidence level for each metacluster in a single call, without a gating hierarchy, manually defined marker-specific thresholds, or task-specific training data. Across four human and mouse datasets spanning blood, bone marrow and gut, a single well-chosen model reproduces the reference annotation at the major-lineage level with 85-98% metacluster-level concordance (cloud Claude Sonnet 4.6 85-98%, best local model 83-96%), substantially exceeding majority-class baselines; chance-corrected agreement is Cohen's k = 0.67-0.83. Local open-weight models perform competitively with the cloud model on the independent benchmarks, enabling application to clinical or patient-derived data subject to data-protection restrictions. Reliability is model-dependent: no local model is best everywhere, and the frontier model Claude Opus 5 was no more reliable than Claude Sonnet 4.6, the most consistent in concordance across datasets. Because no annotation is absolute ground truth, we obtained two additional expert annotations of the same continuous heatmaps, blinded to the reference and to all model output, to establish a method-matched agreement ceiling. The experts agreed on ~99% of major-lineage assignments but only ~73% of T-cell maturation states; on that axis the best models fall within the expert disagreement range. LLM-based annotation is therefore a practical, training-free alternative to rule-based annotation, provided its output is checked against the intended panel and species.
ER -