The ClimaSentAR dataset for evaluating cross-domain transferability of large language models in climate sentiment analysis
- 1College of Technological Innovation, Zayed University
Abstract
Arabic research on climate sentiment analysis remains limited due to linguistic diversity, limited resources, and the scarcity of annotated datasets. Existing approaches primarily rely on questionnaires or general-purpose sentiment analysis models, which often fail to capture domain-specific sentiment expressions. To address this gap, this paper introduces ClimaSentAR, the first large-scale Arabic climate-focused sentiment dataset comprising 23,957 samples from the X platform. Native Arabic speakers annotate the dataset with a moderate inter-annotator agreement of 0.45. It contains 5510 positive, 8140 negative, and 10,307 neutral instances. To validate the importance and robustness of ClimaSentAR and address the gap in Arabic climate change sentiment analysis, this study fine-tunes Arabic-centric pre-trained language models and large language models. In contrast to prior work based solely on general-purpose datasets, the proposed approach systematically examines both in-domain performance and cross-domain generalization, while incorporating adaptation strategies including Domain-Adaptive Pre-training, AdapterFusion, and data augmentation. Experimental results demonstrate that climate-specific fine-tuning substantially improves in-domain climate sentiment performance, with AraBERT achieving the highest macro F1 score of 0.753 on climate-related data. Cross-domain evaluations resulted in degraded model performance, with climate-tuned models achieving a macro F1 score of up to 0.599, compared to less than 0.48 for general-purpose models. To support reproducibility and future research, the fine-tuned AraBERT model is publicly available on Hugging Face with an interactive web interface for testing and further fine-tuning by researchers and practitioners.
Related
- Evaluating parameter-efficient fine-tuning of large language models for sentiment analysis with bayesian optimization
- EComReviews-BD: A benchmark dataset for aspect-based sentiment analysis on multilingual Bangladeshi E-Commerce reviews
- Sentiment Analysis of Clinical Interviews Identifies Language Markers of Identity Diffusion in Adolescents with Personality Disorder
- Evaluating the Transferability of Pathology Foundation Models Across Cancer-related H&E Neurodegeneration-related Immunohistochemical Classification Tasks
- Evaluating Large Language Models as Tools to Navigate Researchers in Rapidly Evolving Research Landscapes: A Case Study in Cancer Drug Response Prediction
BibTeX
@article{kaddoura2026climasentar,
title = {{The ClimaSentAR dataset for evaluating cross-domain transferability of large language models in climate sentiment analysis}},
author = {Sanaa Kaddoura and Mariam Alzaabi and Reem Nassar},
journal = {Scientific Reports},
year = {2026},
month = jul,
doi = {10.1038/s41598-026-59024-6},
url = {https://doi.org/10.1038/s41598-026-59024-6},
note = {CC BY},
}
RIS
TY - JOUR
TI - The ClimaSentAR dataset for evaluating cross-domain transferability of large language models in climate sentiment analysis
AU - Sanaa Kaddoura
AU - Mariam Alzaabi
AU - Reem Nassar
PY - 2026
DA - 2026/07/01
JO - Scientific Reports
DO - 10.1038/s41598-026-59024-6
UR - https://doi.org/10.1038/s41598-026-59024-6
AB - Arabic research on climate sentiment analysis remains limited due to linguistic diversity, limited resources, and the scarcity of annotated datasets. Existing approaches primarily rely on questionnaires or general-purpose sentiment analysis models, which often fail to capture domain-specific sentiment expressions. To address this gap, this paper introduces ClimaSentAR, the first large-scale Arabic climate-focused sentiment dataset comprising 23,957 samples from the X platform. Native Arabic speakers annotate the dataset with a moderate inter-annotator agreement of 0.45. It contains 5510 positive, 8140 negative, and 10,307 neutral instances. To validate the importance and robustness of ClimaSentAR and address the gap in Arabic climate change sentiment analysis, this study fine-tunes Arabic-centric pre-trained language models and large language models. In contrast to prior work based solely on general-purpose datasets, the proposed approach systematically examines both in-domain performance and cross-domain generalization, while incorporating adaptation strategies including Domain-Adaptive Pre-training, AdapterFusion, and data augmentation. Experimental results demonstrate that climate-specific fine-tuning substantially improves in-domain climate sentiment performance, with AraBERT achieving the highest macro F1 score of 0.753 on climate-related data. Cross-domain evaluations resulted in degraded model performance, with climate-tuned models achieving a macro F1 score of up to 0.599, compared to less than 0.48 for general-purpose models. To support reproducibility and future research, the fine-tuned AraBERT model is publicly available on Hugging Face with an interactive web interface for testing and further fine-tuning by researchers and practitioners.
SN - 2045-2322
ER -