SiNFluD: A novel benchmark dataset for sindhi figurative language classification
- 1Department of Data Science, Quaid-e-Awam University of Engineering, Science and Technology, Nawabshah, Pakistan; SoloGenAI Pvt. Ltd, Turku, Finland; Corresponding author at: Department of Data Science, Quaid-e-Awam University of Engineering, Science and Technology, Nawabshah, Pakistan
- 2Department of Information Technology, Faculty of Computing and Information Technology, King Abdulaziz University, Jeddah, Saudi Arabia
- 3SDAIA-KFUPM Joint Research Center for Artificial Intelligence, King Fahd University of Petroleum & Minerals, Dhahran, Kingdom of Saudi Arabia
Abstract
In this article, we introduce SiNFluD, a novel benchmark dataset for Sindhi figurative language classification. We collect phrases from web raw text sourced from various blogs, social media platforms, and literary sources, and subsequently prepare the corpus for annotation. Two native annotators label the data using the Doccano text annotation tool, achieving an inter-annotator agreement of 0.81 (Cohen's Kappa). We then establish baseline results using 5-fold and 10-fold cross-validation. Finally, we evaluate mBERT, XLM-RoBERTa, and XLM-RoBERTa-XL models, along with SetFit. For the SetFit experiments, we exploited the sentence Transformer as the backbone encoder. This multilingual MPNet checkpoint provides cross-lingual sentence embeddings and was fine-tuned using SetFit's parameter-efficient contrastive learning approach, followed by training a lightweight classification head on the Sindhi few-shot dataset. Among these, the pretrained XLM-RoBERTa-XL achieves the best classification performance with an accuracy of 92.27%. The SiNFluD dataset, comprising 4451 annotated instances across five categories (literal, idiom, proverb, metaphor, and simile), provides a foundational resource for figurative language research in Sindhi and other low-resource South Asian languages.
Related
- Cross-Dataset Generalization in Phonocardiogram Murmur Classification: A Benchmark Study of Domain-Conditional Performance and the Role of Multi-Representation Fusion
- Calibration, Uncertainty, and Data Efficiency of Deep Learning Chest Radiograph Classifiers Under Cross-Dataset Distribution Shift: A Multi-Dataset Benchmark
- A Sentinel-2 benchmark dataset for deep-learning active-fire segmentation across 25 California wildfires, July 2020–August 2026
- EComReviews-BD: A benchmark dataset for aspect-based sentiment analysis on multilingual Bangladeshi E-Commerce reviews
- Comparative Evaluation of Large Language Model Interfaces in Third Molar Surgery Complication Scenarios: Response Quality, Clinical Content, Potential Clinical Risk, Readability, and Externally Observable Response Latency—A Cross-Sectional Comparative Benchmark Study
BibTeX
@article{ali2026sinflud,
title = {{SiNFluD: A novel benchmark dataset for sindhi figurative language classification}},
author = {Wazir Ali and Adeeb Noor and Saifullah Tumrani},
journal = {Data in Brief},
year = {2026},
month = aug,
doi = {10.1016/j.dib.2026.113294},
url = {https://doi.org/10.1016/j.dib.2026.113294},
note = {CC BY},
}
RIS
TY - JOUR
TI - SiNFluD: A novel benchmark dataset for sindhi figurative language classification
AU - Wazir Ali
AU - Adeeb Noor
AU - Saifullah Tumrani
PY - 2026
DA - 2026/08/01
JO - Data in Brief
DO - 10.1016/j.dib.2026.113294
UR - https://doi.org/10.1016/j.dib.2026.113294
AB - In this article, we introduce SiNFluD, a novel benchmark dataset for Sindhi figurative language classification. We collect phrases from web raw text sourced from various blogs, social media platforms, and literary sources, and subsequently prepare the corpus for annotation. Two native annotators label the data using the Doccano text annotation tool, achieving an inter-annotator agreement of 0.81 (Cohen's Kappa). We then establish baseline results using 5-fold and 10-fold cross-validation. Finally, we evaluate mBERT, XLM-RoBERTa, and XLM-RoBERTa-XL models, along with SetFit. For the SetFit experiments, we exploited the sentence Transformer as the backbone encoder. This multilingual MPNet checkpoint provides cross-lingual sentence embeddings and was fine-tuned using SetFit's parameter-efficient contrastive learning approach, followed by training a lightweight classification head on the Sindhi few-shot dataset. Among these, the pretrained XLM-RoBERTa-XL achieves the best classification performance with an accuracy of 92.27%. The SiNFluD dataset, comprising 4451 annotated instances across five categories (literal, idiom, proverb, metaphor, and simile), provides a foundational resource for figurative language research in Sindhi and other low-resource South Asian languages.
SN - 2352-3409
ER -