Evaluating parameter-efficient fine-tuning of large language models for sentiment analysis with bayesian optimization
- 1Department of Information Technology, College of Computer, Qassim University
Abstract
The increasing volume of user-generated reviews and feedback available in digital format provides valuable insights across various domains. Sentiment analysis plays a crucial role in extracting opinions from textual data. Large Language Models (LLMs) have demonstrated exceptional performance in numerous Natural Language Processing tasks; however, fine-tuning these models remains resource-intensive and time-consuming. Parameter-Efficient Fine-Tuning (PEFT) techniques offer a more computationally efficient alternative. This study investigates the effectiveness of Low-Rank Adaptation (LoRA) for fine-tuning LLMs in Arabic sentiment analysis by evaluating both sequence classification and instruction-tuning paradigms across the ASTD, LABR, and HARD datasets. The results show that LLaMA-3-8B-Instruct achieves competitive performance compared to LLaMA-3-8B with a classification head. Moreover, SILMA-9B-Instruct shows the highest overall performance, achieving F1-scores of 88.95%, 89.91%, and 96.55% on ASTD, LABR, and HARD respectively. In addition, Bayesian hyperparameter optimization is employed to find optimal LoRA configurations by exploring rank, dropout, and learning rate so that maximum validation scores can be achieved with respect to the F1-score for the entire model.
Related
- The ClimaSentAR dataset for evaluating cross-domain transferability of large language models in climate sentiment analysis
- Sentiment Analysis of Clinical Interviews Identifies Language Markers of Identity Diffusion in Adolescents with Personality Disorder
- Evaluating Large Language Models as Tools to Navigate Researchers in Rapidly Evolving Research Landscapes: A Case Study in Cancer Drug Response Prediction
- Evaluating a Large Language Model for Emergency Triage Using Synthetic Clinical Dialogues: Toward a Multimodal Clinical Safety Framework
- Reply to: "Evaluating the Clinical Readiness of General-Purpose Large Language Models in Evidence Synthesis".
BibTeX
@article{alawaji2026evaluating,
title = {{Evaluating parameter-efficient fine-tuning of large language models for sentiment analysis with bayesian optimization}},
author = {Raghad Alawaji and Shuaa Alharbi and Haifa Alhasson and Abdulrahman Aloraini},
journal = {Scientific Reports},
year = {2026},
month = sep,
doi = {10.1038/s41598-026-69503-5},
url = {https://doi.org/10.1038/s41598-026-69503-5},
note = {CC BY},
}
RIS
TY - JOUR
TI - Evaluating parameter-efficient fine-tuning of large language models for sentiment analysis with bayesian optimization
AU - Raghad Alawaji
AU - Shuaa Alharbi
AU - Haifa Alhasson
AU - Abdulrahman Aloraini
PY - 2026
DA - 2026/09/01
JO - Scientific Reports
DO - 10.1038/s41598-026-69503-5
UR - https://doi.org/10.1038/s41598-026-69503-5
AB - The increasing volume of user-generated reviews and feedback available in digital format provides valuable insights across various domains. Sentiment analysis plays a crucial role in extracting opinions from textual data. Large Language Models (LLMs) have demonstrated exceptional performance in numerous Natural Language Processing tasks; however, fine-tuning these models remains resource-intensive and time-consuming. Parameter-Efficient Fine-Tuning (PEFT) techniques offer a more computationally efficient alternative. This study investigates the effectiveness of Low-Rank Adaptation (LoRA) for fine-tuning LLMs in Arabic sentiment analysis by evaluating both sequence classification and instruction-tuning paradigms across the ASTD, LABR, and HARD datasets. The results show that LLaMA-3-8B-Instruct achieves competitive performance compared to LLaMA-3-8B with a classification head. Moreover, SILMA-9B-Instruct shows the highest overall performance, achieving F1-scores of 88.95%, 89.91%, and 96.55% on ASTD, LABR, and HARD respectively. In addition, Bayesian hyperparameter optimization is employed to find optimal LoRA configurations by exploring rank, dropout, and learning rate so that maximum validation scores can be achieved with respect to the F1-score for the entire model.
SN - 2045-2322
ER -