{
  "abstract": "Large Language Models (LLMs) have advanced multilingual NLP significantly, yet their effectiveness for Bengali, a globally important but low-resource language, remains understudied due to limited linguistic resources and the absence of a standardized evaluation benchmark. This is due to the unavailability of large linguistic resources and the lack of a standard benchmarking framework, which has yet to be established. This study aims to address the underrepresentation of Bengali text classification by leading a systematic evaluation. It involves assessing multiple open-weight LLMs on Bengali text classification tasks and observing their limitations and regular error patterns. The procedure consists of six open-weight LLMs, which include LLaMA-3.2-3B, Mistral-V3-7B, DeepSeek-R1-8B, Phi-4-14B, Gemma-2-27B, and Qwen-2.5-72B, and are evaluated using four critical text classification tasks in Bengali, namely sentiment analysis, emotion recognition, hate-speech detection, and fake-news classification. Furthermore, two prompting methods are compared in the context of zero-shot and few-shot prompting using both the vLLM inference engine and Hugging Face Transformers pipelines to investigate the adaptability of the models in resource-limited settings. Across all four tasks, the two largest models (Qwen-2.5-72B and Gemma-2-27B) obtained the highest scores, although the ranking was not strictly monotonic in parameter count and, because each scale is represented by a single model family, the design cannot attribute these differences to model size alone. When training in zero-shot, Qwen-2.5-72B showed the best performance at 79.90% accuracy and 79.88% Macro-F1 score, showing amazing accuracy on hate-speech detection. In the case of fake news, Gemma-2-27B performed best at 84.45% accuracy and 84.33% Macro-F1 score. With a few-shot set-up, Gemma-2-27B and Qwen-2.5-72B demonstrated a high level of performance in terms of Macro-F1 score, reaching 84.17% for fake-news detection and 66.80% for emotion recognition, respectively. Paired multi-model tests (Cochran’s Q with Holm-corrected McNemar post-hoc comparisons) confirmed significant differences among the six models across all tasks and prompting settings, whereas the effect of five-shot prompting, obtained with a single fixed demonstration set, was significant for only 4 of 24 model–task combinations after correction and never for the three largest models. The findings characterise benchmark classification performance under the evaluated settings rather than real-world effectiveness, and indicate that model scale, multilingual pre-training, and in-context demonstrations are jointly associated with, but cannot be causally isolated as drivers of, performance on Bengali text classification.",
  "abstract_shown": true,
  "affiliations": [
    "Department of Computer Science and Engineering, East Delta University, Chattogram, Bangladesh",
    "Faculty of Engineering and Technology, Multimedia University, Melaka, Malaysia",
    "Centre for Advanced Analytics, COE for Artificial Intelligence, Faculty of Engineering and Technology (FET), Multimedia University, Melaka, Malaysia",
    "Department of Computer Science and Engineering, International Islamic University Chittagong, Chattogram, Bangladesh"
  ],
  "article_type": "Research Article",
  "arxiv": null,
  "author_total": 7,
  "authors": [
    {
      "affs": [
        0
      ],
      "name": "Md. Hamid Hosen"
    },
    {
      "affs": [
        1
      ],
      "name": "Sadia Nawar"
    },
    {
      "affs": [
        2
      ],
      "name": "Rituparna Chowdhury"
    },
    {
      "affs": [
        3
      ],
      "name": "Altaf Uddin"
    },
    {
      "name": "Riadul Islam Rabbi"
    },
    {
      "name": "Khair Razlan Othman"
    },
    {
      "name": "Nurul Karim Symon"
    }
  ],
  "bibtex": "https://research.draconitic.com/paper/6ssmfhwpzreg0.bib",
  "cited_by": 0,
  "doi": "10.1371/journal.pone.0361368",
  "fulltext_url": null,
  "id": "6ssmfhwpzreg0",
  "issns": [
    "1932-6203"
  ],
  "landing_url": "https://doi.org/10.1371/journal.pone.0361368",
  "language": null,
  "licence": "CC-BY-4.0",
  "licence_label": "CC BY 4.0",
  "licence_source": "journal",
  "licence_url": "https://creativecommons.org/licenses/by/4.0/",
  "metadata_licence": null,
  "native_subjects": [
    [
      "openalex",
      "1702|Topic Modeling"
    ],
    [
      "plos",
      "/Biology and life sciences/Neuroscience/Cognitive science/Cognitive psychology/Language"
    ],
    [
      "plos",
      "/Biology and life sciences/Neuroscience/Cognitive science/Cognitive psychology/Language/Multilingualism"
    ],
    [
      "plos",
      "/Biology and life sciences/Psychology/Cognitive psychology/Language"
    ],
    [
      "plos",
      "/Biology and life sciences/Psychology/Cognitive psychology/Language/Multilingualism"
    ],
    [
      "plos",
      "/Biology and life sciences/Psychology/Emotions"
    ],
    [
      "plos",
      "/Computer and information sciences/Information technology/Natural language processing"
    ],
    [
      "plos",
      "/People and places/Population groupings/Ethnicities/Asian people/Bengali people"
    ],
    [
      "plos",
      "/Physical sciences/Mathematics/Discrete mathematics/Combinatorics/Permutation"
    ],
    [
      "plos",
      "/Social sciences/Linguistics/Grammar/Syntax"
    ],
    [
      "plos",
      "/Social sciences/Linguistics/Speech"
    ],
    [
      "plos",
      "/Social sciences/Psychology/Cognitive psychology/Language"
    ],
    [
      "plos",
      "/Social sciences/Psychology/Cognitive psychology/Language/Multilingualism"
    ],
    [
      "plos",
      "/Social sciences/Psychology/Emotions"
    ]
  ],
  "openalex": "W7225396619",
  "pdf_url": null,
  "pmcid": null,
  "pmid": null,
  "published": "2026-11-06",
  "publisher": "Public Library of Science",
  "ris": "https://research.draconitic.com/paper/6ssmfhwpzreg0.ris",
  "sources": [
    {
      "first_seen": "2026-11-07T12:20:24Z",
      "native_id": "10.1371/journal.pone.0361368",
      "source": "plos",
      "url": "https://doi.org/10.1371/journal.pone.0361368"
    }
  ],
  "subjects": [
    {
      "field": "biology",
      "origin": "plos:neuroscience",
      "subfield": "neuroscience"
    },
    {
      "field": "mathematics",
      "origin": "plos:combinatorics",
      "subfield": "combinatorics"
    }
  ],
  "title": "Evaluating the effectiveness of open-weight LLMs for multi-domain Bengali text classification with zero-shot and few-shot prompting",
  "title_text": "Evaluating the effectiveness of open-weight LLMs for multi-domain Bengali text classification with zero-shot and few-shot prompting",
  "type": "peer-reviewed",
  "updated_at": "2026-11-08T12:20:48Z",
  "url": "https://research.draconitic.com/paper/6ssmfhwpzreg0",
  "venue": "PLOS One",
  "versions": []
}