SiBot: A hybrid framework for fine-grained semantic attribution of AI-generated text and its production deployment
- 1Department of Computer Science and Engineering, Dhaka University of Engineering and Technology, Gazipur, Bangladesh
Abstract
Large language models such as ChatGPT, Gemini, Claude and DeepSeek now generate text that is routinely indistinguishable from human writing, yet virtually every deployed detector answers only one binary question: human or AI. This is the wrong question for the settings that matter most. Digital forensics needs to know which model produced a text; academic-integrity adjudication needs to know whether a human was involved at all; and neither can be answered by a single probability, least of all for the hybrid human-AI drafts that now dominate real-world writing. This paper addresses that gap with SiBot, an end-to-end framework that reformulates AI-text detection as fine-grained multi-class source attribution and carries it through to public deployment. We construct two purpose-built corpora using a controlled bank of 3,000 prompts spanning six subject domains and five task types: a 66,000-sample corpus covering 22 AI sources, and a 30,000-sample corpus covering 10 classes that explicitly models five pure sources and four human-AI collaboration workflows; both are publicly released as the TXD-22 benchmark dataset. On the 22-class corpus we benchmark twelve feature-classifier combinations and find that TF-IDF with Random Forest attains 68.2% accuracy and an AUC of 0.742. On the 10-class corpus we benchmark fifteen classical, deep-learning and fine-tuned transformer baselines, and propose a hybrid architecture that fuses sparse TF-IDF lexical features with DeBERTa contextual embeddings at the feature level before Random Forest classification. The proposed hybrid attains 92.38% accuracy, 92.41% precision, 92.38% recall and 92.31% F1 under prompt-aware 10-fold cross-validation, a 17.73-point gain over the strongest fine-tuned transformer (DeBERTa, 74.65%) and a 34.30-point gain over TF-IDF with Random Forest alone, with non-overlapping 95% confidence intervals. An ablation confirms that both branches are necessary, and SHAP and LIME analyses expose the lexical evidence behind individual decisions. Because the 10-class hybrid is both substantially more accurate and interpretable, it is the model served by the live SiBot platform; the 22-class model is reported here as a research benchmark, and extending the deployment to all 22 classes is left to future work. In a controlled three-way benchmark against QuillBot, Turnitin, ZeroGPT and TextGuard, SiBot is the only system that names the generating model of AI text (Gemini, 96.7% confidence) and the only one that recognises hybrid provenance (97.7% confidence, against verdicts ranging from 0% to 76% among the commercial tools on the identical input).
Related
- Analytical framework for computer-generated holography of screen space primitives with point-source integration in the Fourier domain
- QTestBench: A Test Framework for Evaluating LLM-Generated Quantum Code
- The transverse-spin asymmetry AsinϕSΛUUT in Λ production in SIDIS within the collinear framework
- Forward χc production in pp and pA collisions within the CGC+NRQCD framework
- Deployment of Large IQP Circuit Born Machines on Qubit-Limited Hardware Using Noise Injection
BibTeX
@article{iqbal2026sibot,
title = {{SiBot: A hybrid framework for fine-grained semantic attribution of AI-generated text and its production deployment}},
author = {Md. Sadiq Iqbal and Mohammod Abul Kashem},
journal = {PLOS One},
year = {2026},
month = sep,
doi = {10.1371/journal.pone.0357405},
url = {https://doi.org/10.1371/journal.pone.0357405},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - SiBot: A hybrid framework for fine-grained semantic attribution of AI-generated text and its production deployment
AU - Md. Sadiq Iqbal
AU - Mohammod Abul Kashem
PY - 2026
DA - 2026/09/08
JO - PLOS One
DO - 10.1371/journal.pone.0357405
UR - https://doi.org/10.1371/journal.pone.0357405
AB - Large language models such as ChatGPT, Gemini, Claude and DeepSeek now generate text that is routinely indistinguishable from human writing, yet virtually every deployed detector answers only one binary question: human or AI. This is the wrong question for the settings that matter most. Digital forensics needs to know which model produced a text; academic-integrity adjudication needs to know whether a human was involved at all; and neither can be answered by a single probability, least of all for the hybrid human-AI drafts that now dominate real-world writing. This paper addresses that gap with SiBot, an end-to-end framework that reformulates AI-text detection as fine-grained multi-class source attribution and carries it through to public deployment. We construct two purpose-built corpora using a controlled bank of 3,000 prompts spanning six subject domains and five task types: a 66,000-sample corpus covering 22 AI sources, and a 30,000-sample corpus covering 10 classes that explicitly models five pure sources and four human-AI collaboration workflows; both are publicly released as the TXD-22 benchmark dataset. On the 22-class corpus we benchmark twelve feature-classifier combinations and find that TF-IDF with Random Forest attains 68.2% accuracy and an AUC of 0.742. On the 10-class corpus we benchmark fifteen classical, deep-learning and fine-tuned transformer baselines, and propose a hybrid architecture that fuses sparse TF-IDF lexical features with DeBERTa contextual embeddings at the feature level before Random Forest classification. The proposed hybrid attains 92.38% accuracy, 92.41% precision, 92.38% recall and 92.31% F1 under prompt-aware 10-fold cross-validation, a 17.73-point gain over the strongest fine-tuned transformer (DeBERTa, 74.65%) and a 34.30-point gain over TF-IDF with Random Forest alone, with non-overlapping 95% confidence intervals. An ablation confirms that both branches are necessary, and SHAP and LIME analyses expose the lexical evidence behind individual decisions. Because the 10-class hybrid is both substantially more accurate and interpretable, it is the model served by the live SiBot platform; the 22-class model is reported here as a research benchmark, and extending the deployment to all 22 classes is left to future work. In a controlled three-way benchmark against QuillBot, Turnitin, ZeroGPT and TextGuard, SiBot is the only system that names the generating model of AI text (Gemini, 96.7% confidence) and the only one that recognises hybrid provenance (97.7% confidence, against verdicts ranging from 0% to 76% among the commercial tools on the identical input).
SN - 1932-6203
ER -