When Machines Lie Differently: Detecting AI vs Human Fake News
Abstract
The rapid advancement of large language models has introduced AI-generated fake news alongside traditional human-written misinformation, raising questions about whether detectability depends on the source of deceptive content. This study examines that issue through two controlled binary classification tasks: distinguishing real news from human-written fake news and from AI-generated fake news. Each article is represented using features related to lexical diversity, readability, and emotional characteristics, and evaluated with several machine learning models, including logistic regression, random forests, support vector machines, gradient boosting, neural networks, and ensemble methods. Performance is measured using the area under the receiver operating characteristic curve (AUC). Across all models, AI-generated fake news is detected with near-perfect accuracy, while human-written fake news is substantially more difficult to distinguish from real news. Because both tasks use the same modeling pipeline, this performance gap reflects intrinsic statistical differences in the text rather than methodological variation. Feature-level analysis shows that AI-generated fake news exhibits more uniform readability and emotional patterns, producing less overlap with real news. These findings reveal a key asymmetry in misinformation detection: current methods may be highly effective at identifying AI-generated content but remain less reliable against sophisticated human-authored misinformation. Detection systems should therefore account for the source of misinformation and continue adapting as generative models evolve.
Related
- When Can Digital Personas Reliably Approximate Human Survey Findings?
- Learning When to Automate: Queue Control in Human-AI Service Systems
- When Can LLM Digital Twins Reduce Human Measurement? From Behavioral Fidelity to Statistical Substitutability
- Fake quadrics with bounded cohomology property
- When Should a Human Take Back Control? Optimal Delegation under Turbulent AI Risk
BibTeX
@misc{ibenye2026when,
title = {{When Machines Lie Differently: Detecting AI vs Human Fake News}},
author = {Calvin Ibenye and Aya-Vera Jimenez and Samuel Jaeger and Dhrubajyoti Ghosh},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2607.21967},
eprint = {2607.21967},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2607.21967},
note = {CC BY-NC-ND 4.0},
}
RIS
TY - UNPB
TI - When Machines Lie Differently: Detecting AI vs Human Fake News
AU - Calvin Ibenye
AU - Aya-Vera Jimenez
AU - Samuel Jaeger
AU - Dhrubajyoti Ghosh
PY - 2026
DA - 2026/09/27
PB - arXiv
DO - 10.48550/arxiv.2607.21967
UR - https://arxiv.org/abs/2607.21967
AB - The rapid advancement of large language models has introduced AI-generated fake news alongside traditional human-written misinformation, raising questions about whether detectability depends on the source of deceptive content. This study examines that issue through two controlled binary classification tasks: distinguishing real news from human-written fake news and from AI-generated fake news. Each article is represented using features related to lexical diversity, readability, and emotional characteristics, and evaluated with several machine learning models, including logistic regression, random forests, support vector machines, gradient boosting, neural networks, and ensemble methods. Performance is measured using the area under the receiver operating characteristic curve (AUC). Across all models, AI-generated fake news is detected with near-perfect accuracy, while human-written fake news is substantially more difficult to distinguish from real news. Because both tasks use the same modeling pipeline, this performance gap reflects intrinsic statistical differences in the text rather than methodological variation. Feature-level analysis shows that AI-generated fake news exhibits more uniform readability and emotional patterns, producing less overlap with real news. These findings reveal a key asymmetry in misinformation detection: current methods may be highly effective at identifying AI-generated content but remain less reliable against sophisticated human-authored misinformation. Detection systems should therefore account for the source of misinformation and continue adapting as generative models evolve.
ER -