Accuracy and Uncertainty in Machine Deception Detection (Research White Paper)
Abstract
A deception detector is only as useful as the trust its alarms deserve. Like a polygraph, detectors that read a language model's internal activations can catch deceptive responses that look honest. Yet current detectors depend on which layers they read, weaken on broader tests, and do not state how certain their alarms are. This white paper proposes the Machine Deception Detector, which targets both detection accuracy and the uncertainty of detector scores. The detector reads every layer of the language model, learns which tokens carry evidence of deception using only response labels, and calibrates its alarm threshold so that its false--positive rate stays below a prescribed level. An ensemble of such detectors measures the uncertainty of each score and raises an alarm only when its members agree, with a confidence level that sets how certain each alarm must be. The proposed evaluation trains on instructed honest and deceptive statements, tests on five unseen deceptive tasks of Liars' Bench read by six Qwen3 models from 0.6B to 32B parameters, and is designed to also show which layers carry the deception feature and how this changes with model size.
Related
- Posterior Regimes and Latent Deception: Variational Bayesian Inference in Hidden Markov Models for Sequential Fraud Detection in Financial Transactions
- When Timing Uncertainty Limits Spectral Accuracy: Bias, Variance, and Frequency-Dependent Attenuation
- The Role of Uncertainty in Assessing the Fairness of Machine Learning Models
- Machine learning-based thermoelectric materials screening with physical consistency and uncertainty propagation
- Physics-informed random neural architectures on compact boundaryless multiply connected manifolds for random-operator models in machine learning and uncertainty quantification
BibTeX
@misc{chang2026accuracy,
title = {{Accuracy and Uncertainty in Machine Deception Detection (Research White Paper)}},
author = {Heng-Sheng Chang},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.15314},
eprint = {2610.15314},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.15314},
note = {CC BY-NC-SA 4.0},
}
RIS
TY - UNPB
TI - Accuracy and Uncertainty in Machine Deception Detection (Research White Paper)
AU - Heng-Sheng Chang
PY - 2026
DA - 2026/10/11
PB - arXiv
DO - 10.48550/arxiv.2610.15314
UR - https://arxiv.org/abs/2610.15314
AB - A deception detector is only as useful as the trust its alarms deserve. Like a polygraph, detectors that read a language model's internal activations can catch deceptive responses that look honest. Yet current detectors depend on which layers they read, weaken on broader tests, and do not state how certain their alarms are. This white paper proposes the Machine Deception Detector, which targets both detection accuracy and the uncertainty of detector scores. The detector reads every layer of the language model, learns which tokens carry evidence of deception using only response labels, and calibrates its alarm threshold so that its false--positive rate stays below a prescribed level. An ensemble of such detectors measures the uncertainty of each score and raises an alarm only when its members agree, with a confidence level that sets how certain each alarm must be. The proposed evaluation trains on instructed honest and deceptive statements, tests on five unseen deceptive tasks of Liars' Bench read by six Qwen3 models from 0.6B to 32B parameters, and is designed to also show which layers carry the deception feature and how this changes with model size.
ER -