FOCUS: An Adaptive Fractional Graph-Diffusion Attention Operator, with a Preliminary Hallucination-Warning Application
- 1Computer Science and Engineering Department, College of Applied Studies, King Saud University, Riyadh 11437, Saudi Arabia
Abstract
Hallucination in autoregressive language models arises when generation dynamics enter a locally stable but factually unsupported attractor. We propose FOCUS (Fractional Order Confidence for Uncertainty Signalling), a framework that replaces the fixed-geometry softmax attention kernel with the adaptive fractional graph-diffusion operator Aαl,h,t=exp(−τL^αl,h,t), where L^=I−D^−1/2P^D^−1/2 is the symmetrised normalised attention Laplacian of the causal prefix graph of each generation step (spectrum in [0,2], proved without assuming double stochasticity of P^) and αl,h,t∈(0,1) is a differentiable control variable predicted per layer l, head h, and generation step t. The key mathematical result is that for μk<1 (low-frequency modes), higher αl,h,t leads to less attenuation (more energy passes through), while for μk>1 (high-frequency modes), higher αl,h,t leads to more attenuation. By assuming that grounded evidence is low-frequency on the attention graph, high αl,h,t is thus advantageous in both spectral regimes simultaneously. The per-head specialisation of αl,h,t arises not from a static spectral tradeoff within a single head but from a training-dynamics competition between the LM loss (which rewards structural diversity across heads, including low-αl,h,t heads that pass high-frequency syntactic features) and the hallucination penalty. Because this penalty pushes αl,h,t down at hallucinated tokens, it trains αl,h,t as an indicator of hallucination rather than as a protective mechanism, and we state what this implies for the interpretation of the experiments. We prove the complete spectral theory, the causality of the construction, and the Fréchet derivative, along the admissible score-induced family of prefix Laplacians for which the zero eigenvalue stays structurally pinned, of the composite map L^↦exp(−τL^α), via the Daleckii–Kreĭn formula with a spectral-gap-dependent bound, and provide a direct Lanczos strategy for computing exp(−τL^α)v with measured error and cost. The warning mechanism is the spectral imbalance statistic It=α¯tctx/(α¯tnoise+ε), whose drop precedes hallucination onset by 6–7 tokens on hallucinating sequences in preliminary synthetic experiments. We also identify spectral localisation analysis—verifying that hallucination spans concentrate in high-frequency modes of the attention graph—as the critical empirical validation for the framework. The evidence assembled here supports FOCUS as a well-founded spectral operator and a preliminary hallucination-warning statistic; establishing it as a protective decoding mechanism, as the acronym’s original reading suggested, is left as an open empirical target.
Related
- Subdivision-based collocation methods for time fractional convection diffusion reaction equation with environmental and biomedical applications
- Stochastic fractional diffusion equations with spatial Gaussian noise in the Stratonovich regime
- Recovering the Fractional Order and an Initial Value Simultaneously in a Time-Fractional Diffusion-Wave Equation
- Optimal control of fractional diffusion with Dirac measures
- Large deviations, gradient flows and fluctuating hydrodynamics for fractional diffusion
BibTeX
@article{souayah2026focus,
title = {{FOCUS: An Adaptive Fractional Graph-Diffusion Attention Operator, with a Preliminary Hallucination-Warning Application}},
author = {Nizar Souayah},
journal = {Fractal and Fractional},
year = {2026},
month = oct,
doi = {10.3390/fractalfract10100717},
url = {https://www.mdpi.com/2504-3110/10/10/717},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - FOCUS: An Adaptive Fractional Graph-Diffusion Attention Operator, with a Preliminary Hallucination-Warning Application
AU - Nizar Souayah
PY - 2026
DA - 2026/10/13
JO - Fractal and Fractional
DO - 10.3390/fractalfract10100717
UR - https://www.mdpi.com/2504-3110/10/10/717
AB - Hallucination in autoregressive language models arises when generation dynamics enter a locally stable but factually unsupported attractor. We propose FOCUS (Fractional Order Confidence for Uncertainty Signalling), a framework that replaces the fixed-geometry softmax attention kernel with the adaptive fractional graph-diffusion operator Aαl,h,t=exp(−τL^αl,h,t), where L^=I−D^−1/2P^D^−1/2 is the symmetrised normalised attention Laplacian of the causal prefix graph of each generation step (spectrum in [0,2], proved without assuming double stochasticity of P^) and αl,h,t∈(0,1) is a differentiable control variable predicted per layer l, head h, and generation step t. The key mathematical result is that for μk<1 (low-frequency modes), higher αl,h,t leads to less attenuation (more energy passes through), while for μk>1 (high-frequency modes), higher αl,h,t leads to more attenuation. By assuming that grounded evidence is low-frequency on the attention graph, high αl,h,t is thus advantageous in both spectral regimes simultaneously. The per-head specialisation of αl,h,t arises not from a static spectral tradeoff within a single head but from a training-dynamics competition between the LM loss (which rewards structural diversity across heads, including low-αl,h,t heads that pass high-frequency syntactic features) and the hallucination penalty. Because this penalty pushes αl,h,t down at hallucinated tokens, it trains αl,h,t as an indicator of hallucination rather than as a protective mechanism, and we state what this implies for the interpretation of the experiments. We prove the complete spectral theory, the causality of the construction, and the Fréchet derivative, along the admissible score-induced family of prefix Laplacians for which the zero eigenvalue stays structurally pinned, of the composite map L^↦exp(−τL^α), via the Daleckii–Kreĭn formula with a spectral-gap-dependent bound, and provide a direct Lanczos strategy for computing exp(−τL^α)v with measured error and cost. The warning mechanism is the spectral imbalance statistic It=α¯tctx/(α¯tnoise+ε), whose drop precedes hallucination onset by 6–7 tokens on hallucinating sequences in preliminary synthetic experiments. We also identify spectral localisation analysis—verifying that hallucination spans concentrate in high-frequency modes of the attention graph—as the critical empirical validation for the framework. The evidence assembled here supports FOCUS as a well-founded spectral operator and a preliminary hallucination-warning statistic; establishing it as a protective decoding mechanism, as the acronym’s original reading suggested, is left as an open empirical target.
SN - 2504-3110
ER -