The Cross-Substrate Access Assay: What an Indicator Test Must Declare to Travel from Brain to Language Model
Abstract
Brain tests of consciousness are often applied to large language models (LLMs). Here consciousness means access: whether information, such as a heard sound, becomes available for report and use. A brain recording is in microvolts and an LLM's internal activations are arbitrary numbers, so the two share no common measurement unit. A brain test of this kind compares competing statistical models of how responses vary over repeated trials. The test's decision names the best-fitting model. The decision is wrong when the test names two states for responses that vary smoothly, or the reverse. No method we reviewed measures how often a transferred test is wrong. We call such a method a transfer audit and propose one, the cross-entropy audit (CEA). First, our audit holds back part of the trials and fits each model to the remaining trials. It then scores how well each fitted model predicts the held-back responses. Comparing two models gives a score difference in nats per trial. That difference does not depend on whether the recording is in microvolts or in activations, so brain and machine are scored on one scale. Second, the audit writes down five choices that such tests make for brain data: the competing models, their fitting, what counts as one independent sample, what the uncertainty interval describes, and the decision rule. Each may need to change on a machine. Third, the audit runs the test on simulated data built to follow one model, so the right decision is known, and counts wrong decisions. In our example, the two models of a two-state test, carried over unchanged to an LLM, found two states in 989 of 12,000 smoothly varying simulated datasets. The audited version of the test found two states in none of them. It detected 599 of 600 two-state datasets. The audited version's uncertainty interval, however, was too narrow at six of the twelve smoothly varying settings and is being repaired.
Related
- High-throughput mapping of kinase-substrate interactions with protein language models and phospho-PCA
- The development of a technologic approach to improve access to individualized clinical documentation in caregivers’ preferred language
- Mapping the neural basis of the perception of “flow of time”: interval timing and mental time travel in the human brain
- Evaluation of the Biosynex AMPLIQUICK® Fecal Parasitology assays in a Belgian travel clinic.
- Root Causes and Prevention of Mobility Device Damage During Air Travel: A Rapid Review
BibTeX
@misc{rooyen2026crosssubstrate,
title = {{The Cross-Substrate Access Assay: What an Indicator Test Must Declare to Travel from Brain to Language Model}},
author = {Pieter van Rooyen},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.22300},
eprint = {2609.22300},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.22300},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - The Cross-Substrate Access Assay: What an Indicator Test Must Declare to Travel from Brain to Language Model
AU - Pieter van Rooyen
PY - 2026
DA - 2026/09/22
PB - arXiv
DO - 10.48550/arxiv.2609.22300
UR - https://arxiv.org/abs/2609.22300
AB - Brain tests of consciousness are often applied to large language models (LLMs). Here consciousness means access: whether information, such as a heard sound, becomes available for report and use. A brain recording is in microvolts and an LLM's internal activations are arbitrary numbers, so the two share no common measurement unit. A brain test of this kind compares competing statistical models of how responses vary over repeated trials. The test's decision names the best-fitting model. The decision is wrong when the test names two states for responses that vary smoothly, or the reverse. No method we reviewed measures how often a transferred test is wrong. We call such a method a transfer audit and propose one, the cross-entropy audit (CEA). First, our audit holds back part of the trials and fits each model to the remaining trials. It then scores how well each fitted model predicts the held-back responses. Comparing two models gives a score difference in nats per trial. That difference does not depend on whether the recording is in microvolts or in activations, so brain and machine are scored on one scale. Second, the audit writes down five choices that such tests make for brain data: the competing models, their fitting, what counts as one independent sample, what the uncertainty interval describes, and the decision rule. Each may need to change on a machine. Third, the audit runs the test on simulated data built to follow one model, so the right decision is known, and counts wrong decisions. In our example, the two models of a two-state test, carried over unchanged to an LLM, found two states in 989 of 12,000 smoothly varying simulated datasets. The audited version of the test found two states in none of them. It detected 599 of 600 two-state datasets. The audited version's uncertainty interval, however, was too narrow at six of the twelve smoothly varying settings and is being repaired.
ER -