URCHIN: A Horizontal Spiking Language Model for Data-Constrained Pretraining
Abstract
The BabyLM challenge measures how much language a model can learn from developmentally-plausible, child-scale data rather than internet-scale corpora, yet prior language models forgo the biological constraints of the neural circuitry that acquires human language: spiking neurons separated into excitatory and inhibitory populations wired by a recurrent lateral connectome. This paper presents URCHIN (Unified Recurrent Connectome with Horizontal Integrate-and-fire Neurons), which applies the Parallelized Hierarchical Connectome Spiking State-space Model (PHCSSM) to language modeling: leaky integrate-and-fire neurons coupled by a Dale's-law lateral connectome resolve each token through a multi-transmission loop that recirculates activity to a fixed point. The instantiation is deliberately minimal: a single horizontal layer of 128 neurons, no attention, and 4.23M parameters. Two implementations share one set of weights and produce identical benchmark scores, so URCHIN is trained once and deployed either way with no conversion step: a parallel state-space model (SSM) scan that is GPU-efficient for training, or an event-driven recurrent spiking neural network (RSNN) with constant-cost inference for CPU or neuromorphic edge deployment. Across all three BabyLM tracks (Strict-100M, Strict-Small, and Multilingual), URCHIN offers a biologically plausible, efficient, and directly deployable reference point.
Related
- PhysBridge: Response-supervised grid pretraining with physics-constrained local correction for transferable power-system intelligence
- What does domain-specific pretraining buy? A 170,000-parameter plant peptide language model against the ESM-2 scaling curve
- Global proteomic analysis of sea urchin embryos reveals dynamic changes throughout embryogenesis
- Larval thermal tolerances and future dispersal potential of the subtropical-to-temperate sea urchin Tripneustes australiae
- pxGPT: a schema-constrained multimodal large language model framework for plant phenotyping
BibTeX
@misc{chiang2026urchin,
title = {{URCHIN: A Horizontal Spiking Language Model for Data-Constrained Pretraining}},
author = {Po-Han Chiang},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.13899},
eprint = {2609.13899},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.13899},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - URCHIN: A Horizontal Spiking Language Model for Data-Constrained Pretraining
AU - Po-Han Chiang
PY - 2026
DA - 2026/09/12
PB - arXiv
DO - 10.48550/arxiv.2609.13899
UR - https://arxiv.org/abs/2609.13899
AB - The BabyLM challenge measures how much language a model can learn from developmentally-plausible, child-scale data rather than internet-scale corpora, yet prior language models forgo the biological constraints of the neural circuitry that acquires human language: spiking neurons separated into excitatory and inhibitory populations wired by a recurrent lateral connectome. This paper presents URCHIN (Unified Recurrent Connectome with Horizontal Integrate-and-fire Neurons), which applies the Parallelized Hierarchical Connectome Spiking State-space Model (PHCSSM) to language modeling: leaky integrate-and-fire neurons coupled by a Dale's-law lateral connectome resolve each token through a multi-transmission loop that recirculates activity to a fixed point. The instantiation is deliberately minimal: a single horizontal layer of 128 neurons, no attention, and 4.23M parameters. Two implementations share one set of weights and produce identical benchmark scores, so URCHIN is trained once and deployed either way with no conversion step: a parallel state-space model (SSM) scan that is GPU-efficient for training, or an event-driven recurrent spiking neural network (RSNN) with constant-cost inference for CPU or neuromorphic edge deployment. Across all three BabyLM tracks (Strict-100M, Strict-Small, and Multilingual), URCHIN offers a biologically plausible, efficient, and directly deployable reference point.
ER -