REALM: Retrospective Encoder Alignment for LFP Modeling
Abstract
Spike activity has been the dominant neural signal for behavior decoding because its high spatiotemporal resolution supports accurate decoding. However, as intracortical brain-computer interfaces (iBCIs) move toward higher channel counts and wireless operation, the high sampling rates required to record spikes create substantial power and bandwidth demands. Local field potentials (LFPs) offer complementary advantages, including greater long-term stability, lower energy consumption, and lower bandwidth requirements. However, LFP-based decoders often achieve lower accuracy and rely on non-causal architectures that cannot be used directly for real-time deployment. We propose REALM, a retrospective knowledge distillation (RKD) framework for causal LFP behavior decoding. Inspired by offline-to-online distillation in speech recognition, REALM transfers non-causal representational knowledge from a pretrained, multi-session bidirectional LFP teacher to a causal student model. We first pretrain a bidirectional Mamba-2 teacher across multiple recording sessions using continuous masked autoencoding (CMAE), and then distill its representation into a compact causal student using a combined objective of representation alignment and autoencoding. REALM achieves the highest mean accuracy among the compared decoders in both label-free and fine-tuned pipelines, with statistically significant improvements over each baseline, including the state-of-the-art CrossModalDistill. It does so using LFPs alone throughout pretraining, distillation, and decoding, with less than half the parameters of CrossModalDistill's published student and one-tenth of its pretraining time. These results show that a causal LFP-only model can achieve decoding accuracy competitive with a non-causal multi-modal model, offering a practical and scalable approach for next-generation wireless and implantable iBCIs.
Related
- HSCM-Lane: A ResNet-based encoder-decoder architecture with multi-level shifted-window context modeling for pixel-wise lane segmentation
- Alignment-Aware Genomic Coordinate Encoding for Long-Range DNA Modeling
- Generative Language Modeling for Antibody CDR Grafting and Alignment-driven De Novo Design
- Efunaviria, Floreoviria, Pleomoviria, and Volvereviria : four realms of viruses with small DNA genomes to replace realm Monodnaviria
- FoldAbS: Repurposing the Protein Folding Model as a Foundation Encoder for Antibody Screening
BibTeX
@misc{wu2026realm,
title = {{REALM: Retrospective Encoder Alignment for LFP Modeling}},
author = {Peicheng Wu and Zhenyu Bu and Runze Ma and Lin Du},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2605.14867},
eprint = {2605.14867},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2605.14867},
note = {CC BY-NC-SA 4.0},
}
RIS
TY - UNPB
TI - REALM: Retrospective Encoder Alignment for LFP Modeling
AU - Peicheng Wu
AU - Zhenyu Bu
AU - Runze Ma
AU - Lin Du
PY - 2026
DA - 2026/10/27
PB - arXiv
DO - 10.48550/arxiv.2605.14867
UR - https://arxiv.org/abs/2605.14867
AB - Spike activity has been the dominant neural signal for behavior decoding because its high spatiotemporal resolution supports accurate decoding. However, as intracortical brain-computer interfaces (iBCIs) move toward higher channel counts and wireless operation, the high sampling rates required to record spikes create substantial power and bandwidth demands. Local field potentials (LFPs) offer complementary advantages, including greater long-term stability, lower energy consumption, and lower bandwidth requirements. However, LFP-based decoders often achieve lower accuracy and rely on non-causal architectures that cannot be used directly for real-time deployment. We propose REALM, a retrospective knowledge distillation (RKD) framework for causal LFP behavior decoding. Inspired by offline-to-online distillation in speech recognition, REALM transfers non-causal representational knowledge from a pretrained, multi-session bidirectional LFP teacher to a causal student model. We first pretrain a bidirectional Mamba-2 teacher across multiple recording sessions using continuous masked autoencoding (CMAE), and then distill its representation into a compact causal student using a combined objective of representation alignment and autoencoding. REALM achieves the highest mean accuracy among the compared decoders in both label-free and fine-tuned pipelines, with statistically significant improvements over each baseline, including the state-of-the-art CrossModalDistill. It does so using LFPs alone throughout pretraining, distillation, and decoding, with less than half the parameters of CrossModalDistill's published student and one-tenth of its pretraining time. These results show that a causal LFP-only model can achieve decoding accuracy competitive with a non-causal multi-modal model, offering a practical and scalable approach for next-generation wireless and implantable iBCIs.
ER -