MinkowskiPE: Minkowski Positional Encoding for Spatiotemporal Perception
Abstract
Modeling spatiotemporal coupling is a key challenge in building physical intelligence across scales, from microscopic to macroscopic. Existing models capture such structure broadly through physics-motivated dynamical formulations or learning-motivated architectures. The former provide stronger priors but may constrain flexibility, whereas the latter are more flexible but leave the spatiotemporal coupling largely implicit. We therefore seek an approach that combines flexible learning with an explicit geometric bias for jointly modeling time and space. To this end, we propose Minkowski Positional Encoding (MinkowskiPE), which uses joint temporal and spatial coordinates to parameterize Lorentz transformations applied to query and key features. With MinkowskiPE, the query-key attention score depends on position only through the relative spacetime displacement between the two tokens and is therefore invariant to global translation of the coordinates. This paradigm retains the standard dot-product attention interface and remains compatible with efficient attention implementations. We evaluate MinkowskiPE on microscopic molecular dynamics and macroscopic video prediction tasks, achieving the best results on all nine multi-trajectory molecular evaluations and reducing KTH video-prediction MSE by 9.9% relative to the best baseline while using roughly one-tenth as many parameters.
Related
- Spin disorder competing with positional symmetry breaking governs the metal-insulator behavior in oxide paramagnets
- Optimal-work feedback on particles with activity --- gliding on active fluctuations using positional information
- Far Tails of the Positional Distribution in random walk: A Saddle-Point Approach
- Regular hairy black holes by gravitational decoupling: de Sitter and Minkowski cores
- Enhanced photoinduced mass migration in supramolecular azopolymers by H-bond driven positional constraint
BibTeX
@misc{li2026minkowskipe,
title = {{MinkowskiPE: Minkowski Positional Encoding for Spatiotemporal Perception}},
author = {Yuhao Li and Louie Hong Yao and Tianyi Shi and Hanqun Cao and Hongxia Hao and Zhen Zhao and Shengchao Liu},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.33804},
eprint = {2609.33804},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.33804},
note = {CC BY-NC-SA 4.0},
}
RIS
TY - UNPB
TI - MinkowskiPE: Minkowski Positional Encoding for Spatiotemporal Perception
AU - Yuhao Li
AU - Louie Hong Yao
AU - Tianyi Shi
AU - Hanqun Cao
AU - Hongxia Hao
AU - Zhen Zhao
AU - Shengchao Liu
PY - 2026
DA - 2026/09/27
PB - arXiv
DO - 10.48550/arxiv.2609.33804
UR - https://arxiv.org/abs/2609.33804
AB - Modeling spatiotemporal coupling is a key challenge in building physical intelligence across scales, from microscopic to macroscopic. Existing models capture such structure broadly through physics-motivated dynamical formulations or learning-motivated architectures. The former provide stronger priors but may constrain flexibility, whereas the latter are more flexible but leave the spatiotemporal coupling largely implicit. We therefore seek an approach that combines flexible learning with an explicit geometric bias for jointly modeling time and space. To this end, we propose Minkowski Positional Encoding (MinkowskiPE), which uses joint temporal and spatial coordinates to parameterize Lorentz transformations applied to query and key features. With MinkowskiPE, the query-key attention score depends on position only through the relative spacetime displacement between the two tokens and is therefore invariant to global translation of the coordinates. This paradigm retains the standard dot-product attention interface and remains compatible with efficient attention implementations. We evaluate MinkowskiPE on microscopic molecular dynamics and macroscopic video prediction tasks, achieving the best results on all nine multi-trajectory molecular evaluations and reducing KTH video-prediction MSE by 9.9% relative to the best baseline while using roughly one-tenth as many parameters.
ER -