Complex-Plane-Inspired Magnitude Rectifier: An Exploratory Transformer Activation Study
Abstract
This exploratory preprint studies a complex-plane-inspired magnitude rectifier (CIMR), a real-valued activation that retains negative-input magnitude through a beta-scaled orthogonal construction. The scalar form reduces to a smoothed asymmetric absolute-value rectifier; it is not a genuinely complex-valued neural network.
Local experiments provide mixed evidence. Fixed and scalar-learned variants do not consistently outperform ReLU, while an input-conditioned AttentionBeta variant performs best on one modular synthetic task but uses about 24.6% more parameters, so the result is capacity-confounded. The report identifies the experiments as exploratory and specifies parameter-matched, held-out follow-up tests.
Status: non-peer-reviewed exploratory technical report.
AI assistance disclosure: ChatGPT and Sarvam AI assisted with brainstorming and language development. ChatGPT also assisted with reconstructing project history, auditing local code and results, literature triage, and drafting. AI systems are not authors; Chaman Prakash Kanth remains responsible for the manuscript and public metadata.
Related
- Transformer-inspired architectures make machine-learned electron densities accurate and transferable enough to bypass self-consistency
- QiT: Quantum-Inspired Transformer for Visual Recognition Task
- Adaptive Signed Transformations for Budgeted Transformer Memory: Learned Activation Routing, CReLU Sign Preservation, and Exponential Retention
- Testing a dual-process-inspired questionnaire-primary residual framework for personality prediction: an exploratory study with static line drawings.
- Stress entropic load as an exploratory thermodynamic indicator of physiological activation during a laboratory gaming task
BibTeX
@misc{kanth2026complexplaneinspired,
title = {{Complex-Plane-Inspired Magnitude Rectifier: An Exploratory Transformer Activation Study}},
author = {Chaman Prakash Kanth},
howpublished = {Zenodo},
year = {2026},
month = oct,
doi = {10.5281/zenodo.23623393},
url = {https://zenodo.org/records/23623393},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Complex-Plane-Inspired Magnitude Rectifier: An Exploratory Transformer Activation Study
AU - Chaman Prakash Kanth
PY - 2026
DA - 2026/10/24
PB - Zenodo
DO - 10.5281/zenodo.23623393
UR - https://zenodo.org/records/23623393
AB - This exploratory preprint studies a complex-plane-inspired magnitude rectifier (CIMR), a real-valued activation that retains negative-input magnitude through a beta-scaled orthogonal construction. The scalar form reduces to a smoothed asymmetric absolute-value rectifier; it is not a genuinely complex-valued neural network. Local experiments provide mixed evidence. Fixed and scalar-learned variants do not consistently outperform ReLU, while an input-conditioned AttentionBeta variant performs best on one modular synthetic task but uses about 24.6% more parameters, so the result is capacity-confounded. The report identifies the experiments as exploratory and specifies parameter-matched, held-out follow-up tests. Status: non-peer-reviewed exploratory technical report. AI assistance disclosure: ChatGPT and Sarvam AI assisted with brainstorming and language development. ChatGPT also assisted with reconstructing project history, auditing local code and results, literature triage, and drafting. AI systems are not authors; Chaman Prakash Kanth remains responsible for the manuscript and public metadata.
ER -