The Mode of Null-A: Compositional Computation of a Generalized Inverse
Abstract
We present a novel algorithm for calculating the preimage of an affine space through a product J=JT−1⋯J0 of matrices Jt of special form: finding the largest input space X such that x∈X implies Jx∈ Y, where Y is a given output affine space. These special matrices arise in AD, where the Jacobians J describing the linearized computation have precisely this structure: the product of a series of linearized primitive numeric operations. This allows us to use the new algorithm to formulate Null-A mode preimage AD, which finds the affine preimage through the Jacobian or Jacobian transpose of a numeric computation. This is a generalization of the inverse AD problem of solving Jacutex∗=acutey∗ or JTgravey∗=gravex∗. The key is to represent affine spaces in a fashion which lends itself to efficient preimage calculation, in a compositional and quasi-local fashion, through a succession of matrices Jt. Unlike previous methods, Null-A preimage mode AD allows the Jt matrices to be non-square, corresponding to a computer program whose number of active variables swells and shrinks during the computation. When J is square and the initial affine space is a single point, this finds the conventional inverse. But in the more general case, having the entire affine space provides freedom which can be leveraged in a problem-specific manner. We apply the method to small problems on-CPU where the Jt are linearized scalar unary or binary numeric functions; and to larger problems on-GPU where the Jt are linearized aggregate array operations like convolution and attention.
Related
- Efficient computation of mixture confidence sequences in generalized linear models
- The generalized core-EP inverse of operator matrices with generalized Schur complements
- Generalized Cancellation of Capacitor Parasitic Inductance Using a Lattice Network and Its Application to Common-Mode Noise Reduction
- A higher order family of iterative methods for approximating inverse and generalized inverse matrices
- Generalized [v,w]-core inverse in Banach *-algebras
BibTeX
@misc{pearlmutter2026mode,
title = {{The Mode of Null-A: Compositional Computation of a Generalized Inverse}},
author = {Barak A. Pearlmutter and Jeffrey Mark Siskind},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.38497},
eprint = {2609.38497},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.38497},
}
RIS
TY - UNPB
TI - The Mode of Null-A: Compositional Computation of a Generalized Inverse
AU - Barak A. Pearlmutter
AU - Jeffrey Mark Siskind
PY - 2026
DA - 2026/09/29
PB - arXiv
DO - 10.48550/arxiv.2609.38497
UR - https://arxiv.org/abs/2609.38497
AB - We present a novel algorithm for calculating the preimage of an affine space through a product J=J_T−1⋯J_0 of matrices J_t of special form: finding the largest input space X such that x∈X implies Jx∈ Y, where Y is a given output affine space. These special matrices arise in AD, where the Jacobians J describing the linearized computation have precisely this structure: the product of a series of linearized primitive numeric operations. This allows us to use the new algorithm to formulate Null-A mode preimage AD, which finds the affine preimage through the Jacobian or Jacobian transpose of a numeric computation. This is a generalization of the inverse AD problem of solving Jacutex^∗=acutey^∗ or J^Tgravey^∗=gravex^∗. The key is to represent affine spaces in a fashion which lends itself to efficient preimage calculation, in a compositional and quasi-local fashion, through a succession of matrices J_t. Unlike previous methods, Null-A preimage mode AD allows the J_t matrices to be non-square, corresponding to a computer program whose number of active variables swells and shrinks during the computation. When J is square and the initial affine space is a single point, this finds the conventional inverse. But in the more general case, having the entire affine space provides freedom which can be leveraged in a problem-specific manner. We apply the method to small problems on-CPU where the J_t are linearized scalar unary or binary numeric functions; and to larger problems on-GPU where the J_t are linearized aggregate array operations like convolution and attention.
ER -