Selection, Representation, and Execution in Sparse Fourier Neural Operators
Abstract
Sparse representations are often expected to make models smaller and also reduce inference cost. For Fourier Neural Operators (FNOs), these objectives are not equivalent or do not always align: removing parts of the learned operator can leave the underlying transforms and dense computations unchanged, while changing the grid on which the model is evaluated can introduce overhead of its own. We therefore distinguish sparsity in the representation, in the stored parameters, in the theoretical operation count, and in measured runtime, and present an empirical study of several routes toward sparse FNOs that tests each transition between them separately. Coarsening the execution grid reduces the theoretical cost without reducing measured latency, and adding a correction term recovers accuracy at the cost of making the model slower. Even an 83\% parameter reduction remains slower than the dense baseline under ordinary execution. These results motivate a stricter definition of useful sparsity: the deployed operator must preserve solution accuracy and map its reduced support to a genuinely cheaper execution path.
Related
- Optimal Extrapolation Bounds for Sparse Fourier Sums
- Fourier decay and endpoint estimates for lacunary maximal operators
- Sparse bounds for rough non-translation-invariant Calderón-Zygmund operators
- Singular Legendre operators on graphs: trace formulas and Fourier singular support
- Training-Adaptive Convolutional Sparse Coding via Information Bottleneck for Robust Visual Representation
BibTeX
@misc{ibrahim2026selection,
title = {{Selection, Representation, and Execution in Sparse Fourier Neural Operators}},
author = {Abdul Qadir Ibrahim and Martin Burger},
howpublished = {arXiv},
year = {2026},
month = aug,
doi = {10.48550/arxiv.2608.30070},
eprint = {2608.30070},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2608.30070},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Selection, Representation, and Execution in Sparse Fourier Neural Operators
AU - Abdul Qadir Ibrahim
AU - Martin Burger
PY - 2026
DA - 2026/08/30
PB - arXiv
DO - 10.48550/arxiv.2608.30070
UR - https://arxiv.org/abs/2608.30070
AB - Sparse representations are often expected to make models smaller and also reduce inference cost. For Fourier Neural Operators (FNOs), these objectives are not equivalent or do not always align: removing parts of the learned operator can leave the underlying transforms and dense computations unchanged, while changing the grid on which the model is evaluated can introduce overhead of its own. We therefore distinguish sparsity in the representation, in the stored parameters, in the theoretical operation count, and in measured runtime, and present an empirical study of several routes toward sparse FNOs that tests each transition between them separately. Coarsening the execution grid reduces the theoretical cost without reducing measured latency, and adding a correction term recovers accuracy at the cost of making the model slower. Even an 83\% parameter reduction remains slower than the dense baseline under ordinary execution. These results motivate a stricter definition of useful sparsity: the deployed operator must preserve solution accuracy and map its reduced support to a genuinely cheaper execution path.
ER -