Learning Conditional Expectation Operators via Functional Newton Updates
Abstract
We introduce the Functional Spectral-Newton Method (FSNM) for learning the leading singular structure of a conditional expectation operator without fixing a basis or reproducing kernel Hilbert space. FSNM fits a low-rank representation of the centered joint-to-product density ratio kernel by alternating functional Newton updates. Each update reduces to a preconditioned regression, which we approximate with vector-valued regression trees in a stagewise boosting procedure. At the population level, we establish descent and an O(1/T) best-iterate block-stationarity rate under a relative weak-learner accuracy condition, and show that every nondegenerate local minimum over the full centered L2 spaces is a globally optimal rank-d approximation. Synthetic experiments show that FSNM recovers a low-rank density ratio and its leading spectral structure, and that the same learned kernel can answer multiple conditional queries without refitting.
Related
- Multi-Task Learning of Conditional Mean Operators: applications to dynamical systems and uncertainty quantification
- The conditional expectation of the product of the first n−1 Hermite polynomials in a multivariate normal distribution with respect to the n-th variable. A fresh perspective on the Kibble-Slepian formula
- Dimension Reduction for the Trivariate Normal Product Distribution: An Efficient Numerical Algorithm via Conditional Expectation
- A plug-in estimator of the non-iterative conditional expectation g-formula using long short-term memory models
- Continuous Distributional Characterizations via a Single Characterizing Function and Lambert W Conditional Expectation Identities
BibTeX
@misc{ramos2026learning,
title = {{Learning Conditional Expectation Operators via Functional Newton Updates}},
author = {Thiago Ramos and Alek Fröhlich and Daniel Perazzo and Massimiliano Pontil},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.35598},
eprint = {2609.35598},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.35598},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Learning Conditional Expectation Operators via Functional Newton Updates
AU - Thiago Ramos
AU - Alek Fröhlich
AU - Daniel Perazzo
AU - Massimiliano Pontil
PY - 2026
DA - 2026/09/28
PB - arXiv
DO - 10.48550/arxiv.2609.35598
UR - https://arxiv.org/abs/2609.35598
AB - We introduce the Functional Spectral-Newton Method (FSNM) for learning the leading singular structure of a conditional expectation operator without fixing a basis or reproducing kernel Hilbert space. FSNM fits a low-rank representation of the centered joint-to-product density ratio kernel by alternating functional Newton updates. Each update reduces to a preconditioned regression, which we approximate with vector-valued regression trees in a stagewise boosting procedure. At the population level, we establish descent and an O(1/T) best-iterate block-stationarity rate under a relative weak-learner accuracy condition, and show that every nondegenerate local minimum over the full centered L^2 spaces is a globally optimal rank-d approximation. Synthetic experiments show that FSNM recovers a low-rank density ratio and its leading spectral structure, and that the same learned kernel can answer multiple conditional queries without refitting.
ER -