Direct Message Approximation (DMA): A Consistency-Based Framework for Tractable Approximate Inference on…

Abstract

Approximate message passing on factor graphs underlies two dominant families of probabilistic inference algorithms: expectation propagation (EP) and variational message passing (VMP). Both methods approximate the marginal at each factor edge, forcing an iterative round-robin schedule, risking negative-precision messages, and, for VMP, collapsing to point estimates at Dirac-delta factors. We introduce Direct Message Approximation (DMA), which approximates factor-to-variable messages directly rather than the marginal. For normalisable factors, we define a consistency condition (requiring exactness when all other incoming messages are Dirac deltas) to guide message construction. We prove a master theorem (proper messages, any graph) bounding marginal KL from message KL, with three structural corollaries: Dirac-input consistency, no EP-style inner-loop iteration, and no negative-precision messages. Further, we prove a complementary O(1/r²) guarantee for the inherently improper backward message of the product factor, whose closed-form treatment has resisted prior work. As a concrete instantiation, we derive explicit DMA messages for the product and leaky-ReLU factors and assemble a Bayesian neural network (BNN) inference algorithm with one forward/backward sweep per training example and no gradient learning-rate hyperparameter, validating that the structural guarantees translate to predictive uncertainty that widens in data-sparse regions, including under model mismatch.

A computationally-tractable measure of global sensitivity for sampling-based Bayesian inference
Arina Odnoblyudova, Charita Dellaporta, François-Xavier Briol
arXiv6 Oct 2026PreprintCC BY 4.0Full textStatisticsarXiv
Tractable Inference with Kolmogorov-Arnold Networks
Juan Diego Toscano, Sascha Ranftl, Zhiwei Gao +2
arXiv22 Oct 2026PreprintCC BY-NC-SA 4.0Full textNumerical analysis & scientific computing Computational physicsarXiv
Kurdyka-Łojasiewicz-based convergence analysis with approximate weakly convex subdifferentials: bundle, quadratic penalty, and moving balls approximation methods
Jiefeng Xu, Guoyin Li, Ting Kei Pong
arXiv21 Oct 2026PreprintCC BY 4.0Full textOptimisation & controlarXiv
Approximation, Consistency, and Prediction in Generalized Neural Network Mixed Models for Longitudinal and Clustered Data
Mahmud Hasan, Md Moinul Ahsan, Mathias Nthiani Muia +2
arXiv11 Oct 2026PreprintarXiv licenceFull textStatisticsarXiv
Possibilistic Radial Transport for Approximate IM Inference
Jungeum Kim, Percy Zhai
arXiv7 Oct 2026PreprintarXiv licenceFull textStatisticsarXiv

Cite

BibTeX RIS

BibTeX

@misc{herbrich2026direct,
  title = {{Direct Message Approximation (DMA): A Consistency-Based Framework for Tractable Approximate Inference on Factor Graphs}},
  author = {Ralf Herbrich and Rainer Schlosser and Jan Lemcke and Johann Ukrow and Anna Kazachkova and Nicolas Alder and Leonhard Hennicke and Theo Bardey and Nico Grimm and Luca Kleinschmidt and Philipp Kolbe and Cezary Kujath and Johanna Schlimme and Karl Matti Schütz},
  howpublished = {arXiv},
  year = {2026},
  month = sep,
  doi = {10.48550/arxiv.2609.29466},
  eprint = {2609.29466},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2609.29466},
  note = {CC BY 4.0},
}

RIS

TY  - UNPB
TI  - Direct Message Approximation (DMA): A Consistency-Based Framework for Tractable Approximate Inference on Factor Graphs
AU  - Ralf Herbrich
AU  - Rainer Schlosser
AU  - Jan Lemcke
AU  - Johann Ukrow
AU  - Anna Kazachkova
AU  - Nicolas Alder
AU  - Leonhard Hennicke
AU  - Theo Bardey
AU  - Nico Grimm
AU  - Luca Kleinschmidt
AU  - Philipp Kolbe
AU  - Cezary Kujath
AU  - Johanna Schlimme
AU  - Karl Matti Schütz
PY  - 2026
DA  - 2026/09/24
PB  - arXiv
DO  - 10.48550/arxiv.2609.29466
UR  - https://arxiv.org/abs/2609.29466
AB  - Approximate message passing on factor graphs underlies two dominant families of probabilistic inference algorithms: expectation propagation (EP) and variational message passing (VMP). Both methods approximate the marginal at each factor edge, forcing an iterative round-robin schedule, risking negative-precision messages, and, for VMP, collapsing to point estimates at Dirac-delta factors. We introduce Direct Message Approximation (DMA), which approximates factor-to-variable messages directly rather than the marginal. For normalisable factors, we define a consistency condition (requiring exactness when all other incoming messages are Dirac deltas) to guide message construction. We prove a master theorem (proper messages, any graph) bounding marginal KL from message KL, with three structural corollaries: Dirac-input consistency, no EP-style inner-loop iteration, and no negative-precision messages. Further, we prove a complementary O(1/r^2) guarantee for the inherently improper backward message of the product factor, whose closed-form treatment has resisted prior work. As a concrete instantiation, we derive explicit DMA messages for the product and leaky-ReLU factors and assemble a Bayesian neural network (BNN) inference algorithm with one forward/backward sweep per training example and no gradient learning-rate hyperparameter, validating that the structural guarantees translate to predictive uncertainty that widens in data-sparse regions, including under model mismatch.
ER  -

Abstract

Related

Cite