Relevance Propagation as Entropic Optimal Transport on Filippov Neural Networks: Sinkhorn Propagation and Region Relevance in Image Classifiers
Abstract
Relevance propagation, which redistributes a network's prediction backward through the layers, is usually treated as a heuristic. We show that it is the solution of a single variational problem: Layer-wise Relevance Propagation (LRP) is the divergence-free flow on the active computational graph of zero relative entropy to the backward transition law of the forward activation flow; for a non-negative reference, this is a relative entropy between Markov measures on backward paths. The rules LRP0, LRPz+ and LRPα,β follow from this principle by the choice of a reference, and are instances of one rule, LRPNα,β, whose weights may vary from node to node, for instance with the node's activity, in the spirit of Hebb's principle. The principle rests on a measure-theoretic formalization of feed-forward networks as directed acyclic graphs of Filippov operators, which covers piecewise-smooth and discontinuous architectures (e.g. hard-routed Mixture-of-Experts) with a Clarke-Filippov generalized-gradient calculus; it also characterizes backpropagation as the adjoint equation of a Lagrangian relaxation. On networks of ReLU and max-pooling nodes, LRP0 is Gradient×Input, with an explicit jump across a kink where LRPz+ stays continuous. Taking the entropy edge by edge yields a new rule, Sinkhorn Relevance Propagation. Aggregated over an input region, the flow of a non-negative reference tells, with exact accounting, which hidden units carry its evidence, the conditioned flow being a Schrödinger bridge. On VGG-16, the conservation identities hold to round-off, and the rules are told apart by the sign of their maps, the share of the output left to the bias nodes, faithfulness, localization and class dependence; on ConvNeXt-Tiny, LRPNα,β conserves the output and its Hebbian weights give the highest ABPC of the rules tested.
Related
- On The Relevance Of Using Temperature For Mechanical Fault Detection In Industrial Robots Based On Dynamic Modeling
- Identifying Variable Relations Using Relevance Logic
- Distributional Extension of the FKF Transform and Its Relevance to Supply-Chain Systems
- The price of a weak shadow variable: inference under nonignorable missingness as relevance vanishes
- On the Relevance of Incorporating Decision Dependence in Distributional Ambiguity
BibTeX
@misc{gibaru2026relevance,
title = {{Relevance Propagation as Entropic Optimal Transport on Filippov Neural Networks: Sinkhorn Propagation and Region Relevance in Image Classifiers}},
author = {Olivier Gibaru and Eric Nyiri},
howpublished = {arXiv},
year = {2026},
month = nov,
doi = {10.48550/arxiv.2611.05594},
eprint = {2611.05594},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2611.05594},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Relevance Propagation as Entropic Optimal Transport on Filippov Neural Networks: Sinkhorn Propagation and Region Relevance in Image Classifiers
AU - Olivier Gibaru
AU - Eric Nyiri
PY - 2026
DA - 2026/11/03
PB - arXiv
DO - 10.48550/arxiv.2611.05594
UR - https://arxiv.org/abs/2611.05594
AB - Relevance propagation, which redistributes a network's prediction backward through the layers, is usually treated as a heuristic. We show that it is the solution of a single variational problem: Layer-wise Relevance Propagation (LRP) is the divergence-free flow on the active computational graph of zero relative entropy to the backward transition law of the forward activation flow; for a non-negative reference, this is a relative entropy between Markov measures on backward paths. The rules LRP_0, LRP_z^+ and LRP_α,β follow from this principle by the choice of a reference, and are instances of one rule, LRPN_α,β, whose weights may vary from node to node, for instance with the node's activity, in the spirit of Hebb's principle. The principle rests on a measure-theoretic formalization of feed-forward networks as directed acyclic graphs of Filippov operators, which covers piecewise-smooth and discontinuous architectures (e.g. hard-routed Mixture-of-Experts) with a Clarke-Filippov generalized-gradient calculus; it also characterizes backpropagation as the adjoint equation of a Lagrangian relaxation. On networks of ReLU and max-pooling nodes, LRP_0 is Gradient×Input, with an explicit jump across a kink where LRP_z^+ stays continuous. Taking the entropy edge by edge yields a new rule, Sinkhorn Relevance Propagation. Aggregated over an input region, the flow of a non-negative reference tells, with exact accounting, which hidden units carry its evidence, the conditioned flow being a Schrödinger bridge. On VGG-16, the conservation identities hold to round-off, and the rules are told apart by the sign of their maps, the share of the output left to the bias nodes, faithfulness, localization and class dependence; on ConvNeXt-Tiny, LRPN_α,β conserves the output and its Hebbian weights give the highest ABPC of the rules tested.
ER -