Latent-MoE: Domain-Aware Mixture-of-Experts for PDEs with Multi-Regime Physics
Abstract
Physics-informed neural networks (PINNs) struggle on PDEs whose governing physics varies across the domain. We trace this to a structural property of standard coordinate networks: their neural tangent kernel (NTK) is translation-variant and lets training points of large coordinate magnitude disproportionately influence predictions elsewhere, producing long-range coupling and gradient conflict during training. We show analytically and empirically that mixture-of-experts (MoE) architectures with centered, compact-support routers yield a uniformly banded NTK whose kernel-regression weights decay exponentially with distance, localizing the learning. Building on this, we propose \emph{Latent-MoE}, which interleaves domain-aware MoE blocks within a shared backbone. Unlike FB-PINNs or X-PINNs, which rigidly partition both the domain and the parameters so that the parameters on different subdomains are updated independently, Latent-MoE is designed to preserve the localization benefit of domain-aware routing while allowing capacity to flow across regions through the shared backbone. On standard homogeneous-physics benchmarks Latent-MoE is competitive with established baselines; on benchmarks with multi-stage time-variable physics, where global models and rigid domain decompositions both fall into spurious solutions, it improves over them by more than an order of magnitude, with markedly reduced gradient conflict during training.
Related
- Conditional Capacity and Routing in Mixture-of-Experts Particle Transformers
- Physics-Informed Latent Neural Operator for Three-Dimensional Compressor Cascade Flow Prediction
- Physics-Informed Latent Neural ODEs for Reduced-Order Modeling of Hydrogen, Ammonia, and Methane Combustion Kinetics
- High-fidelity dual-domain photoacoustic data synthesis with latent diffusion models
- Learn the Dynamics, Enforce the Physics: Generative Latent Operators for High-Dimensional Beam Evolution
BibTeX
@misc{wang2026latentmoe,
title = {{Latent-MoE: Domain-Aware Mixture-of-Experts for PDEs with Multi-Regime Physics}},
author = {Hanwen Wang and Paris Perdikaris},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.07814},
eprint = {2609.07814},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.07814},
}
RIS
TY - UNPB
TI - Latent-MoE: Domain-Aware Mixture-of-Experts for PDEs with Multi-Regime Physics
AU - Hanwen Wang
AU - Paris Perdikaris
PY - 2026
DA - 2026/09/07
PB - arXiv
DO - 10.48550/arxiv.2609.07814
UR - https://arxiv.org/abs/2609.07814
AB - Physics-informed neural networks (PINNs) struggle on PDEs whose governing physics varies across the domain. We trace this to a structural property of standard coordinate networks: their neural tangent kernel (NTK) is translation-variant and lets training points of large coordinate magnitude disproportionately influence predictions elsewhere, producing long-range coupling and gradient conflict during training. We show analytically and empirically that mixture-of-experts (MoE) architectures with centered, compact-support routers yield a uniformly banded NTK whose kernel-regression weights decay exponentially with distance, localizing the learning. Building on this, we propose \emph{Latent-MoE}, which interleaves domain-aware MoE blocks within a shared backbone. Unlike FB-PINNs or X-PINNs, which rigidly partition both the domain and the parameters so that the parameters on different subdomains are updated independently, Latent-MoE is designed to preserve the localization benefit of domain-aware routing while allowing capacity to flow across regions through the shared backbone. On standard homogeneous-physics benchmarks Latent-MoE is competitive with established baselines; on benchmarks with multi-stage time-variable physics, where global models and rigid domain decompositions both fall into spurious solutions, it improves over them by more than an order of magnitude, with markedly reduced gradient conflict during training.
ER -