Particle-based Generalised Stochastic Optimisation

Abstract

We develop a class of general diffusion-based stochastic particle optimisation methods for loss functions with intractable gradients. Specifically, we consider problems in which the loss gradient is an integral with respect to a parameter-dependent distribution, a structure that includes training generative models, fine-tuning, and learning latent-variable models. We introduce mean-field dynamics and its interacting-particle approximations, which contain several existing algorithms as special cases and provide a route to constructing new methods. Under well-posedness and joint contractivity assumptions, we prove exponential convergence and show that the continuous-time particle system admits a non-asymptotic error bound. We illustrate our framework by developing higher-order Langevin and momentum optimisers and evaluating them on maximum marginal-likelihood estimation and energy-based-model training.

Stochastic Mirror-Prox Optimisation of Group Distributionally Robust Multimodal Logistic Regression for Multilingual Fact-Checking Verdict Prediction
Shuyun Li, Fan Si, Dongxu Mo +1
Mathematics2 Oct 2026Peer-reviewedCC BY 4.0General mathematicsCrossref
Weak solutions to granular medium equations via stochastic particle approximations
Łukasz Wodnicki
arXiv3 Nov 2026PreprintCC BY 4.0Full textProbabilityarXiv
Structural identifiability of partially-observed stochastic processes: from single-particle trajectories to total particle density data
Arianna Ceccarelli, Alexander P. Browning, Ruth E. Baker
arXiv30 Oct 2026PreprintCC BY 4.0Full textBioinformatics & computational biology Analysis & PDEsarXiv
Deterministic and stochastic particle methods for the Fokker-Planck equation in S--formulation with application to point set registration
Klaas Willems, Angelo Iollo, Giovanni Russo +1
arXiv1 Oct 2026PreprintCC BY 4.0Full textNumerical analysis & scientific computingarXiv
Large-time existence for a generalised Serre-Green-Naghdi (Debyaoui-Ersoy) model with variable width and slowly varying bottom
Mehmet Ersoy
HAL16 Oct 2026PreprintPublisher termsFull textGeneral mathematicsHAL

Cite

BibTeX RIS

BibTeX

@misc{zhang2026particlebased,
  title = {{Particle-based Generalised Stochastic Optimisation}},
  author = {Jiechen Jackie Zhang and O. Deniz Akyildiz},
  howpublished = {arXiv},
  year = {2026},
  month = oct,
  doi = {10.48550/arxiv.2608.02844},
  eprint = {2608.02844},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2608.02844},
  note = {CC BY 4.0},
}

RIS

TY  - UNPB
TI  - Particle-based Generalised Stochastic Optimisation
AU  - Jiechen Jackie Zhang
AU  - O. Deniz Akyildiz
PY  - 2026
DA  - 2026/10/09
PB  - arXiv
DO  - 10.48550/arxiv.2608.02844
UR  - https://arxiv.org/abs/2608.02844
AB  - We develop a class of general diffusion-based stochastic particle optimisation methods for loss functions with intractable gradients. Specifically, we consider problems in which the loss gradient is an integral with respect to a parameter-dependent distribution, a structure that includes training generative models, fine-tuning, and learning latent-variable models. We introduce mean-field dynamics and its interacting-particle approximations, which contain several existing algorithms as special cases and provide a route to constructing new methods. Under well-posedness and joint contractivity assumptions, we prove exponential convergence and show that the continuous-time particle system admits a non-asymptotic error bound. We illustrate our framework by developing higher-order Langevin and momentum optimisers and evaluating them on maximum marginal-likelihood estimation and energy-based-model training.
ER  -

Abstract

Related

Cite