Bayesian Implementation of the Factor‐Analytic Mixed Model and Application to Embeddings
Abstract
Mixed models and neural networks each offer complementary frameworks for prediction. Theoretically grounded in inference, mixed models can unveil latent variance structure notably with the factor‐analytic approach. In this article, we propose to bridge the gap between the two frameworks, leveraging embedding data from a neural network encoder to structure the covariance matrix in a mixed model. First, we implemented a Bayesian factor‐analytic mixed model and challenged it using simulations. The model showed sensitivity to the prior, with a propensity to overfitting under some hyperparameterizations. However, with an appropriate hyperparameterization it showed high predictive performance and a good ability to infer the latent structure (accuracy between 0.881 and 0.919 on the two first factorial axes). Using a challenging design in which each individual was observed a single time in a single environment, we showed that the embedding information enabled prediction of unobserved individuals. A small but significant ( p ‐value < 0.05) amount of real bibliometric data variance was captured using this factor‐analytic component applied within a negative‐binomial mixed model. These results call for further investigation of the use of the mixed model framework to analyse embedding data.
Related
- New tools for sparse graph embeddings: Hybrid expanders and application to nested cycles
- Nonlinear parameter-varying embeddings for nonlinear state estimation with application to a two-link robot manipulator
- Exact barycentric rational interpolation of analytic functions with an application to the Bessel functions
- On matrix-analytic Markov modulated models for evolution: application potential and future directions required to advance the field.
- Weak Factor Recovery in Sparse Bayesian Factor Models under Exchangeable Shrinkage Process Prior
BibTeX
@article{marchal2026bayesian,
title = {{Bayesian Implementation of the Factor‐Analytic Mixed Model and Application to Embeddings}},
author = {Alexandre Marchal and Maxence Lamure and Tanguy Lefort and Frank Imbach},
journal = {Stat},
year = {2026},
month = jul,
doi = {10.1002/sta4.70164},
url = {https://imt-mines-ales.hal.science/hal-05740791v1},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - Bayesian Implementation of the Factor‐Analytic Mixed Model and Application to Embeddings
AU - Alexandre Marchal
AU - Maxence Lamure
AU - Tanguy Lefort
AU - Frank Imbach
PY - 2026
DA - 2026/07/03
JO - Stat
DO - 10.1002/sta4.70164
UR - https://imt-mines-ales.hal.science/hal-05740791v1
AB - Mixed models and neural networks each offer complementary frameworks for prediction. Theoretically grounded in inference, mixed models can unveil latent variance structure notably with the factor‐analytic approach. In this article, we propose to bridge the gap between the two frameworks, leveraging embedding data from a neural network encoder to structure the covariance matrix in a mixed model. First, we implemented a Bayesian factor‐analytic mixed model and challenged it using simulations. The model showed sensitivity to the prior, with a propensity to overfitting under some hyperparameterizations. However, with an appropriate hyperparameterization it showed high predictive performance and a good ability to infer the latent structure (accuracy between 0.881 and 0.919 on the two first factorial axes). Using a challenging design in which each individual was observed a single time in a single environment, we showed that the embedding information enabled prediction of unobserved individuals. A small but significant ( p ‐value < 0.05) amount of real bibliometric data variance was captured using this factor‐analytic component applied within a negative‐binomial mixed model. These results call for further investigation of the use of the mixed model framework to analyse embedding data.
SN - 2049-1573
ER -