Iterative Atom Refinement: A Monotonicity Principle for Dictionary Learning
Abstract
Dictionary learning seeks to recover an unknown dictionary A from observations bf yi = Abf xi with sparse coefficient vectors bf xi. We introduce the \emph{Iterative Atom Refinement} (IAR) algorithm, a simple procedure for recovering individual dictionary atoms. Starting from a random direction, IAR repeatedly selects the observations most strongly correlated with the current iterate and updates the direction by averaging the selected data. Our main contribution is a rigorous convergence theory of IAR. Using high-dimensional probabilistic estimates and a novel monotonicity principle for atom-selection probabilities, we show that a small initial advantage of one atom is amplified until that atom is isolated. Under our model assumptions, IAR identifies a generating atom after only three refinement steps. Numerical experiments support the theory and show that the resulting dynamics accurately capture the behavior observed in dictionary refinement.
Related
- An analysis of iterative refinement for quantum linear system solvers
- Progressive Reliability-Guided Refinement with Cross-Modal Consistency and Iterative Correction for Weakly Supervised PET/CT Tumor Segmentation
- Sparse Dictionary Learning for Measure-Valued Data via Linearized Optimal Transport: Efficient Algorithms and Sample Complexity
- Geometric Dictionary Learning of Dynamical Systems with Optimal Transport
- Iterative Calibration and Learning: Mitigating Teacher Bias in LLM Distillation without Target-Domain Reward Feedback
BibTeX
@misc{christie2026iterative,
title = {{Iterative Atom Refinement: A Monotonicity Principle for Dictionary Learning}},
author = {Alexander Christie and Miguel Moscoso and Alexei Novikov and George Papanicolaou and Chrysoula Tsogka},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.23812},
eprint = {2609.23812},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.23812},
}
RIS
TY - UNPB
TI - Iterative Atom Refinement: A Monotonicity Principle for Dictionary Learning
AU - Alexander Christie
AU - Miguel Moscoso
AU - Alexei Novikov
AU - George Papanicolaou
AU - Chrysoula Tsogka
PY - 2026
DA - 2026/09/20
PB - arXiv
DO - 10.48550/arxiv.2609.23812
UR - https://arxiv.org/abs/2609.23812
AB - Dictionary learning seeks to recover an unknown dictionary A from observations bf y_i = Abf x_i with sparse coefficient vectors bf x_i. We introduce the \emph{Iterative Atom Refinement} (IAR) algorithm, a simple procedure for recovering individual dictionary atoms. Starting from a random direction, IAR repeatedly selects the observations most strongly correlated with the current iterate and updates the direction by averaging the selected data. Our main contribution is a rigorous convergence theory of IAR. Using high-dimensional probabilistic estimates and a novel monotonicity principle for atom-selection probabilities, we show that a small initial advantage of one atom is amplified until that atom is isolated. Under our model assumptions, IAR identifies a generating atom after only three refinement steps. Numerical experiments support the theory and show that the resulting dynamics accurately capture the behavior observed in dictionary refinement.
ER -