Meta Non-Autoregressive Models: A Framework for Generating, Selecting, and Adapting Non-Autoregressive Generation Mechanisms
Abstract
Non-autoregressive (NAR) models aim to reduce the sequential dependencies associated with conventional autoregressive generation by predicting multiple output positions in parallel. Despite progress in parallel decoding, iterative refinement, masked prediction, and related generation strategies, the choice and configuration of a generation mechanism are often determined by a predefined architecture, training procedure, or inference algorithm. This paper introduces Meta Non-Autoregressive Models (Meta-NAR) as a conceptual framework for generating and adapting the mechanisms through which non-autoregressive models produce outputs. Rather than treating a non-autoregressive model as a fixed generator, Meta-NAR treats its generation architecture, dependency structure, masking policy, refinement schedule, stopping criterion, and computational allocation as objects of meta-level optimization. We formalize the framework as a constrained bilevel optimization problem in which an outer controller constructs or selects generation mechanisms and an inner generator produces outputs under the selected configuration. We further propose a task-conditioned mechanism representation, a cost-aware objective, an adaptive mechanism-selection procedure, and an evaluation protocol covering generation quality, latency, computational expenditure, robustness, and adaptation overhead. The framework distinguishes mechanism-level generation from ordinary output generation and from the direct optimization of a fixed non-autoregressive decoder. Importantly, its individual components overlap with established research on iterative refinement, adaptive decoding, neural architecture search, and automated machine learning. Consequently, the contribution claimed here is a unified research formulation rather than an experimentally established algorithmic breakthrough. The framework provides a basis for investigating whether meta-level adaptation can improve the quality-efficiency trade-off of non-autoregressive generation across heterogeneous tasks.
Related
- Meta-Masked Generative Models: A Unified Framework for Generating, Selecting, and Optimizing Masking Mechanisms
- Meta-Generation: A General Framework for Generating Generators, Generative Mechanisms, and the Rules That Construct Them
- Meta-Autoregressive Modeling: A Higher-Order Framework for Generating, Adapting, and Optimizing Autoregressive Models
- Evidence-First Generation: A Multi-Granular Framework for Evidence-Prioritized Autoregressive Text Generation, Content Selection, and Dynamic Reorganization
- Selecting and generating web search based features using small language models for tabular data classification
BibTeX
@misc{zhang2026meta,
title = {{Meta Non-Autoregressive Models: A Framework for Generating, Selecting, and Adapting Non-Autoregressive Generation Mechanisms}},
author = {Jincheng Zhang},
howpublished = {Zenodo},
year = {2026},
month = oct,
doi = {10.5281/zenodo.23530090},
url = {https://zenodo.org/records/23530090},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Meta Non-Autoregressive Models: A Framework for Generating, Selecting, and Adapting Non-Autoregressive Generation Mechanisms
AU - Jincheng Zhang
PY - 2026
DA - 2026/10/21
PB - Zenodo
DO - 10.5281/zenodo.23530090
UR - https://zenodo.org/records/23530090
AB - Non-autoregressive (NAR) models aim to reduce the sequential dependencies associated with conventional autoregressive generation by predicting multiple output positions in parallel. Despite progress in parallel decoding, iterative refinement, masked prediction, and related generation strategies, the choice and configuration of a generation mechanism are often determined by a predefined architecture, training procedure, or inference algorithm. This paper introduces Meta Non-Autoregressive Models (Meta-NAR) as a conceptual framework for generating and adapting the mechanisms through which non-autoregressive models produce outputs. Rather than treating a non-autoregressive model as a fixed generator, Meta-NAR treats its generation architecture, dependency structure, masking policy, refinement schedule, stopping criterion, and computational allocation as objects of meta-level optimization. We formalize the framework as a constrained bilevel optimization problem in which an outer controller constructs or selects generation mechanisms and an inner generator produces outputs under the selected configuration. We further propose a task-conditioned mechanism representation, a cost-aware objective, an adaptive mechanism-selection procedure, and an evaluation protocol covering generation quality, latency, computational expenditure, robustness, and adaptation overhead. The framework distinguishes mechanism-level generation from ordinary output generation and from the direct optimization of a fixed non-autoregressive decoder. Importantly, its individual components overlap with established research on iterative refinement, adaptive decoding, neural architecture search, and automated machine learning. Consequently, the contribution claimed here is a unified research formulation rather than an experimentally established algorithmic breakthrough. The framework provides a basis for investigating whether meta-level adaptation can improve the quality-efficiency trade-off of non-autoregressive generation across heterogeneous tasks.
ER -