A Multi-View Text Classification Framework Combining Transformers and Topic Modeling
- 1IDMEC, Instituto Superior Técnico, University of Lisbon, Av. Rovisco Pais 1, 1049-001 Lisbon, Portugal
Abstract
Text classification models often rely on transformer-based embeddings or topic-modeling techniques independently, limiting their ability to capture both contextual and thematic information. We propose the Topic-Transformer (TT) framework, a multi-view learning framework that combines transformer-based encoders and topic-modeling methods to improve text classification performance. TT processes text simultaneously through a transformer model and a topic-modeling technique. The resulting contextual embeddings and topic representations are then fed into a classification head. TT is tested using five distinct transformer encoders and three topic-modeling techniques, resulting in fifteen configurations of TT. The tests are conducted on six benchmark datasets and the results are evaluated in terms of macro-average F1-score, precision, recall, and training time. The results show that, for every combination of dataset and transformer, at least one TT configuration outperforms its standalone transformer counterpart. However, the magnitude and statistical significance of these improvements vary across datasets and configurations. In four of the six datasets, at least one TT configuration yields a statistically significant improvement at a significance level of 0.05 after multiple-testing correction. The largest significant gains are observed on 20Newsgroups, where improvements reach 2.8% in macro-average F1-score, with statistically significant improvements found in most transformer–TT comparisons.
Related
- An Uncertainty-Aware Hybrid Symbolic-Lexicon Framework for Multi-Domain Sentiment Classification
- Modeling Transition Dynamics and Network Structure in Cross-National Process Data: A Hierarchical Multi-State Survival Framework
- A coarse-to-fine modulation classification scheme utilizing Transformers and Liquid Neural Networks for Smart Grids
- Generative Modeling for Cascading Extremes via Autoregressive Transformers
- Bayesian Hierarchical Modeling Approaches for Combining Information From Multiple Data Sources to Produce Annual Estimates of National Immunization Coverage.
BibTeX
@article{santos2026multiview,
title = {{A Multi-View Text Classification Framework Combining Transformers and Topic Modeling}},
author = {Filipe M. P. Santos and João M. C. Sousa and Susana M. Vieira},
journal = {Mathematics},
year = {2026},
month = nov,
doi = {10.3390/math14213996},
url = {https://www.mdpi.com/2227-7390/14/21/3996},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - A Multi-View Text Classification Framework Combining Transformers and Topic Modeling
AU - Filipe M. P. Santos
AU - João M. C. Sousa
AU - Susana M. Vieira
PY - 2026
DA - 2026/11/02
JO - Mathematics
DO - 10.3390/math14213996
UR - https://www.mdpi.com/2227-7390/14/21/3996
AB - Text classification models often rely on transformer-based embeddings or topic-modeling techniques independently, limiting their ability to capture both contextual and thematic information. We propose the Topic-Transformer (TT) framework, a multi-view learning framework that combines transformer-based encoders and topic-modeling methods to improve text classification performance. TT processes text simultaneously through a transformer model and a topic-modeling technique. The resulting contextual embeddings and topic representations are then fed into a classification head. TT is tested using five distinct transformer encoders and three topic-modeling techniques, resulting in fifteen configurations of TT. The tests are conducted on six benchmark datasets and the results are evaluated in terms of macro-average F1-score, precision, recall, and training time. The results show that, for every combination of dataset and transformer, at least one TT configuration outperforms its standalone transformer counterpart. However, the magnitude and statistical significance of these improvements vary across datasets and configurations. In four of the six datasets, at least one TT configuration yields a statistically significant improvement at a significance level of 0.05 after multiple-testing correction. The largest significant gains are observed on 20Newsgroups, where improvements reach 2.8% in macro-average F1-score, with statistically significant improvements found in most transformer–TT comparisons.
SN - 2227-7390
ER -