Calibrated term presence modelling for CCUS patent abstracts: a case study with variational autoencoders and retrieval baselines
- 1Mathematics-Driven Innovation Centre and The Maxwell Institute for Mathematical Sciences, Heriot-Watt University 1 , Edinburgh, UK
- 2School of Mathematical and Computer Sciences and The Maxwell Institute for Mathematical Sciences, Duncan Laboratory for Insurance Data Science, Heriot-Watt University 2 , Edinburgh, UK
- 3Edinburgh Business School, Heriot-Watt University 3 , Edinburgh, UK
Abstract
Patent codes may lack granularity when carbon capture, utilization and storage (CCUS) inventions combine climate and chemical engineering concepts. We evaluate term presence models on 10 603 English patent abstracts through document completion, calibration and cooperative patent classification (CPC) prefix retrieval. We compare two variational autoencoders (VAEs), a product of the experts latent Dirichlet allocation model, a unigram prior, term frequency-inverse document frequency (TF-IDF), PatentSBERTa and PaECTER. With a target masking rate of 50 per cent for positive terms, TF-IDF nearest neighbours give the best ranked completion. The direct Bernoulli VAE ranks slightly above the negative binomial to Bernoulli VAE (NB VAE), while the latter has lower expected calibration error (ECE). Conditional posterior predictive draws retain positive rank associations for document and term counts, but the NB VAE and direct Bernoulli VAE predict 1.47 and 1.63 times the observed positive count, respectively. PaECTER and the TF-IDF representation after singular value decomposition (SVD) give the strongest CPC prefix retrieval, while VAE probabilities could support inspectable term screening.
Related
- Variational Autoencoders for Medical Image Segmentation: A Review
- Meta-Variational Autoencoders: A Framework for Learning to Construct, Adapt, and Optimize Variational Autoencoders
- PAC-Bayesian Reconstruction Guarantees for Time Series Variational Autoencoders
- Elucidating structure-property correlations in ferroelectric Hf0.5Zr0.5O2 films using variational autoencoders
- Enhanced Uncertainty Quantification Variational Autoencoders for the Solution of Bayesian Inverse Problems
BibTeX
@article{matsantonis2026calibrated,
title = {{Calibrated term presence modelling for CCUS patent abstracts: a case study with variational autoencoders and retrieval baselines}},
author = {Charalampos Matsantonis and George Tzougas and Dimitris Christopoulos and David Dekker and George Streftaris},
journal = {Royal Society Open Science},
year = {2026},
month = oct,
doi = {10.1098/rsos.261365},
url = {https://royalsocietypublishing.org/rsos/article/13/10/261365/483955/Calibrated-term-presence-modelling-for-CCUS-patent},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - Calibrated term presence modelling for CCUS patent abstracts: a case study with variational autoencoders and retrieval baselines
AU - Charalampos Matsantonis
AU - George Tzougas
AU - Dimitris Christopoulos
AU - David Dekker
AU - George Streftaris
PY - 2026
DA - 2026/10/07
JO - Royal Society Open Science
DO - 10.1098/rsos.261365
UR - https://royalsocietypublishing.org/rsos/article/13/10/261365/483955/Calibrated-term-presence-modelling-for-CCUS-patent
AB - Patent codes may lack granularity when carbon capture, utilization and storage (CCUS) inventions combine climate and chemical engineering concepts. We evaluate term presence models on 10 603 English patent abstracts through document completion, calibration and cooperative patent classification (CPC) prefix retrieval. We compare two variational autoencoders (VAEs), a product of the experts latent Dirichlet allocation model, a unigram prior, term frequency-inverse document frequency (TF-IDF), PatentSBERTa and PaECTER. With a target masking rate of 50 per cent for positive terms, TF-IDF nearest neighbours give the best ranked completion. The direct Bernoulli VAE ranks slightly above the negative binomial to Bernoulli VAE (NB VAE), while the latter has lower expected calibration error (ECE). Conditional posterior predictive draws retain positive rank associations for document and term counts, but the NB VAE and direct Bernoulli VAE predict 1.47 and 1.63 times the observed positive count, respectively. PaECTER and the TF-IDF representation after singular value decomposition (SVD) give the strongest CPC prefix retrieval, while VAE probabilities could support inspectable term screening.
SN - 2054-5703
ER -