PaECTER: Patent-level representation learning using citation-informed transformers
- 1Innovation & Entrepreneurship Research, Max Planck Institute for Innovation and Competition, Munich, Germany
- 2Ludwig Maximilian University of Munich, Munich, Germany
- 3Centre for Economic Policy Research (CEPR), London, United Kingdom
Abstract
PaECTER is an open-source document-level encoder specific to patents. We fine-tune BERT for Patents with examiner-added citation information to generate numerical representations for patent documents. PaECTER performs better in similarity tasks than current state-of-the-art models used in the patent domain. More specifically, our model outperforms the patent specific pre-trained language model (BERT for Patents) and general-purpose text embedding models (e.g., E5, GTE, and BGE) on our patent citation prediction test dataset on different rank evaluation metrics. PaECTER predicts at least one most similar patent at a rank of 1.32 on average when compared against 25 irrelevant patents. Numerical representations generated by PaECTER from patent text can be used for downstream tasks such as classification, tracing knowledge flows, or semantic similarity search. Semantic similarity search is especially relevant in the context of prior art search for both inventors and patent examiners.
Related
- Spatially Informed and Direction Sensitive Representation Learning for Spatial Transcriptomics
- Understanding Performance Plateaus and Representation Collapse in 3D MRI Transformers Trained to Classify Brain Disease
- PINR-ESI: Physics-Informed Self-Supervised Implicit Neural Representation Framework for EEG Extended-Source Imaging
- When Does Depth Matter For In-Context Learning? Adaptive Inference in Deep Transformers
- flodl-bio: First Rust data, transformers and model layer for entire deep learning on biological sequences
BibTeX
@article{ghosh2026paecter,
title = {{PaECTER: Patent-level representation learning using citation-informed transformers}},
author = {Mainak Ghosh and Michael E. Rose and Sebastian Erhardt and Erik Buunk and Dietmar Harhoff},
journal = {PLOS One},
year = {2026},
month = oct,
doi = {10.1371/journal.pone.0358188},
url = {https://doi.org/10.1371/journal.pone.0358188},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - PaECTER: Patent-level representation learning using citation-informed transformers
AU - Mainak Ghosh
AU - Michael E. Rose
AU - Sebastian Erhardt
AU - Erik Buunk
AU - Dietmar Harhoff
PY - 2026
DA - 2026/10/27
JO - PLOS One
DO - 10.1371/journal.pone.0358188
UR - https://doi.org/10.1371/journal.pone.0358188
AB - PaECTER is an open-source document-level encoder specific to patents. We fine-tune BERT for Patents with examiner-added citation information to generate numerical representations for patent documents. PaECTER performs better in similarity tasks than current state-of-the-art models used in the patent domain. More specifically, our model outperforms the patent specific pre-trained language model (BERT for Patents) and general-purpose text embedding models (e.g., E5, GTE, and BGE) on our patent citation prediction test dataset on different rank evaluation metrics. PaECTER predicts at least one most similar patent at a rank of 1.32 on average when compared against 25 irrelevant patents. Numerical representations generated by PaECTER from patent text can be used for downstream tasks such as classification, tracing knowledge flows, or semantic similarity search. Semantic similarity search is especially relevant in the context of prior art search for both inventors and patent examiners.
SN - 1932-6203
ER -