Doc2LoRA Provides Decodable Representations of Scientific Ideas
Abstract
Representing scientific papers as points in a space lets us search for similar papers and inquire about how fields relate to one another and drive innovation. Beyond search, the vector space of papers invites generation: mixing papers through simple vector operations creates new points, mirroring combinatorial novelty, the recombination of existing ideas into new ones. However, a mixed point often represents an idea no paper has yet realized, with no papers nearby to identify the idea. We propose representing each paper by a LoRA adapter generated by the Doc-to-LoRA hypernetwork. Every point in the space, including mixtures, thus represents a large language model (LLM) open to questions and instructions in natural language. On papers from the American Physical Society (APS), we instruct the LLM at the average of each subfield to name the field in a few words and obtain labels closer to the official names than the labels of five baselines, as judged by word overlap and a panel of five LLM judges. We also ask the LLMs at points between two APS papers to write an abstract and obtain descriptions shifting from one paper to the other in step with the mixing weight. While Doc-to-LoRA is trained for generation, a small invertible transform makes the embeddings competitive for search, on par with SPECTER2 and EmbeddingGemma and close to SBERT. Because the transform is invertible, every point in the transformed space still maps back to an LLM. The embeddings thus serve both search and generation, enabling researchers to question the idea at any point in the space as a starting point for generating new ideas.
Related
- Explicit Capacity-Achieving Quantum LDPC Codes List Decodable in Near-linear Time
- Science of Nuclear Fusion: Insights and Ideas
- Bell State Analysis Provides an Optimal Basis Saturating the Quantum Cramer-Rao in Rotation Sensing
- Anisometric Dielectric Snowman-Particle Provides Improved Control Over Photonic Hook
- Boundary-contact repolarization provides a mechanism for directional guidance of neural crest cell collectives
BibTeX
@misc{mansuri2026doc2lora,
title = {{Doc2LoRA Provides Decodable Representations of Scientific Ideas}},
author = {Chand Sahil Mansuri and Joel Zachariah and Sadamori Kojaku},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.38374},
eprint = {2609.38374},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.38374},
}
RIS
TY - UNPB
TI - Doc2LoRA Provides Decodable Representations of Scientific Ideas
AU - Chand Sahil Mansuri
AU - Joel Zachariah
AU - Sadamori Kojaku
PY - 2026
DA - 2026/09/29
PB - arXiv
DO - 10.48550/arxiv.2609.38374
UR - https://arxiv.org/abs/2609.38374
AB - Representing scientific papers as points in a space lets us search for similar papers and inquire about how fields relate to one another and drive innovation. Beyond search, the vector space of papers invites generation: mixing papers through simple vector operations creates new points, mirroring combinatorial novelty, the recombination of existing ideas into new ones. However, a mixed point often represents an idea no paper has yet realized, with no papers nearby to identify the idea. We propose representing each paper by a LoRA adapter generated by the Doc-to-LoRA hypernetwork. Every point in the space, including mixtures, thus represents a large language model (LLM) open to questions and instructions in natural language. On papers from the American Physical Society (APS), we instruct the LLM at the average of each subfield to name the field in a few words and obtain labels closer to the official names than the labels of five baselines, as judged by word overlap and a panel of five LLM judges. We also ask the LLMs at points between two APS papers to write an abstract and obtain descriptions shifting from one paper to the other in step with the mixing weight. While Doc-to-LoRA is trained for generation, a small invertible transform makes the embeddings competitive for search, on par with SPECTER2 and EmbeddingGemma and close to SBERT. Because the transform is invertible, every point in the transformed space still maps back to an LLM. The embeddings thus serve both search and generation, enabling researchers to question the idea at any point in the space as a starting point for generating new ideas.
ER -