Fine-Tuning Small Language Models for Reliable VASP INCAR Generation
Abstract
Language models can prepare VASP INCAR files from natural-language requests, but so far only large proprietary cloud models come close to handling the tightly coupled, physics-sensitive settings reliably, a dependence that fits poorly with local, high-throughput materials workflows where privacy, cost, and offline deployment matter. We show that a small language model (SLM) can close this gap. The SLM is fine-tuned on reference VASP calculations and paired with VASPGuard, a deterministic post-processor that checks syntax, workflow, and material-dependent constraints; we call the combined model INCAR-SLM. On INCARBench, a benchmark for VASP INCAR generation, INCAR-SLM built on Qwen3-4B outperforms every general-purpose LLM evaluated, exceeding GPT-5.4 by 15.55 points on the 100-point INCAR Score. Most of this gain comes from fine-tuning, with VASPGuard correcting the errors that remain. We further find that model size matters less than expected: once fine-tuning and post-processing are applied, performance saturates at a few billion parameters, and Qwen3-4B outperforms larger models in the same family.
Related
- INCARBench: A Benchmark for Scientific Configuration in VASP INCAR by Large Language Models
- LLM-Based FORM Code Generation with Verification-Driven Fine-Tuning
- Language-Guided Hypotheses Generation for Sparse SMEFT Analyses
- Variational Quantum Transformer Architecture for Synthetic Language Generation
- Inverse-Designed High-Q and Small-Mode-Volume Silicon Nitride Photonic Crystal Cavities for Second- and Third-Harmonic Generation
BibTeX
@misc{zhang2026finetuning,
title = {{Fine-Tuning Small Language Models for Reliable VASP INCAR Generation}},
author = {Xinyue Zhang and Jixiang Li and Bin Shao and Baishun Yang and Zhiyang Liu and Weichao Wang},
howpublished = {arXiv},
year = {2026},
month = aug,
doi = {10.48550/arxiv.2608.05387},
eprint = {2608.05387},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2608.05387},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Fine-Tuning Small Language Models for Reliable VASP INCAR Generation
AU - Xinyue Zhang
AU - Jixiang Li
AU - Bin Shao
AU - Baishun Yang
AU - Zhiyang Liu
AU - Weichao Wang
PY - 2026
DA - 2026/08/05
PB - arXiv
DO - 10.48550/arxiv.2608.05387
UR - https://arxiv.org/abs/2608.05387
AB - Language models can prepare VASP INCAR files from natural-language requests, but so far only large proprietary cloud models come close to handling the tightly coupled, physics-sensitive settings reliably, a dependence that fits poorly with local, high-throughput materials workflows where privacy, cost, and offline deployment matter. We show that a small language model (SLM) can close this gap. The SLM is fine-tuned on reference VASP calculations and paired with VASPGuard, a deterministic post-processor that checks syntax, workflow, and material-dependent constraints; we call the combined model INCAR-SLM. On INCARBench, a benchmark for VASP INCAR generation, INCAR-SLM built on Qwen3-4B outperforms every general-purpose LLM evaluated, exceeding GPT-5.4 by 15.55 points on the 100-point INCAR Score. Most of this gain comes from fine-tuning, with VASPGuard correcting the errors that remain. We further find that model size matters less than expected: once fine-tuning and post-processing are applied, performance saturates at a few billion parameters, and Qwen3-4B outperforms larger models in the same family.
ER -