{
  "abstract": "The design of optical materials relies on highly specialized knowledge, which creates a major bottleneck for data-driven scientific discovery in the optical material domain. Large language models open new possibilities for scientific research, but general-purpose models struggle with the domainspecific physical logic and terminology. More importantly, no standardized evaluation benchmark exists to systematically assess how well LLMs perform on optical materials tasks. To fill this gap, we first construct two evaluation benchmarks, Optic_Choice and Optic_QA, designed specifically for the optical materials domain. We build a high-quality instruction-tuning dataset of 23,225 question-answer pairs through an automated multi-stage pipeline from a curated corpus of papers and textbook. We then develop OptiMatLLM, a 7-billion-parameter LLM fine-tuned LLM on this corpus. Evaluation results show that OptiMatLLM achieves top performance on our optical benchmarks, reaching 0.79 accuracy on Optic_Choice, while also performing competitively on broader scientific tests such as CMHChem, ChemBench, and MaScQA. The model retains strong general capabilities as well, scoring 0.91 on the GSM8K mathematical reasoning benchmark. We further provide a systematic comparison of multiple leading models on these benchmarks, establishing a reference for future work. All benchmarks, dataset, and model are publicly released to support further research on LLMs for optical and optoelectronic materials design.",
  "abstract_shown": true,
  "affiliations": [
    "Key Laboratory of Spectral Imaging Technology",
    "Xi’an Institute of Optics and Precision Mechanics of the Chinese Academy of Sciences",
    "School of Software",
    "Nanjing University of Information Science & Technology"
  ],
  "article_type": null,
  "arxiv": null,
  "author_total": 3,
  "authors": [
    {
      "affs": [
        0,
        1,
        2,
        3
      ],
      "name": "Xinyi Wang"
    },
    {
      "affs": [
        2,
        3
      ],
      "name": "Youle Wang"
    },
    {
      "affs": [
        0,
        1
      ],
      "name": "Lei Zhang",
      "orcid": "0000-0001-6873-7314"
    }
  ],
  "bibtex": "https://research.draconitic.com/paper/f1j6tdqjh8fcz.bib",
  "cited_by": 0,
  "doi": "10.26434/chemrxiv.15004991/v2",
  "fulltext_url": null,
  "id": "f1j6tdqjh8fcz",
  "issns": [],
  "landing_url": "https://chemrxiv.org/doi/full/10.26434/chemrxiv.15004991/v2",
  "language": null,
  "licence": "CC-BY",
  "licence_label": "CC BY",
  "licence_source": "journal",
  "licence_url": "https://creativecommons.org/licenses/by/4.0/",
  "metadata_licence": null,
  "native_subjects": [
    [
      "openalex",
      "2505|Machine Learning in Materials Science"
    ]
  ],
  "openalex": "W7208778033",
  "pdf_url": "https://chemrxiv.org/doi/pdf/10.26434/chemrxiv.15004991/v2",
  "pmcid": null,
  "pmid": null,
  "published": "2026-09-04",
  "publisher": "American Chemical Society (ACS)",
  "ris": "https://research.draconitic.com/paper/f1j6tdqjh8fcz.ris",
  "sources": [
    {
      "first_seen": "2026-10-30T12:17:17Z",
      "native_id": "10.26434/chemrxiv.15004991",
      "source": "chemrxiv",
      "url": "https://chemrxiv.org/doi/full/10.26434/chemrxiv.15004991/v2"
    }
  ],
  "subjects": [
    {
      "field": "chemistry",
      "origin": "openalex:2505|Machine Learning in Materials Science",
      "subfield": "materials-polymers"
    }
  ],
  "title": "OptiMatLLM: A Benchmark Suite and a Lightweight Domain-Specific LLM for Optical Materials Research",
  "title_text": "OptiMatLLM: A Benchmark Suite and a Lightweight Domain-Specific LLM for Optical Materials Research",
  "type": "preprint",
  "updated_at": "2026-10-30T12:17:17Z",
  "url": "https://research.draconitic.com/paper/f1j6tdqjh8fcz",
  "venue": "ChemRxiv",
  "versions": []
}