Experiment automation agents: automating materials characterization with vision language model agents
- 1Advanced Photon Source, Argonne National Laboratory, Lemont, IL, USA
- 2Data Science and Learning Division, Argonne National Laboratory, Lemont, IL, USA
- 3Department of Radiation Oncology, Northwestern University, Chicago, IL, USA
Abstract
We present Experiment Automation Agents (EAA), a vision-language-model-driven agentic system designed to automate complex experimental microscopy workflows. EAA integrates multimodal reasoning, tool-augmented action, and optional long-term memory to support both autonomous procedures and interactive user-guided measurements. Built on a flexible task-manager architecture, the system enables workflows ranging from fully agent-driven automation to logic-defined routines that embed localized LLM queries. EAA further provides a modern tool ecosystem with two-way compatibility for Model Context Protocol (MCP), allowing instrument-control tools to be consumed or served across applications. We demonstrate EAA at an imaging beamline at the Advanced Photon Source, including automated zone plate focusing, natural language-described feature search, and interactive data acquisition. These results illustrate how vision-capable agents can enhance beamline efficiency, reduce operational burden, and lower the expertise barrier for users.
Related
- Advanced High-Throughput Characterization of Nanoporous Materials using Pore Graphs
- Advances in characterization approaches toward in-depth understanding of microporous and mesoporous materials
- Correlative SEM–AFM Characterization of 2D Materials: From Sequential Measurements to In-Situ Multimodal Platforms
- Understanding of critical materials for reversible solid oxide cells through in-situ and operando characterization techniques
- Reinterpreting Simulation Results for New Materials Applications with a Large Language Model
BibTeX
@article{du2026experiment,
title = {{Experiment automation agents: automating materials characterization with vision language model agents}},
author = {Ming Du and Yanqi Luo and Srutarshi Banerjee and Michael Wojcik and Jelena Popovic and Mathew J. Cherukara},
journal = {npj Computational Materials},
year = {2026},
month = jul,
doi = {10.1038/s41524-026-02213-8},
url = {https://doi.org/10.1038/s41524-026-02213-8},
note = {CC BY},
}
RIS
TY - JOUR
TI - Experiment automation agents: automating materials characterization with vision language model agents
AU - Ming Du
AU - Yanqi Luo
AU - Srutarshi Banerjee
AU - Michael Wojcik
AU - Jelena Popovic
AU - Mathew J. Cherukara
PY - 2026
DA - 2026/07/30
JO - npj Computational Materials
DO - 10.1038/s41524-026-02213-8
UR - https://doi.org/10.1038/s41524-026-02213-8
AB - We present Experiment Automation Agents (EAA), a vision-language-model-driven agentic system designed to automate complex experimental microscopy workflows. EAA integrates multimodal reasoning, tool-augmented action, and optional long-term memory to support both autonomous procedures and interactive user-guided measurements. Built on a flexible task-manager architecture, the system enables workflows ranging from fully agent-driven automation to logic-defined routines that embed localized LLM queries. EAA further provides a modern tool ecosystem with two-way compatibility for Model Context Protocol (MCP), allowing instrument-control tools to be consumed or served across applications. We demonstrate EAA at an imaging beamline at the Advanced Photon Source, including automated zone plate focusing, natural language-described feature search, and interactive data acquisition. These results illustrate how vision-capable agents can enhance beamline efficiency, reduce operational burden, and lower the expertise barrier for users.
SN - 2057-3960
ER -