Harness engineering enables a multi-agent LLM system to generate cross-domain hypotheses beyond single-investigator reach
- 1Osaka University
Abstract
Large language models (LLMs) are increasingly used to generate scientific hypotheses, and a more capable model is widely assumed to drive output quality. We tested this with a biology-focused multi-agent system coupling an LLM to five Model Context Protocol servers exposing 66 external tools. Upgrading the Synthesizer and Critic from Claude Sonnet 4.6 to Opus 4.8 initially appeared to improve quality, count, and reliability. Under a fully matched harness (identical prompts, schema, hypothesis count, and queries), the differences vanished: scores were indistinguishable on an iPSC-reprogramming task and favoured Sonnet 4.6 on a CHO cell-line-engineering task (18.4 vs 17.4; Welch t = 2.89, p < 0.05). The dominant, controllable driver of quality is therefore the prompting harness, not the model tier. We then introduce the ABCD-Stack (contract-gated prompting, diversity-first generation, adversarial refinement, and cross-family evaluation), which builds these properties in as architecture and shifts outputs toward explicitly falsifiable, causally framed hypotheses beyond the reach of a single-domain investigator. The hypotheses are falsifiable by construction but not yet experimentally validated.
Related
- Bifunctional methacryloyl-norbornene gelatin chemistry enables tunable bioinks for soft tissue engineering via digital light processing
- HelixWings enables AI-driven enzyme engineering for biomimetic cofactor utilization
- Engineering endosperm storage compartments enables simultaneous enrichment of multiple vitamins in rice
- Joint protein-DNA language modeling enables prediction and engineering of DNA recognition specificity
- AI-Guided Multi-Objective Engineering of Glucoamylase Enables Acidification-Free Starch Saccharification
BibTeX
@misc{omasa2026harness,
title = {{Harness engineering enables a multi-agent LLM system to generate cross-domain hypotheses beyond single-investigator reach}},
author = {T. Omasa and K. Watanabe},
howpublished = {bioRxiv},
year = {2026},
month = oct,
doi = {10.64898/2026.10.25.761203},
url = {https://www.biorxiv.org/content/10.64898/2026.10.25.761203v1},
note = {CC BY},
}
RIS
TY - UNPB
TI - Harness engineering enables a multi-agent LLM system to generate cross-domain hypotheses beyond single-investigator reach
AU - T. Omasa
AU - K. Watanabe
PY - 2026
DA - 2026/10/25
PB - bioRxiv
DO - 10.64898/2026.10.25.761203
UR - https://www.biorxiv.org/content/10.64898/2026.10.25.761203v1
AB - Large language models (LLMs) are increasingly used to generate scientific hypotheses, and a more capable model is widely assumed to drive output quality. We tested this with a biology-focused multi-agent system coupling an LLM to five Model Context Protocol servers exposing 66 external tools. Upgrading the Synthesizer and Critic from Claude Sonnet 4.6 to Opus 4.8 initially appeared to improve quality, count, and reliability. Under a fully matched harness (identical prompts, schema, hypothesis count, and queries), the differences vanished: scores were indistinguishable on an iPSC-reprogramming task and favoured Sonnet 4.6 on a CHO cell-line-engineering task (18.4 vs 17.4; Welch t = 2.89, p < 0.05). The dominant, controllable driver of quality is therefore the prompting harness, not the model tier. We then introduce the ABCD-Stack (contract-gated prompting, diversity-first generation, adversarial refinement, and cross-family evaluation), which builds these properties in as architecture and shifts outputs toward explicitly falsifiable, causally framed hypotheses beyond the reach of a single-domain investigator. The hypotheses are falsifiable by construction but not yet experimentally validated.
ER -