{
  "abstract": "Protein design aims to compose amino-acid sequences that fold into stable three-dimensional structures while satisfying targeted functional properties. The field is increasingly shifting toward vibe protein design, where a single model is expected to generate novel sequences, engineer existing proteins, and reason about protein characteristics through flexible natural-language constraints. However, existing benchmarks typically evaluate isolated aspects of protein design or assume predefined structured inputs, making them ill-suited to assess this broad, open-ended setting. To address this gap, we present Vibe Protein Design Benchmark (VibeProteinBench), a language-interfaced benchmark that evaluates whether a single model can operate across three complementary stages of a computational protein-design workflow: recognition, engineering, and generation. Each stage is grounded in expert-curated mechanistic rationales and multi-faceted in silico validation, to computationally verify whether model outputs are biologically plausible. Across diverse general-purpose and domain-specialized LLMs, evaluated with and without tool access, no model performs strongly across all three stages at once. Beyond measuring end-task success, VibeProteinBench exposes where current systems break: we identify recurring failure patterns and characterize how tool-augmented agents select tool arguments. Together, these findings show that vibe protein design remains a substantial open challenge.",
  "abstract_shown": true,
  "affiliations": [],
  "article_type": null,
  "arxiv": "2605.10978",
  "author_total": 19,
  "authors": [
    {
      "name": "Hyunjin Seo"
    },
    {
      "name": "Hongjoon Ahn"
    },
    {
      "name": "Jimin Park"
    },
    {
      "name": "Sungjun Han"
    },
    {
      "name": "Gyubok Lee"
    },
    {
      "name": "Soojung Yang"
    },
    {
      "name": "Cheng-Hao Liu"
    },
    {
      "name": "Joseph S Brown"
    },
    {
      "name": "Leo Chen"
    },
    {
      "name": "Gina El Nesr"
    },
    {
      "name": "Feyisayo Eweje"
    },
    {
      "name": "Sarah Gurev"
    },
    {
      "name": "Hyejin Lee"
    },
    {
      "name": "Junlang Liu"
    },
    {
      "name": "Zhihui Qi"
    },
    {
      "name": "Gyu Rie Lee"
    },
    {
      "name": "Sungsoo Ahn"
    },
    {
      "name": "Jamin Shin"
    },
    {
      "name": "Sangwon Jung"
    }
  ],
  "bibtex": "https://research.draconitic.com/paper/1h0pgsbkxzx4e.bib",
  "cited_by": 0,
  "doi": "10.48550/arxiv.2605.10978",
  "fulltext_url": null,
  "id": "1h0pgsbkxzx4e",
  "issns": [],
  "landing_url": "https://arxiv.org/abs/2605.10978",
  "language": null,
  "licence": "CC-BY-4.0",
  "licence_label": "CC BY 4.0",
  "licence_source": "article",
  "licence_url": "http://creativecommons.org/licenses/by/4.0/",
  "metadata_licence": "CC0-1.0",
  "native_subjects": [
    [
      "arxiv",
      "q-bio.QM"
    ],
    [
      "openalex",
      "1312|Protein Structure and Dynamics"
    ]
  ],
  "openalex": "W7161052478",
  "pdf_url": "https://arxiv.org/pdf/2605.10978",
  "pmcid": null,
  "pmid": null,
  "published": "2026-09-26",
  "publisher": null,
  "ris": "https://research.draconitic.com/paper/1h0pgsbkxzx4e.ris",
  "sources": [
    {
      "first_seen": "2026-10-30T12:25:21Z",
      "native_id": "2605.10978",
      "source": "arxiv",
      "url": "https://arxiv.org/abs/2605.10978"
    }
  ],
  "subjects": [
    {
      "field": "biology",
      "origin": "arxiv:q-bio.QM",
      "subfield": "bioinformatics"
    }
  ],
  "title": "VibeProteinBench: An Evaluation Benchmark for Language-interfaced Vibe Protein Design",
  "title_text": "VibeProteinBench: An Evaluation Benchmark for Language-interfaced Vibe Protein Design",
  "type": "preprint",
  "updated_at": "2026-10-30T12:25:21Z",
  "url": "https://research.draconitic.com/paper/1h0pgsbkxzx4e",
  "venue": "arXiv",
  "versions": []
}