{
  "abstract": "Cross-jurisdictional pharmaceutical compliance requires comparison of regulatory requirements across administrative systems such as the US Food and Drug Administration and China's National Medical Products Administration. Although large language models (LLMs) are increasingly explored for healthcare and regulatory applications, their performance in cross-jurisdictional pharmaceutical regulation has not been systematically evaluated using a dedicated benchmark. We introduce Sino-US-DrugQA, a bilingual multiple-choice benchmark covering Monolingual, Comparative, and Parallel regulatory question–answer tasks. The 11,871 candidate items underwent deterministic structural validation and full-dataset semantic quality screening, followed by risk-stratified independent review of 1,432 items by two regulatory experts. The final release comprised 11,444 items, including 10,122 classified as pass and 1,322 as borderline. Among 500 items sampled from the semantic screen-negative population, 18 were subsequently classified as material errors, corresponding to an observed residual material-error proportion of 3.60% (Wilson 95% confidence interval, 2.29%–5.62%). Four representative LLMs—gpt-5.6-terra, gemini-3.6-flash, deepseek-v4-flash, and qwen-3.5-max—were evaluated under a standardized zero-shot protocol. Overall accuracy ranged from 83.21% to 86.43%. Comparative accuracy was consistently lower than Monolingual accuracy, with absolute differences of 4.42–8.98 percentage points across models. The two highest-scoring models, GPT and Gemini, did not differ significantly after adjustment for multiple comparisons. These results indicate that explicit comparison across non-equivalent regulatory systems remains more challenging than single-jurisdiction question answering. Sino-US-DrugQA provides a validated and reproducible resource for evaluating bilingual regulatory reasoning. The findings support further investigation of expert-supervised decision-support workflows rather than autonomous regulatory interpretation. The stable dataset release and evaluation resources are available at https://github.com/DodgeLU/Sino-US-DrugQA.",
  "abstract_shown": true,
  "affiliations": [
    "Information Application Research Center of Shanghai Municipal Administration for Market Regulation, Shanghai, China",
    "Shanghai Artificial Intelligence Research Institute Co., Ltd., Shanghai, China"
  ],
  "article_type": "Research Article",
  "arxiv": null,
  "author_total": 3,
  "authors": [
    {
      "affs": [
        0
      ],
      "name": "Xuejing Fu"
    },
    {
      "affs": [
        1
      ],
      "name": "Zhen Chen"
    },
    {
      "name": "Wentao Lu"
    }
  ],
  "bibtex": "https://research.draconitic.com/paper/dvc88982y6z10.bib",
  "cited_by": 0,
  "doi": "10.1371/journal.pone.0343858",
  "fulltext_url": null,
  "id": "dvc88982y6z10",
  "issns": [
    "1932-6203"
  ],
  "landing_url": "https://doi.org/10.1371/journal.pone.0343858",
  "language": null,
  "licence": "CC-BY-4.0",
  "licence_label": "CC BY 4.0",
  "licence_source": "journal",
  "licence_url": "https://creativecommons.org/licenses/by/4.0/",
  "metadata_licence": null,
  "native_subjects": [
    [
      "openalex",
      "2718|Artificial Intelligence in Healthcare and Education"
    ],
    [
      "plos",
      "/Biology and life sciences/Bioengineering/Biotechnology/Medical devices and equipment"
    ],
    [
      "plos",
      "/Biology and life sciences/Neuroscience/Cognitive science/Cognitive psychology/Language"
    ],
    [
      "plos",
      "/Biology and life sciences/Neuroscience/Cognitive science/Cognitive psychology/Language/Multilingualism"
    ],
    [
      "plos",
      "/Biology and life sciences/Psychology/Cognitive psychology/Language"
    ],
    [
      "plos",
      "/Biology and life sciences/Psychology/Cognitive psychology/Language/Multilingualism"
    ],
    [
      "plos",
      "/Biology and life sciences/Taxonomy"
    ],
    [
      "plos",
      "/Computer and information sciences/Data management/Metadata"
    ],
    [
      "plos",
      "/Computer and information sciences/Data management/Taxonomy"
    ],
    [
      "plos",
      "/Engineering and technology/Bioengineering/Biotechnology/Medical devices and equipment"
    ],
    [
      "plos",
      "/Medicine and health sciences/Medical devices and equipment"
    ],
    [
      "plos",
      "/Medicine and health sciences/Pharmaceutics/Drug therapy/Drug administration"
    ],
    [
      "plos",
      "/Medicine and health sciences/Pharmacology/Drug research and development/Drug regulation"
    ],
    [
      "plos",
      "/Social sciences/Law and legal sciences/Regulations"
    ],
    [
      "plos",
      "/Social sciences/Psychology/Cognitive psychology/Language"
    ],
    [
      "plos",
      "/Social sciences/Psychology/Cognitive psychology/Language/Multilingualism"
    ]
  ],
  "openalex": "W7213295942",
  "pdf_url": "https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0343858&type=printable",
  "pmcid": null,
  "pmid": null,
  "published": "2026-09-15",
  "publisher": "Public Library of Science",
  "ris": "https://research.draconitic.com/paper/dvc88982y6z10.ris",
  "sources": [
    {
      "first_seen": "2026-10-30T12:17:50Z",
      "native_id": "10.1371/journal.pone.0343858",
      "source": "plos",
      "url": "https://doi.org/10.1371/journal.pone.0343858"
    }
  ],
  "subjects": [
    {
      "field": "biology",
      "origin": "plos:neuroscience",
      "subfield": "neuroscience"
    },
    {
      "field": "biology",
      "origin": "plos:biotechnology",
      "subfield": "systems-synthetic"
    },
    {
      "field": "medicine",
      "origin": "plos:pharmacology",
      "subfield": "pharmacology-toxicology"
    }
  ],
  "title": "Sino-US-DrugQA: A benchmark for evaluating large language models in cross-jurisdictional pharmaceutical regulation",
  "title_text": "Sino-US-DrugQA: A benchmark for evaluating large language models in cross-jurisdictional pharmaceutical regulation",
  "type": "peer-reviewed",
  "updated_at": "2026-10-30T12:17:50Z",
  "url": "https://research.draconitic.com/paper/dvc88982y6z10",
  "venue": "PLOS One",
  "versions": []
}