{
  "abstract": "Objective\n\nLarge language model (LLM) chatbots show promise for pre-visit planning, but their English-centric development raises equity concerns for multilingual deployment. We evaluated the multilingual capabilities of the Primary Care Pre-visit Chatbot (PCP-Bot), a GPT-4o-based pre-visit chatbot, across English, Mandarin Chinese, Spanish, and Hindi.Materials and methods\n\nIn this mixed-methods study, 31 bilingual participants (11 Mandarin, 10 Spanish, and 10 Hindi) role-played as patients across 5 synthetic primary care cases, each first in English, then in their second language. Participants completed validated instruments (Bot Usability Scale [BUS-11], NASA Task Load Index [NASA-TLX], Human-Computer Trust Scale [HCTS], and Technology Acceptance Model-Perceived Ease of Use [TAM-PEU]), an adapted summary quality rubric (fluency, adequacy, meaning, and severity), and provided open-ended feedback.Results\n\nEnglish interactions scored higher than non-English interactions on usability (difference = 0.15), conversation quality (0.62), and summary quality (0.25; all P < .001). Usability and conversation quality gaps varied by language (both P < .01), driven by Spanish and Hindi; Mandarin Chinese showed neither (P = .087). Trust, workload, and ease-of-use remained consistent across languages. Participants described PCP-Bot as natural and accurate overall but reported repetition, transcription errors, and more frequent breakdowns in non-English interactions.Discussion\n\nMultilingual capability in pre-visit workflows appears feasible for LLMs under controlled conditions in the 4 high-resource languages evaluated, with high absolute performance in each. Disparities were language-dependent: conversational performance declined most for Spanish and Hindi, while summary quality showed a small, uniform English advantage, suggesting addressable language-specific weaknesses rather than a fixed limitation of English-developed systems.Conclusion\n\nLarge language models can support multilingual pre-visit interactions, but performance varies by language. Per-language validation and further testing are needed prior to clinical rollout.",
  "abstract_shown": true,
  "affiliations": [
    "AI-4-AI Lab, University of Pennsylvania, 3700 Hamilton Walk, Philadelphia, PA, 19104, United States.",
    "Palliative and Advanced Illness Research (PAIR) Center, Perelman School of Medicine, University of Pennsylvania, 3600 Civic Center Boulevard 7th Floor East, Philadelphia, PA, 19104, United States."
  ],
  "article_type": "research article",
  "arxiv": null,
  "author_total": 5,
  "authors": [
    {
      "affs": [
        0
      ],
      "name": "Pei-Lun Chen",
      "orcid": "0009-0009-9061-3399"
    },
    {
      "affs": [
        0
      ],
      "name": "Amogh Ananda Rao"
    },
    {
      "affs": [
        0
      ],
      "name": "Sydney Pugh",
      "orcid": "0000-0002-1465-983X"
    },
    {
      "affs": [
        1
      ],
      "name": "Yingying Lu"
    },
    {
      "affs": [
        0
      ],
      "name": "Kevin B Johnson"
    }
  ],
  "bibtex": "https://research.draconitic.com/paper/fk2a7n93zzjkf.bib",
  "cited_by": 0,
  "doi": "10.1093/jamiaopen/ooag207",
  "fulltext_url": "https://europepmc.org/articles/PMC13694105",
  "id": "fk2a7n93zzjkf",
  "issns": [
    "2574-2531"
  ],
  "landing_url": "https://europepmc.org/article/MED/42921364",
  "language": "eng",
  "licence": "CC-BY-NC",
  "licence_label": "CC BY-NC",
  "licence_source": "article",
  "licence_url": "https://creativecommons.org/licenses/by-nc/4.0/",
  "metadata_licence": null,
  "native_subjects": [
    [
      "openalex",
      "2718|Artificial Intelligence in Healthcare and Education"
    ]
  ],
  "openalex": "W7223392010",
  "pdf_url": "https://europepmc.org/articles/PMC13694105?pdf=render",
  "pmcid": "PMC13694105",
  "pmid": "42921364",
  "published": "2026-10-23",
  "publisher": null,
  "ris": "https://research.draconitic.com/paper/fk2a7n93zzjkf.ris",
  "sources": [
    {
      "first_seen": "2026-10-30T12:20:33Z",
      "native_id": "MED:42921364",
      "source": "europepmc",
      "url": "https://europepmc.org/article/MED/42921364"
    }
  ],
  "subjects": [
    {
      "field": "medicine",
      "origin": "openalex:2718|Artificial Intelligence in Healthcare and Education",
      "subfield": "epidemiology-public-health"
    }
  ],
  "title": "Multilingual evaluation of a large language model-based primary care chatbot.",
  "title_text": "Multilingual evaluation of a large language model-based primary care chatbot.",
  "type": "peer-reviewed",
  "updated_at": "2026-10-30T12:20:33Z",
  "url": "https://research.draconitic.com/paper/fk2a7n93zzjkf",
  "venue": "JAMIA open",
  "versions": []
}