{
  "abstract": "Background: Artificial intelligence (AI) chatbots are increasingly used by medical students for learning and examination preparation. However, their reliability in mechanistically demanding disciplines such as respiratory physiology remains insufficiently evaluated using authentic examination material. This study aimed to assess and compare the performance of three AI chatbots on validated undergraduate respiratory physiology examination questions. Methods: We conducted a cross-sectional analytical study including all respiratory physiology multiple-choice questions (MCQs), each comprising five propositions, used in official undergraduate examinations at the Faculty of Medicine of Tunis during the academic years 2023–2024 and 2024–2025 (101 questions). Three AI chatbots (ChatGPT-5.1, Qwen-3 Max, and Perplexity AI) were evaluated using standardized French-language prompts. Exact question-level concordance with the faculty-validated answer key, established by the teaching staff responsible for respiratory physiology examinations, was compared using Cochran’s Q test and pairwise McNemar tests. At the proposition level, responses were analyzed as binary outcomes (true/false) to compute accuracy, sensitivity, and specificity. Results: Question-level exact concordance differed significantly across models (Cochran’s Q = 10.18, p = 0.006). ChatGPT showed the highest concordance rate (79.2%), followed by Qwen (74.3%) and Perplexity (62.4%). At the proposition level (505 propositions), ChatGPT achieved the highest overall accuracy (94.1%) and sensitivity for true propositions (96.5%), whereas Qwen demonstrated the highest specificity for false propositions (94.2%). Conclusion: Although all evaluated AI chatbots performed well on respiratory physiology MCQs, substantial variability was observed across models. These findings ..( abstract truncated at 250 words).",
  "abstract_shown": true,
  "affiliations": [
    "Laboratory of Physiology, Faculty of Medicine of Tunis, University of Tunis El Manar, Tunis, Tunisia"
  ],
  "article_type": "research article",
  "arxiv": null,
  "author_total": 6,
  "authors": [
    {
      "affs": [
        0
      ],
      "name": "Khouloud Kchaou"
    },
    {
      "affs": [
        0
      ],
      "name": "Salma Mokaddem"
    },
    {
      "affs": [
        0
      ],
      "name": "Soumaya Khaldi"
    },
    {
      "affs": [
        0
      ],
      "name": "Yacine Ouahchi"
    },
    {
      "affs": [
        0
      ],
      "name": "Khadija Ayed"
    },
    {
      "affs": [
        0
      ],
      "name": "Rym Baati"
    }
  ],
  "bibtex": "https://research.draconitic.com/paper/1j6x66116sgbf.bib",
  "cited_by": 0,
  "doi": "10.62438/tunismed.v104i05.6620",
  "fulltext_url": "https://pmc.ncbi.nlm.nih.gov/articles/PMC13751038/",
  "id": "1j6x66116sgbf",
  "issns": [
    "0041-4131",
    "2724-7031"
  ],
  "landing_url": "https://doi.org/10.62438/tunismed.v104i05.6620",
  "language": "en",
  "licence": "CC-BY-NC-ND-4.0",
  "licence_label": "CC BY-NC-ND 4.0",
  "licence_source": "article",
  "licence_url": "https://creativecommons.org/licenses/by-nc-nd/4.0/",
  "metadata_licence": null,
  "native_subjects": [
    [
      "heading",
      "Article"
    ],
    [
      "openalex",
      "2718|Artificial Intelligence in Healthcare and Education"
    ]
  ],
  "openalex": "W7162240367",
  "pdf_url": "https://www.latunisiemedicale.com/index.php/tunismed/article/download/6620/3904",
  "pmcid": "PMC13751038",
  "pmid": "42578767",
  "published": "2026-05-24",
  "publisher": "Tunisian Society of Medical Sciences",
  "ris": "https://research.draconitic.com/paper/1j6x66116sgbf.ris",
  "sources": [
    {
      "first_seen": "2026-11-14T12:23:30Z",
      "native_id": "PMC13751038",
      "source": "pmc",
      "url": "https://doi.org/10.62438/tunismed.v104i05.6620"
    }
  ],
  "subjects": [
    {
      "field": "medicine",
      "origin": "openalex:2718|Artificial Intelligence in Healthcare and Education",
      "subfield": "epidemiology-public-health"
    }
  ],
  "title": "Comparative benchmarking of three artificial intelligence chatbots (ChatGPT-5.1, Qwen- 3 Max, and Perplexity AI) on faculty validated undergraduate respiratory physiology multiple-choice examinations",
  "title_text": "Comparative benchmarking of three artificial intelligence chatbots (ChatGPT-5.1, Qwen- 3 Max, and Perplexity AI) on faculty validated undergraduate respiratory physiology multiple-choice examinations",
  "type": "peer-reviewed",
  "updated_at": "2026-11-14T12:23:30Z",
  "url": "https://research.draconitic.com/paper/1j6x66116sgbf",
  "venue": "La Tunisie Médicale",
  "versions": []
}