{
  "abstract": "BackgroundImage-based dietary assessment is valuable for precision diabetes care. Multimodal large language models (mLLMs) may substantially facilitate dietary self-monitoring. However, comprehensive evaluation of their real-world accuracy remains limited.\n\nMethodsWe developed SeeFood, an open-source benchmark, and evaluated eight commercial and open-source mLLMs on 80 real-world meal photographs from the SNAPMe dataset for nutrient and energy estimation (calories, carbohydrates, protein, fat, fiber), food-item identification, and day-level diet quality (Healthy Eating Index, HEI-2020). Three finalists were evaluated on the complete dataset (1680 photographs, 275 participant-days), and the best model was analyzed by meal type, portion size, sizing-marker use, ingredient visibility and meal complexity.\n\nResultsGPT-4o mini achieved the lowest errors, with a significant advantage for carbohydrates; the three finalists performed similarly on other outcomes and on HEI-2020. Mean absolute percentage errors (MAPE) remained high (52%-90%), and the direction of bias was model-specific. The carbohydrate mean absolute error was 13.3 g (95% CI 12.4-14.3), and only 57% of meals were within {+/-}10 g, a clinically relevant threshold for insulin dosing in type 1 diabetes. Calorie errors for lunch and dinner were in the range reported for human estimation on other images ([~]55% MAPE), whereas drink-only images had the highest relative errors (312% MAPE) because of dissolved sugars. Food items were reliably identified when fully visible (Dice = 0.74), and HEI-2020 estimates showed moderate agreement with reference values (RMSE = 12.5 points; r = 0.61). Larger portions were increasingly underestimated, and sizing references did not improve accuracy. Newer or larger models and chain-of-thought prompting did not consistently improve performance.\n\nConclusionsCurrent mLLMs can extract meaningful dietary information from real-world meal photographs, but substantial errors in nutrient quantification and portion-size estimation limit their use for precision diabetes management. Integrating visual assessment with additional contextual information may enable more reliable, lower-burden dietary monitoring.",
  "abstract_shown": true,
  "affiliations": [
    "Stanford University"
  ],
  "article_type": "PUBLISHAHEADOFPRINT",
  "arxiv": null,
  "author_total": 13,
  "authors": [
    {
      "name": "L. Gravellier"
    },
    {
      "name": "C. Zhang"
    },
    {
      "name": "Y. Wu"
    },
    {
      "name": "V. Parameswaran"
    },
    {
      "name": "A. Scheideman"
    },
    {
      "name": "M. Shao"
    },
    {
      "name": "E. Zhang"
    },
    {
      "name": "D. Perelman"
    },
    {
      "name": "C. D. Gardner"
    },
    {
      "name": "D. C. Klonoff"
    },
    {
      "name": "H. Raeder"
    },
    {
      "name": "M. P. Snyder"
    },
    {
      "affs": [
        0
      ],
      "name": "T. Wang"
    }
  ],
  "bibtex": "https://research.draconitic.com/paper/eaknx85byjwka.bib",
  "cited_by": 0,
  "doi": "10.64898/2026.10.21.26365949",
  "fulltext_url": "https://www.medrxiv.org/content/10.64898/2026.10.21.26365949v1.full",
  "id": "eaknx85byjwka",
  "issns": [],
  "landing_url": "https://www.medrxiv.org/content/10.64898/2026.10.21.26365949v1",
  "language": "en",
  "licence": "CC-BY-NC-ND",
  "licence_label": "CC BY-NC-ND",
  "licence_source": "article",
  "licence_url": "https://creativecommons.org/licenses/by-nc-nd/4.0/",
  "metadata_licence": null,
  "native_subjects": [
    [
      "heading",
      "Article"
    ],
    [
      "medrxiv",
      "health informatics"
    ],
    [
      "openalex",
      "2718|Artificial Intelligence in Healthcare and Education"
    ]
  ],
  "openalex": "W7223460275",
  "pdf_url": "https://www.medrxiv.org/content/10.64898/2026.10.21.26365949v1.full.pdf",
  "pmcid": "PMC13738896",
  "pmid": "42996283",
  "published": "2026-10-21",
  "publisher": "Cold Spring Harbor Laboratory Preprints",
  "ris": "https://research.draconitic.com/paper/eaknx85byjwka.ris",
  "sources": [
    {
      "first_seen": "2026-10-30T12:21:40Z",
      "native_id": "10.64898/2026.10.21.26365949",
      "source": "medrxiv",
      "url": "https://www.medrxiv.org/content/10.64898/2026.10.21.26365949v1"
    },
    {
      "first_seen": "2026-11-09T12:23:48Z",
      "native_id": "PMC13738896",
      "source": "pmc",
      "url": "https://doi.org/10.64898/2026.10.21.26365949"
    }
  ],
  "subjects": [
    {
      "field": "medicine",
      "origin": "medrxiv:health informatics",
      "subfield": "epidemiology-public-health"
    }
  ],
  "title": "Image-Based Dietary Assessment Using Multimodal Large Language Models: Performance, Limitations, and Clinical Applicability in Diabetes",
  "title_text": "Image-Based Dietary Assessment Using Multimodal Large Language Models: Performance, Limitations, and Clinical Applicability in Diabetes",
  "type": "preprint",
  "updated_at": "2026-11-09T12:23:48Z",
  "url": "https://research.draconitic.com/paper/eaknx85byjwka",
  "venue": "medRxiv",
  "versions": []
}