{
  "abstract": "A reproducible public-data benchmark (not a GMP line qualification) asking, with honest metrics, whether frontier AI can be trusted in the decision step of pharmaceutical visual inspection. Built on the public ResiDual dataset (Rosati, Marchesini, Lumetti, et al., \"Identifying Impurities in Liquids of Pharmaceutical Vials,\" ICPR 2024, DOI 10.1007/978-3-031-78447-7_18; 730 spin-stop sequences / 13,880 frames / 5 classes). Images are NOT redistributed — only code, predictions, metrics, and audit trails; obtain ResiDual from UNIMORE AImageLab and cite the authors.\n\nStudy 1 (pre-registered): a trained ResNet-18 baseline vs a zero-shot panel (GPT-4o, GPT-5.1, Gemini-2.5-Pro, Qwen2.5-VL-72B, Claude-Sonnet-5.5, Grok-4.7) and two Jev 1.14 arms (image + text-judge), scored sequence-level with recall / specificity / AUROC + silent-miss + ECE calibration + per-impurity recall. Finding: VLMs rank dirty above clean well (AUROC 0.81–0.99) but cannot be trusted to decide zero-shot — they either reject most clean vials or silently miss defects; no model is calibrated (ECE 0.13–0.41); a small trained CNN solves the clean benchmark.\n\nStudy 2: an under-trained (imperfect) CNN baseline on degraded images (recall 1.00, specificity 0.656, 11 false-rejects, 0 misses), then a decision-model shadow layer (Jev 1.14, pre-registered; plus exploratory OpenAI GPT-6 Luna Decisions API and GPT-6.1 chat) tested for whether it can recover false-rejects without clearing real defects. Cross-vendor finding: no decision-model, in either mode, is a safe recovery layer — every layer that recovers false-rejects also clears real defects; the only \"safe\" ones recover almost nothing. This holds across two independent vendors (TypeSafe Jev and OpenAI Luna Decisions).\n\nScope / honesty: public-data benchmark on (degraded) research images, NOT a validated GMP line; Jev 1.14 image = experimental preview snapshot; the GPT-6.1/Luna arms were not in the frozen pre-registration (labeled exploratory); zero-shot contamination not excludable (the trained CNN is the controlled anchor); \"PoD\" is a class-level proxy. Pre-registered on OSF (DOI 10.17605/OSF.IO/UH9ZA; protocol frozen before analysis, git commit fe4dba9). Independent research; no employer; no monetization.",
  "abstract_shown": true,
  "affiliations": [],
  "article_type": null,
  "arxiv": null,
  "author_total": 1,
  "authors": [
    {
      "name": "Gourav Pandey",
      "orcid": "0009-0003-7755-8707"
    }
  ],
  "bibtex": "https://research.draconitic.com/paper/20hbhgzzkjcdn.bib",
  "cited_by": 0,
  "doi": "10.5281/zenodo.23462165",
  "fulltext_url": null,
  "id": "20hbhgzzkjcdn",
  "issns": [],
  "landing_url": "https://zenodo.org/records/23462165",
  "language": null,
  "licence": "CC-BY-4.0",
  "licence_label": "CC BY 4.0",
  "licence_source": "article",
  "licence_url": "https://creativecommons.org/licenses/by/4.0/",
  "metadata_licence": null,
  "native_subjects": [],
  "openalex": null,
  "pdf_url": "https://zenodo.org/api/records/23462165/files/ShadowMode_Vial_Benchmark_preprint.pdf/content",
  "pmcid": null,
  "pmid": null,
  "published": "2026-10-19",
  "publisher": null,
  "ris": "https://research.draconitic.com/paper/20hbhgzzkjcdn.ris",
  "sources": [
    {
      "first_seen": "2026-11-02T12:21:55Z",
      "native_id": "23462165",
      "source": "zenodo",
      "url": "https://zenodo.org/records/23462165"
    }
  ],
  "subjects": [],
  "title": "A Shadow-Mode Benchmark of Frontier Vision-Language and Decision Models for Parenteral Vial Inspection",
  "title_text": "A Shadow-Mode Benchmark of Frontier Vision-Language and Decision Models for Parenteral Vial Inspection",
  "type": "preprint",
  "updated_at": "2026-11-02T12:21:55Z",
  "url": "https://research.draconitic.com/paper/20hbhgzzkjcdn",
  "venue": "Zenodo",
  "versions": []
}