{
  "abstract": "EHR foundation models read medical events within a fixed context window, and long context has so far been measured only in tokens. How far back in time a model must look to capture the earliest predictive signal is unknown. We introduce time-context, the calendar lookback window before the prediction point, and provide, to our knowledge, the first systematic evaluation of decade-scale time-context for an EHR foundation model. We pretrained a decoder-only transformer (34.6M parameters) on 1.4B coded events from 3.6M primary-care patients in North West London and fine-tuned it to predict incident colorectal cancer under a 12-month pre-diagnostic buffer. A frozen cohort of 1,008 cases and 9,582 matched controls, all registered for at least 30 years, was evaluated at 15 nested lookback windows from 2 to 30 years, so that every truncation is compared within the same patients. Discrimination rose from an AUROC of 0.551 at 2 years to 0.590 at 18 years, then plateaued, with about 80% of the gain reached within 10 years; an XGBoost baseline converged to the same level (peak AUROC 0.589). A simulation further suggests that rotary position embeddings (RoPE) over elapsed time alias beyond about 20 years at the default base frequency, motivating a retuned $θ$. Time-context is a measurable design axis for clinical prediction models, distinct from token-context, and should be reported.",
  "abstract_shown": true,
  "affiliations": [],
  "article_type": null,
  "arxiv": "2611.07265",
  "author_total": 3,
  "authors": [
    {
      "name": "Achille Raviez"
    },
    {
      "name": "Lan Wang"
    },
    {
      "name": "Brendan Delaney"
    }
  ],
  "bibtex": "https://research.draconitic.com/paper/azh731d5q15jc.bib",
  "cited_by": 0,
  "doi": "10.48550/arxiv.2611.07265",
  "fulltext_url": null,
  "id": "azh731d5q15jc",
  "issns": [],
  "landing_url": "https://arxiv.org/abs/2611.07265",
  "language": null,
  "licence": "CC-BY-4.0",
  "licence_label": "CC BY 4.0",
  "licence_source": "article",
  "licence_url": "http://creativecommons.org/licenses/by/4.0/",
  "metadata_licence": "CC0-1.0",
  "native_subjects": [
    [
      "arxiv",
      "cs.AI"
    ],
    [
      "arxiv",
      "cs.LG"
    ],
    [
      "arxiv",
      "q-bio.QM"
    ]
  ],
  "openalex": null,
  "pdf_url": "https://arxiv.org/pdf/2611.07265",
  "pmcid": null,
  "pmid": null,
  "published": "2026-11-05",
  "publisher": null,
  "ris": "https://research.draconitic.com/paper/azh731d5q15jc.ris",
  "sources": [
    {
      "first_seen": "2026-11-06T18:21:35Z",
      "native_id": "2611.07265",
      "source": "arxiv",
      "url": "https://arxiv.org/abs/2611.07265"
    }
  ],
  "subjects": [
    {
      "field": "biology",
      "origin": "arxiv:q-bio.QM",
      "subfield": "bioinformatics"
    }
  ],
  "title": "How Deep is the Signal? Time-Context up to 30 Years for Colorectal Cancer Prediction in a Primary-Care EHR Model",
  "title_text": "How Deep is the Signal? Time-Context up to 30 Years for Colorectal Cancer Prediction in a Primary-Care EHR Model",
  "type": "preprint",
  "updated_at": "2026-11-09T00:20:50Z",
  "url": "https://research.draconitic.com/paper/azh731d5q15jc",
  "venue": "arXiv",
  "versions": []
}