{
  "abstract": "Understanding how humans and artificial intelligence systems predict and plan by interacting with their environment is a fundamental challenge at the intersection of neuroscience and machine learning. Most brain-encoding studies focus on aligning artificial models with brain activity during language comprehension or passive visual processing, while interactive brain alignment studies have to date been largely limited to reinforcement-learning (RL) agents and theory-based models. To address this gap, we study brain alignment of representative models from two foundation-model types, namely vision-language models (VLMs) and large-action models (LAMs), using fMRI recordings from participants playing naturalistic Atari-style video games. Specifically, we examine how action-focused and reasoning-focused prompts shape the models' internal representations and their alignment with fMRI brain activity. First, we find that both VLMs and LAMs achieve significantly higher voxel-wise encoding performance than RL baselines, with the advantage holding even under matched feature dimensionality. Second, compared to a no-prompt baseline, prompt-driven gains are larger in higher-order frontal-parietal and motor-planning regions than in early visual cortex, roughly 2-2.5$\\times$ when averaged over region-of-interest (ROI) groups, although individual regions are heterogeneous. Third, variance partitioning reveals a qualitatively different representational organization. VLM representations are prompt-symmetric (12.4% unique action vs. 9.5% unique reasoning), whereas LAM representations are action-dominant (25.6% unique action vs.-8.2% unique reasoning), with the asymmetry strongest in frontal-motor cortex. Together, these results associate action specialization with distinct cortical alignment patterns in multimodal game-state representations, revealing differences hidden by similar prediction accuracy.",
  "abstract_shown": true,
  "affiliations": [],
  "article_type": null,
  "arxiv": "2605.19352",
  "author_total": 7,
  "authors": [
    {
      "name": "Subba Reddy Oota"
    },
    {
      "name": "Anant Khandelwal"
    },
    {
      "name": "Khushbu Pahwa"
    },
    {
      "name": "Satya Sai Srinath Namburi"
    },
    {
      "name": "Tanmoy Chakraborty"
    },
    {
      "name": "Bapi S. Raju"
    },
    {
      "name": "Manish Gupta"
    }
  ],
  "bibtex": "https://research.draconitic.com/paper/4v2073kz49vw2.bib",
  "cited_by": 0,
  "doi": "10.48550/arxiv.2605.19352",
  "fulltext_url": null,
  "id": "4v2073kz49vw2",
  "issns": [],
  "landing_url": "https://arxiv.org/abs/2605.19352",
  "language": null,
  "licence": "CC-BY-4.0",
  "licence_label": "CC BY 4.0",
  "licence_source": "article",
  "licence_url": "http://creativecommons.org/licenses/by/4.0/",
  "metadata_licence": "CC0-1.0",
  "native_subjects": [
    [
      "arxiv",
      "cs.AI"
    ],
    [
      "arxiv",
      "cs.LG"
    ],
    [
      "arxiv",
      "q-bio.NC"
    ],
    [
      "openalex",
      "1707|Multimodal Machine Learning Applications"
    ]
  ],
  "openalex": "W7161772368",
  "pdf_url": "https://arxiv.org/pdf/2605.19352",
  "pmcid": null,
  "pmid": null,
  "published": "2026-10-07",
  "publisher": null,
  "ris": "https://research.draconitic.com/paper/4v2073kz49vw2.ris",
  "sources": [
    {
      "first_seen": "2026-10-30T12:27:43Z",
      "native_id": "2605.19352",
      "source": "arxiv",
      "url": "https://arxiv.org/abs/2605.19352"
    }
  ],
  "subjects": [
    {
      "field": "biology",
      "origin": "arxiv:q-bio.NC",
      "subfield": "neuroscience"
    }
  ],
  "title": "Brain alignment of reasoning and action representations from vision-language and action models during naturalistic gameplay",
  "title_text": "Brain alignment of reasoning and action representations from vision-language and action models during naturalistic gameplay",
  "type": "preprint",
  "updated_at": "2026-10-30T12:27:43Z",
  "url": "https://research.draconitic.com/paper/4v2073kz49vw2",
  "venue": "arXiv",
  "versions": []
}