Skepticism vs. Convenience: Physics Students' Perceptions and Use of Large Language Models Before and After Instruction
Abstract
The recent emergence of large language models (LLMs) has produced research focusing on the ability of these tools to solve physics problems, evaluate student work, or otherwise impact the problem-solving process of students. However, studies exploring how physics students perceive LLMs (in terms of capabilities, educational impacts, usage, and role in problem solving) remain limited. This study evaluates the first-year physics students' perceptions toward LLMs and further explores how these perceptions change after practicing problem-solving with and without LLMs and engaging in a reflective lesson on the functioning and educational impacts of LLMs. We find that student opinions toward LLMs vary, with generally favorable perceptions of their capabilities but greater skepticism regarding their value for learning. Despite this skepticism, a majority of students self-report regularly using LLMs to obtain help, commonly reporting deadlines and convenience as motivating factors. Following the lesson, students expressed greater skepticism toward LLMs in several areas, with 88% of students believing that LLMs can leave them with a false sense of confidence about their understanding, up from 58% before the lesson.
Related
- Correlation Between AI Usage and Students' Perceptions in Learning Kirchhoff's Law at Senior High School in South Jakarta
- A Hardware-Native Trapped-Ion Hybrid Qudit-Oscillator Instruction Set
- Architecting Instruction Execution for Scalable Fault-Tolerant Quantum Computing
- Time-Deterministic and Instruction-Dynamic Execution: A field-programmable gate array quantum-control processor
- Dirac notation: difficulties of the third-year undergraduate physics students
BibTeX
@misc{obrien2026skepticism,
title = {{Skepticism vs. Convenience: Physics Students' Perceptions and Use of Large Language Models Before and After Instruction}},
author = {Jake O'Brien and Margaret Ellis and Alma Robinson and John Simonetti and Naren Ramakrishnan},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.21037},
eprint = {2609.21037},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.21037},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Skepticism vs. Convenience: Physics Students' Perceptions and Use of Large Language Models Before and After Instruction
AU - Jake O'Brien
AU - Margaret Ellis
AU - Alma Robinson
AU - John Simonetti
AU - Naren Ramakrishnan
PY - 2026
DA - 2026/09/17
PB - arXiv
DO - 10.48550/arxiv.2609.21037
UR - https://arxiv.org/abs/2609.21037
AB - The recent emergence of large language models (LLMs) has produced research focusing on the ability of these tools to solve physics problems, evaluate student work, or otherwise impact the problem-solving process of students. However, studies exploring how physics students perceive LLMs (in terms of capabilities, educational impacts, usage, and role in problem solving) remain limited. This study evaluates the first-year physics students' perceptions toward LLMs and further explores how these perceptions change after practicing problem-solving with and without LLMs and engaging in a reflective lesson on the functioning and educational impacts of LLMs. We find that student opinions toward LLMs vary, with generally favorable perceptions of their capabilities but greater skepticism regarding their value for learning. Despite this skepticism, a majority of students self-report regularly using LLMs to obtain help, commonly reporting deadlines and convenience as motivating factors. Following the lesson, students expressed greater skepticism toward LLMs in several areas, with 88% of students believing that LLMs can leave them with a false sense of confidence about their understanding, up from 58% before the lesson.
ER -