Mechanics Cognitive Diagnostic: Testing Fine-Grained Learning Objectives in Introductory Physics
Abstract
Physics courses use research-based assessments (RBAs) such as the Force Concept Inventory (FCI), Force and Motion Conceptual Evaluation (FMCE), and Energy and Momentum Conceptual Survey (EMCS) to measure learning in introductory mechanics, but their fixed-length, pretest-posttest design makes them retrospective: posttest scores summarize completed instruction and arrive after a course ends. We are developing the Mechanics Cognitive Diagnostic (MCD), a cognitive diagnostic computerized adaptive test that reports students' mastery of fine-grained learning objectives (LOs) throughout instruction. Using evidence-centered design, we defined 14 LOs from introductory mechanics textbooks and AP Physics standards, mapped FCI, FMCE, and EMCS items onto them with a Q-matrix, and refined the mapping with the deterministic inputs, noisy "and" gate (DINA) model, using posttest responses from 24,394 students in 807 courses across 79 institutions through LASSO. The FCI and EMCS achieved good DINA model fit; the FMCE showed marginal fit. Classification accuracy for most LOs met or exceeded benchmarks for low-stakes formative assessment. RBA items, though not developed for LO-level diagnosis, support it reliably, giving the MCD a working 14-LO item bank built from RBAs that physics courses already use. As data accumulate, we can revise or retire weak LOs and items and add new items through online calibration without interrupting testing. We plan to expand the MCD to 35 LOs, two per week, to cover a typical introductory mechanics course.
Related
- Active Learning with Generative AI in Introductory Physics: A Multi-Institutional Study of the P-A.I.R. Framework
- Diversity-Aware Active Learning Builds Materials Datasets for Current and Future Objectives
- The Mechanics of Delta Learning: Target Design for Generalizable Scientific Machine Learning
- Learning consistent molecular mechanics force fields from first principles
- Finite-step statistical mechanics of scale-invariant learning: spectral equations of state and curvature transport
BibTeX
@misc{le2026mechanics,
title = {{Mechanics Cognitive Diagnostic: Testing Fine-Grained Learning Objectives in Introductory Physics}},
author = {Vy Le and Jayson M. Nissen and Jason W. Morphew and Hua Hua Chang and Ben Van Dusen},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.09584},
eprint = {2609.09584},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.09584},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Mechanics Cognitive Diagnostic: Testing Fine-Grained Learning Objectives in Introductory Physics
AU - Vy Le
AU - Jayson M. Nissen
AU - Jason W. Morphew
AU - Hua Hua Chang
AU - Ben Van Dusen
PY - 2026
DA - 2026/09/09
PB - arXiv
DO - 10.48550/arxiv.2609.09584
UR - https://arxiv.org/abs/2609.09584
AB - Physics courses use research-based assessments (RBAs) such as the Force Concept Inventory (FCI), Force and Motion Conceptual Evaluation (FMCE), and Energy and Momentum Conceptual Survey (EMCS) to measure learning in introductory mechanics, but their fixed-length, pretest-posttest design makes them retrospective: posttest scores summarize completed instruction and arrive after a course ends. We are developing the Mechanics Cognitive Diagnostic (MCD), a cognitive diagnostic computerized adaptive test that reports students' mastery of fine-grained learning objectives (LOs) throughout instruction. Using evidence-centered design, we defined 14 LOs from introductory mechanics textbooks and AP Physics standards, mapped FCI, FMCE, and EMCS items onto them with a Q-matrix, and refined the mapping with the deterministic inputs, noisy "and" gate (DINA) model, using posttest responses from 24,394 students in 807 courses across 79 institutions through LASSO. The FCI and EMCS achieved good DINA model fit; the FMCE showed marginal fit. Classification accuracy for most LOs met or exceeded benchmarks for low-stakes formative assessment. RBA items, though not developed for LO-level diagnosis, support it reliably, giving the MCD a working 14-LO item bank built from RBAs that physics courses already use. As data accumulate, we can revise or retire weak LOs and items and add new items through online calibration without interrupting testing. We plan to expand the MCD to 35 LOs, two per week, to cover a typical introductory mechanics course.
ER -