Developing and validating a productive thinking assessment for sustainable university science learning
Abstract
Assessing productive thinking in sustainable science learning requires instruments that capture coordinated cognitive and metacognitive processes during authentic problem-solving. Existing higher-order thinking assessments often examine critical thinking, creativity, or metacognition separately and provide limited validity evidence in performance-based sustainability contexts. This study developed and evaluated a performance-based Productive Thinking Assessment (PTA) measuring three interrelated dimensions: critical–analytical, creative thinking, and thinking process management within microplastic and bioplastic contexts. A multi-method instrument development design was used to obtain complementary validity evidence. Content validation involved seven experts, while psychometric evaluation involved 200 pre-service elementary science teachers from three Indonesian universities. Response-process evidence was examined using Think-Aloud Protocols with six students. Content validity was evaluated using Aiken’s V, internal structure using Principal Component Analysis (PCA) and PLS-SEM, item functioning using Rasch analysis, and scoring reliability using intraclass correlation coefficients. Aiken’s V coefficients ranged from 0.81 to 0.95. PCA supported the proposed three-dimensional structure, and PLS-SEM indicated high indicator reliability, internal consistency, and convergent validity. Rasch analysis showed acceptable item fit, with item difficulties ranging from − 0.829 to 1.289 logits and ordered, estimable rating-category thresholds. Inter-rater reliability ranged from moderate to good for individual ratings and from good to excellent for average ratings. The think-aloud findings illustrated evidence-based reasoning, solution generation, and metacognitive regulation during task completion. The findings provide complementary validity evidence supporting the interpretation of PTA scores for assessing productive thinking within sustainability-oriented science learning, while indicating the need for further validation across broader populations and science contexts.
Related
- Developing and Validating a Digital Literacy Scale for Young Children Aged 3–6: Associations with Problematic Media Use
- Developing and Validating a Patient-Centered Measure of Well-Being in Bipolar Disorder: The WELL-B.
- Strategies for establishing and developing overseas teaching hospitals : A mixed-methods case study of Nanjing Medical University
- Developing a sustainable dietary pattern for Korean adults using quadratic programming
- Diagnostic Performance of the Radiographic Assessment of Lung Edema Score for Acute Respiratory Distress Syndrome-Post Hoc Analysis of the "Validating Acute Lung Injury Biomarkers for Diagnosis" Study.
BibTeX
@article{sari2026developing,
title = {{Developing and validating a productive thinking assessment for sustainable university science learning}},
author = {Ratih Permana Sari and Sulistyo Saputro and Murni Ramli},
journal = {Scientific Reports},
year = {2026},
month = oct,
doi = {10.1038/s41598-026-73782-3},
url = {https://www.nature.com/articles/s41598-026-73782-3},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - Developing and validating a productive thinking assessment for sustainable university science learning
AU - Ratih Permana Sari
AU - Sulistyo Saputro
AU - Murni Ramli
PY - 2026
DA - 2026/10/04
JO - Scientific Reports
DO - 10.1038/s41598-026-73782-3
UR - https://www.nature.com/articles/s41598-026-73782-3
AB - Assessing productive thinking in sustainable science learning requires instruments that capture coordinated cognitive and metacognitive processes during authentic problem-solving. Existing higher-order thinking assessments often examine critical thinking, creativity, or metacognition separately and provide limited validity evidence in performance-based sustainability contexts. This study developed and evaluated a performance-based Productive Thinking Assessment (PTA) measuring three interrelated dimensions: critical–analytical, creative thinking, and thinking process management within microplastic and bioplastic contexts. A multi-method instrument development design was used to obtain complementary validity evidence. Content validation involved seven experts, while psychometric evaluation involved 200 pre-service elementary science teachers from three Indonesian universities. Response-process evidence was examined using Think-Aloud Protocols with six students. Content validity was evaluated using Aiken’s V, internal structure using Principal Component Analysis (PCA) and PLS-SEM, item functioning using Rasch analysis, and scoring reliability using intraclass correlation coefficients. Aiken’s V coefficients ranged from 0.81 to 0.95. PCA supported the proposed three-dimensional structure, and PLS-SEM indicated high indicator reliability, internal consistency, and convergent validity. Rasch analysis showed acceptable item fit, with item difficulties ranging from − 0.829 to 1.289 logits and ordered, estimable rating-category thresholds. Inter-rater reliability ranged from moderate to good for individual ratings and from good to excellent for average ratings. The think-aloud findings illustrated evidence-based reasoning, solution generation, and metacognitive regulation during task completion. The findings provide complementary validity evidence supporting the interpretation of PTA scores for assessing productive thinking within sustainability-oriented science learning, while indicating the need for further validation across broader populations and science contexts.
SN - 2045-2322
ER -