From Retrieval to Reflection: Profiling the Cognitive Levels and Stimulus Characteristics of Summative Reading Literacy Items in Indonesia

Abstract

Indonesia faces significant literacy challenges, as evidenced by persistently low Programme for International Student Assessment (PISA) scores. To address this issue, a multifaceted analysis is required, particularly examining the characteristics of literacy questions featured in Indonesian language assessments administered to students. One critical factor contributing to poor literacy outcomes is the quality of instruction, which necessitates further scrutiny of assessment design to evaluate the efficacy of learning materials. This study investigates the structural and cognitive dimensions of reading literacy questions in Indonesian language assessments to identify potential shortcomings. Employing a descriptive research approach, the study analyzes the typology of questions, cognitive demand levels (including 1) information retrieval, 2) comprehension, and 3) evaluation and reflection), question quantity, and textual stimuli utilized in summative assessments for eleventh-grade high school students. The findings reveal that the assessments predominantly consist of multiple-choice questions, with a total of 30–50 items per test. The distribution of cognitive levels was uneven: Level 1 (basic information retrieval) accounted for 17% of questions, Level 2 (comprehension) constituted 66%, while Level 3 (evaluation and reflection) represented only 4%. Additionally, the textual stimuli were primarily presented as excerpts or fragmented continuous texts. These findings are subject to limitations, as the study did not account for teacher backgrounds or testing infrastructure conditions. Nevertheless, the results provide valuable insights for Indonesian language educators, serving as a foundation for refining assessment strategies and diversifying evaluation methods to enhance literacy development.