TY - JOUR
T1 - A tale of two tests
T2 - The role of topic and general academic knowledge in traditional versus contemporary scenario-based reading
AU - Wang, Zuowei
AU - O'Reilly, Tenaha
AU - Sabatini, John
AU - McCarthy, Kathryn S.
AU - McNamara, Danielle S.
N1 - Funding Information:
The research reported here was supported by the U.S. Department of Education Institute of Education Sciences, Award No. R305A150176 to Educational Testing Service and a subaward to Arizona State University, and IES Award R305A180144 and Office of Naval Research award, N00014-17-1-2300, to Arizona State University. The opinions expressed are those of the authors and do not represent views of the U.S. Department of Education, ONR, Educational Testing Service or Arizona State University. Special thanks to Kelsey Dreier, Kelly Bruce, Kietha Biggers, Jonathan Steinberg, Jonathan Weeks, Szu-Fu Chao, Don Powers, Mo Zhang, and Paul Deane as well as Kevin Kent, Tricia Guerrero, Ashleigh Horowitz, and Joseph Aubele with their help with various aspects of the project.
Publisher Copyright:
© 2021 Elsevier Ltd
PY - 2021/6
Y1 - 2021/6
N2 - We compared high school students’ performance in a traditional comprehension assessment requiring them to identify key information and draw inferences from single texts, and a scenario-based assessment (SBA) requiring them to integrate, evaluate and apply information across multiple sources. Both assessments focused on a non-academic topic. Performance on the two assessments were moderately correlated (r = 0.57), but the SBA was more difficult (Study 1; n = 342). The two assessments similarly depended on basic reading skills but diverged in the relation to academic knowledge and (non-academic) topic knowledge (Study 2; n = 1107). Academic knowledge was highly predictive of traditional comprehension, but less so for SBA. Topic knowledge was more predictive of SBA than traditional comprehension. Thus, the two assessments tap into similar constructs related to comprehension; however, the level of topic knowledge is more important for performance on scenario-based, multiple-source reading tasks, whereas academic knowledge is more important for traditional reading comprehension tasks.
AB - We compared high school students’ performance in a traditional comprehension assessment requiring them to identify key information and draw inferences from single texts, and a scenario-based assessment (SBA) requiring them to integrate, evaluate and apply information across multiple sources. Both assessments focused on a non-academic topic. Performance on the two assessments were moderately correlated (r = 0.57), but the SBA was more difficult (Study 1; n = 342). The two assessments similarly depended on basic reading skills but diverged in the relation to academic knowledge and (non-academic) topic knowledge (Study 2; n = 1107). Academic knowledge was highly predictive of traditional comprehension, but less so for SBA. Topic knowledge was more predictive of SBA than traditional comprehension. Thus, the two assessments tap into similar constructs related to comprehension; however, the level of topic knowledge is more important for performance on scenario-based, multiple-source reading tasks, whereas academic knowledge is more important for traditional reading comprehension tasks.
KW - Assessment
KW - Basic reading skills
KW - Multiple text comprehension
KW - Prior knowledge
KW - Scenario-based reading
UR - http://www.scopus.com/inward/record.url?scp=85101720951&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85101720951&partnerID=8YFLogxK
U2 - 10.1016/j.learninstruc.2021.101462
DO - 10.1016/j.learninstruc.2021.101462
M3 - Article
AN - SCOPUS:85101720951
SN - 0959-4752
VL - 73
JO - Learning and Instruction
JF - Learning and Instruction
M1 - 101462
ER -