Contribution to Book
How Can the Use of Data from Computer-Delivered Assessments Improve the Measurement of Twenty-First Century Skills?
Assessment and Teaching of 21st Century Skills: Research and applications
(2018)
Abstract
Technology is continuing to change the way we live. Given its centrality to our lives, it is not surprising that its use in educational assessment has been increasing, and has been an important focus of education assessment research in recent years. While the initial motivation for computer-delivered assessments was gains in assessment efficiency, this chapter demonstrates that computer delivery can enhance validity and reliability through the capture of process data. When an assessment is computer-delivered, every interaction of the test-taker with the environment may be recorded as process data. The use of process data holds much promise for providing previously inaccessible insights into not just whether a student solved a task, but how they did so. Further, it is the processes that contribute to twenty-first century skills that are likely to be amenable to direct targeting in terms of teaching and learning. However, collecting large amounts of information in the absence of a plan for its analysis and use is unlikely to lead to useful outcomes. Through item response theory analysis of process data collected in the Digital Reading Assessment included as part of the 2012 cycle of the Programme for International Student Assessment (PISA), this chapter illustrates how process data that relates to the way a student navigates the problem space can be used to improve validity and reliability. By fitting alternative item response models to the data, it is shown that measurement can be improved by using process data if a clear connection is made between these data and theories of developing competence in the domain of interest.
Keywords
- student assessment,
- digital reading,
- reading skills
Disciplines
Publication Date
January 1, 2018
Editor
Mark Wilson, Patrick Griffin & Esther Care
Publisher
Springer
Series
Educational Assessment in an Information Age
ISBN
9783319653686
DOI
10.1007/978-3-319-65368-6_13
Citation Information
Ramalingam, D., & Adams, R.J. (2018). How Can the Use of Data from Computer-Delivered Assessments Improve the Measurement of Twenty-First Century Skills?. In: Care E., Griffin P., Wilson M. (eds) Assessment and Teaching of 21st Century Skills. Educational Assessment in an Information Age. Springer, Cham