Skip to main content
Article
Automated writing evaluation for formative assessment of second language writing: investigating the accuracy and usefulness of feedback as part of argument-based validation
Educational Psychology: An International Journal of Experimental Educational Psychology
  • Jim Ranalli, Iowa State University
  • Stephanie Link, Iowa State University
  • Evgeny Chukharev-Khudilaynen, Iowa State University
Document Type
Article
Publication Version
Accepted Manuscript
Publication Date
2-1-2016
DOI
10.1080/01443410.2015.1136407
Abstract

An increasing number of studies on the use of tools for automated writing evaluation (AWE) in writing classrooms suggest growing interest in their potential for formative assessment. As with all assessments, these applications should be validated in terms of their intended interpretations and uses. A recent argument-based validation framework outlined inferences that require backing to support integration of one AWE tool, Criterion, into a college-level English as a Second Language (ESL) writing course. The present research appraised evidence for the assumptions underlying two inferences in this argument. In the first of two studies, we assessed evidence for the evaluation inference, which includes the assumption that Criterion provides students with accurate feedback. The second study focused on the utilisation inference involving the assumption that Criterion feedback is useful for students to make decisions about revisions. Results showed accuracy varied considerably across error types, as did students’ abilities to use Criterion feedback to correct written errors. The findings can inform discussion of whether and how to integrate the use of AWE into writing classrooms while raising important questions regarding standards for validation of AWE as formative assessment, Criterion developers’ approach to accuracy, and instructors’ assumptions about the underlying purposes of AWE-based writing activities.

Comments

This is an Accepted Manuscript of an article published by Taylor & Francis in Educational Psychology: An International Journal of Experimental Educational Psychology on February 1, 2016, available online: http://www.tandf.com/10.1080/01443410.2015.1136407.

Copyright Owner
Taylor & Francis
Language
en
File Format
application/pdf
Citation Information
Jim Ranalli, Stephanie Link and Evgeny Chukharev-Khudilaynen. "Automated writing evaluation for formative assessment of second language writing: investigating the accuracy and usefulness of feedback as part of argument-based validation" Educational Psychology: An International Journal of Experimental Educational Psychology (2016)
Available at: http://works.bepress.com/jim-ranalli/7/