Presentation
Designing OSCE Scoring Instrument with Built-In Validity Features Based on A Comprehensive Measurement Framework: Principles & Guidelines
Ottawa Conference 2016
(2016)
Abstract
Pre-Conference Workshop
Designing OSCE Scoring instrument with built-in validity features based on a comprehensive measurement framework: Principles and guidelines
Author(s) – Professor Jane Courtney (Domain Chair Clinical and Communication Practice); Professor Carole Steketee (Associate Dean Teaching and Learning); Professor Donna Mak (Domain Chair Population and Preventive Health); Associate Professor Chris Skinner (Domain Chair Professional and Personal Development); ; Elina Tor (Associate Professor, Assessment and Psychometrics)
Affiliation(s) – School of Medicine Fremantle, The University of Notre Dame Australia
Presenters – All authors
Introduction
Capturing multiple dimensions of examinees’ performance is at the heart of criterion-reference assessment frameworks. The validity and reliability of OSCE scores will be enhanced if the assessment criteria includes the dimensions of content, process (both technical procedures and humanistic dimension of doctor patient interaction), and the overall outcome of patient care. This can be achieved through the use of scoring instruments that are designed based on a comprehensive measurement framework. This workshop aims to provide a platform for a community of practice (health professional educators and assessment practitioners) to deliberate on this measurement framework, and apply it in designing scoring instruments for OSCEs.
Objectives
By the end of the workshop, participants will be able to:
1. Understand a comprehensive measurement framework for OSCEs and incorporate it in designing an OSCE scoring instrument;
2. Understand and apply other quality assurance practices to ensure validity of OSCE scores interpretation
and use;
Content & Structure
Part I
- Short plenary introduction to a comprehensive measurement framework to underpin OSCE scoring;
- Small group activity to analyse and critique on sample OSCE scoring instruments
- Introducing multiple scoring scales and formats designed based on the framework for specific tasks assessed in OSCEs at the University of Notre Dame Fremantle
- Small group activity to design a scoring instrument with specific scoring scales appropriate for OSCE stations based on the measurement framework
*participants are invited to bring one or two existing OSCE stations (hard copies and soft copy on participant’s laptop) for use in this task
Part II
1. Short plenary to introduce guidelines on quality assurance measures to ensure validity of
OSCE scores interpretation and use (possible threats to validity and ways to address them);
2. Short presentation to share the quality assurance practices for OSCEs adopted at the School
of Medicine Fremantle;
3. Small group sharing of best practices for quality assurance measures in OSCEs at
participants’ university;
Who should attend
All health professional educators and educational assessment practitioners.
Level of workshop
Intermediate
Keywords
- Medical Education; Assessment of Clinical Competence; OSCE; Validity
Disciplines
Publication Date
March 19, 2016
Location
Perth Convention Centre, Perth Western Australia
Citation Information
Elina Tor, Carole Steketee, Donna Mak, Jane Courtney, et al.. "Designing OSCE Scoring Instrument with Built-In Validity Features Based on A Comprehensive Measurement Framework: Principles & Guidelines" Ottawa Conference 2016 (2016) Available at: http://works.bepress.com/elina-tor/6/