Meeting the challenges of short response items in the Queensland Core Skills Test : a report on the 1991 pilot study

Gabrielle Matters, Queensland. Board of Senior Secondary School Studies

Abstract

The Board of Senior Secondary School Studies (BSSSS) is developing the Queensland

Core Skills (QCS) Test, which will be administered to Queensland Year 12 students for

the first time in September 1992. One paper of the QCS Test will consist of Short

Response Items (SRIs).

The SRI format is new to Queensland. SRIs present significant challenges to test

constructors in their design, selection, presentation and marking. In 1991, following the

initial developmental stages, the Board conducted a pilot study involving a 30-item

paper, 4500 students, 42 schools and 55 markers.

This report documents that pilot study. It describes the Board's experiences in

implementing the Government's decision to include SRIs in the QCS Test, by relating

one cycle of the design process in action - creating and refining items, selecting and

sequencing items and presenting them as a testpaper, then administering and marking

that test.

It records what happened at each stage, what was learned along the way, what issues

emerged, and what changes in procedure are planned.

This report does not debate issues already discussed in the Vivian Report (1990), in the

community, and in earlier reports on tertiary entrance such as the Pitman Report (1987).

No answers are supplied to the questions: 'Why have a core skills test?'; 'Why have

short answer questions?' Nor are the relative merits of the two other modes of

assessment - extended writing and multiple-choice - documented here.

The results of the pilot study suggest ways in which the challenges presented by SRIs

can be met in the first year of administering the test, and that SRIs provide worthwhile

information about student achievement not accessible through the other modes.

[ISBN: 072425094]