Presentation
Evaluate item dependency in an aptitude reasoning test
81st Annual Meeting of the Psychometric Society
(2016)
Abstract
The Special Tertiary Admissions Test (STAT), developed by the Australian Council for Educational Research (ACER), is as an alternate method of gaining entry into Australian university courses, for people who do not hold a recent Year 12 certificate in Australia. The test compromises 64 scored multiple-choice items, half of which are Verbal Reasoning questions and the other half, Quantitative Reasoning questions. Items were designed in testlets, where they could share the same stimulus (example, text paragraph). A natural question here was if the local independency assumption was violated in any sets of the items of the testlets. To investigate the level of item dependency within each testlet, two methods were implemented and compared for 2014/15 STAT data, comprising more than 8,000 candidates, 55% of which were female and 45% of which were male. The first popular method was computing residual correlations using RUMM software (Andrich, Sheridan, & Luo, 2012). Residual correlations are often used to determine if any pair of items shows a sign of dependency. The other method was using item bundle fit statistics from ConQuest software (Adams, Wu, & Wilson, 2011) for each testlet. Additional simulation data sets with similar structure to STAT were also generated to compare the two dependency detecting methods. Initial results show that items in both domains fitted well to the Rasch model. The level of item dependency in each of the testlets was low. Additionally, the study suggests that item bundle fit statistics by ConQuest could overcome a limitation of the residual correlations for detecting independency among a set of three or more items rather than two.
Keywords
- University entrance examinations,
- Aptitude tests,
- Cognitive skills,
- Test items,
- Item analysis,
- Multiple choice tests
Disciplines
Publication Date
July, 2016
Location
Asheville, NC, USA
Citation Information
Luc T Le. "Evaluate item dependency in an aptitude reasoning test" 81st Annual Meeting of the Psychometric Society (2016) Available at: http://works.bepress.com/luc_le/32/