Dr. John Ainley Copyright (c) 2013 All rights reserved. http://works.bepress.com/john_ainley Recent documents in Dr. John Ainley en-us Tue, 19 Feb 2013 01:41:40 PST 3600 National Assessment Program : Information Communication and Technology Literacy : year 6 and year 10 : technical report 2005 http://works.bepress.com/john_ainley/159 http://works.bepress.com/john_ainley/159 Sun, 17 Feb 2013 18:02:23 PST This report describes the technical aspects of the National ITC Literacy Sample Assessment and summarises the main activities involved in the data collection, the data collection instruments and the analysis and reporting of the data. Chapter 2 summarises the development of the assessment domain and describes the process of item development and construction of the instruments. Chapter 3 reviews the sample design and describes the sampling process. Chapter 3 also describes the process of weighting to derive population estimates. Chapter 4 summarises the field administration and data management procedures, including quality control and the cleaning and coding of the data. Chapter 5 describes the scaling procedures, including equating, item calibration, the creation of plausible values and the standardisation of student scores. Chapter 6 examines the process of standards-setting and creation of Proficiency Levels used to describe student achievement. Chapter 7 discusses the reporting of student results, including the procedures used to estimate sampling and measurement error.

]]> John Ainley et al. National assessment program : ICT literacy technical report http://works.bepress.com/john_ainley/158 http://works.bepress.com/john_ainley/158 Mon, 03 Dec 2012 09:20:37 PST This report describes the technical aspects of the National Assessment Program – ICT Literacy (NAP – ICTL) 2011 sample assessment and summarises the main activities involved in the data collection, the data collection instruments and the analysis and reporting of the data. Chapter 2 summarises the development of the assessment domain and describes the process of item development and construction of the instruments. Chapter 3 reviews the sample design and describes the sampling process. It also describes the weighting procedures that were implemented to derive population estimates and calculation of participation rates. Chapter 4 summarises the field administration of the assessment. Chapter 5 deals with management procedures, including quality control and the cleaning and coding of the data. Chapter 6 describes the scaling model and procedures, item calibration, the creation of plausible values and the standardisation of student scores. It discusses the procedures used for vertical (Year 6 to Year 10) and horizontal (2011 to 2008 and 2005) equating with procedures for estimating equating errors. Chapter 7 outlines the proficiency levels and standards. Chapter 8 discusses the reporting of student results, including the procedures used to estimate sampling and measurement variance and the reporting of statistics for, and comparisons among, jurisdictions and designated groups of students over time.

]]> Eveline Gebhardt et al. Technological issues for computer-based assessment http://works.bepress.com/john_ainley/157 http://works.bepress.com/john_ainley/157 Mon, 26 Nov 2012 14:07:17 PST This chapter reviews the contribution of new information-communication technologies to the advancement of educational assessment. Improvements can be described in terms of precision in detecting the actual values of the observed variables, efficiency in collecting and processing information, and speed and frequency of feedback given to the participants and stakeholders. The chapter reviews previous research and development in two ways, describing the main tendencies in four continents (Asia, Australia, Europe and the US) as well as summarising research on how technology advances assessment in certain crucial dimensions (assessment of established constructs, extension of assessment domains, assessment of new constructs and in dynamic situations). As there is a great variety of applications of assessment in education, each one requiring different technological solutions, the chapter classifies assessment domains, purposes and contexts and identifies the technological needs and solutions for each. The chapter reviews the contribution of technology to the advancement of the entire educational evaluation process, from authoring and automatic generation and storage of items, through delivery methods (Internet-based, local server, removable media, mini-computer labs) to forms of task presentation made possible with technology for response capture, scoring and automated feedback and reporting. Finally, the chapter identifies areas for which further research and development is needed (migration strategies, security, availability, accessibility, comparability, framework and instrument compliance) and lists themes for research projects feasible for inclusion in the Assessment and Teaching of Twenty-first Century Skills project.

]]> Beno Csapó et al. ICCS 2009 Asian report : civic knowledge, attitudes, and engagement among lower secondary students in five Asian countries http://works.bepress.com/john_ainley/156 http://works.bepress.com/john_ainley/156 Wed, 24 Oct 2012 17:32:00 PDT This report describes results from the Asian regional module of the International Civic and Citizenship Education Study (ICCS). The five countries from the Asian region that participated in the international study—Chinese Taipei, Hong Kong SAR, Indonesia, the Republic of Korea, and Thailand—also participated in the Asian regional module. Each of these countries is located in either eastern or southeastern Asia. The report focuses on aspects of particular relevance for this geographic region, and should be viewed as part of the broader set of publications from ICCS. ICCS investigates the ways in which countries prepare their young people to undertake their roles as citizens. It studies student knowledge and understanding of civics and citizenship as well as student attitudes, perceptions, and activities related to civics and citizenship. It also examines differences among countries in relation to these outcomes of civic and citizenship education, and it explores how differences among countries relate to student characteristics, school and community contexts, and national characteristics.

]]> John Ainley et al. National Assessment Program : ICT Literacy Years 6 and 10 report 2011 http://works.bepress.com/john_ainley/155 http://works.bepress.com/john_ainley/155 Thu, 23 Aug 2012 19:16:55 PDT Under the National Assessment Program, national samples of Year 6 and Year 10 students were assessed to determine their levels of confidence, creativity and skill development in the use of information and communication technologies. This report compares the results of Australian school students by state and territory and student sub-groups, and provides details of their achievement against an ICT literacy scale. It also enables the most recent achievements of students to be compared against those from the previous national assessments of ICT literacy, conducted first in 2005 and again in 2008. In addition, a survey of student access to and use of computers was conducted alongside the ICT Literacy assessment. With assessment results showing that computer use by students had increased considerably over the six years from 2005 to 2011, this survey provides interesting and useful insights into how young people are using their access to new technologies, both in schools and at home.

]]> John Ainley et al. Overview of the IEA International Civic and Citizenship Education Study http://works.bepress.com/john_ainley/154 http://works.bepress.com/john_ainley/154 Thu, 23 Aug 2012 19:16:54 PDT The International Civic and Citizenship Education Study (ICCS) studied the ways in which young people are prepared to undertake their roles as citizens. ICCS was based on the premise that preparing students for citizenship roles involves developing relevant knowledge and understanding as well as helping them form positive attitudes toward being a citizen and participating in activities related to civic and citizenship education. It also examined differences among countries in relation to these outcomes of civic and citizenship education, and it explored how differences among countries relate to student characteristics, school and community contexts, and national characteristics. This technical report provides information about the data and analytic procedures that have been used in the study.

]]> John Ainley et al. Session J - Lessons for improvement from international comparative studies http://works.bepress.com/john_ainley/153 http://works.bepress.com/john_ainley/153 Thu, 23 Aug 2012 19:16:53 PDT Concurrent Session Block 2

]]> John Ainley Children’s numeracy skills http://works.bepress.com/john_ainley/152 http://works.bepress.com/john_ainley/152 Thu, 23 Aug 2012 19:16:51 PDT This chapter uses data from the K cohort of Growing Up in Australia: The Longitudinal Study of Australian Children (LSAC) to examine the numeracy skills demonstrated by children at different ages, and whether levels of numeracy skills vary for children from different socio-demographic backgrounds. The socio-demographic groups examined in this chapter are categorised by: child gender; socio-economic characteristics including family socio-economic position (SEP) (lowest 25%, middle 50%, highest 25%); mother’s working hours including mothers not working (those unemployed, on maternity leave, and not in the labour force), working less than 35 hours per week, or working 35 hours or more per week; and family characteristics, including family type (two-parent family, lone-mother family); and number of siblings in the household (no siblings, one or two siblings, three or more siblings).

]]> Galina Daraganova et al. The role of civic participation in national and school curricula http://works.bepress.com/john_ainley/151 http://works.bepress.com/john_ainley/151 Thu, 23 Aug 2012 19:16:50 PDT Civic and citizenship education is increasingly conceptualized in national curricula within a lifelong learning perspective that encompasses formal curricula (that specify content and approaches in a subject or set of subjects), non-formal curricula (through co-curricular activities in schools) and informal curricula (through a school and classroom ethos that embodies citizenship values). One review of education policies in Europe documents attempts to encourage ‘active citizenship’ supported by ‘democratic schools’ offering a ‘participatory school culture’. However, there is evidence of a gap between declarations of principle and the implementation of these principles in schools as well as between policies and practices. This paper uses data from the IEA International Civic and Citizenship Education Study (ICCS 2009) to document the variation among 38 countries in the way provision is made for civic and citizenship education. In examining patterns of civic and citizenship education it considers the extent to which curricula envisage wider forms of future participation as citizens than through formal political systems. It also examines the extent to which the experience of civic participation at school and in adult life is viewed as an important aspect of civic and citizenship education by education systems, principals and teachers. In addition the paper analyses the association between the inclusion of school participation as part of this learning area and the extent to civic and citizenship education is viewed as important by schools and teachers.

]]> John Ainley et al. Can improving working memory prevent academic difficulties? A school based randomised controlled trial. http://works.bepress.com/john_ainley/150 http://works.bepress.com/john_ainley/150 Thu, 28 Jun 2012 16:18:09 PDT Low academic achievement is common and is associated with adverse outcomes such as grade repetition, behavioural disorders and unemployment. The ability to accurately identify these children and intervene before they experience academic failure would be a major advance over the current 'wait to fail' model. Recent research suggests that a possible modifiable factor for low academic achievement is working memory, the ability to temporarily store and manipulate information in a 'mental workspace'. Children with working memory difficulties are at high risk of academic failure. It has recently been demonstrated that working memory can be improved with adaptive training tasks that encourage improvements in working memory capacity. The authors' trial will determine whether the intervention is efficacious as a selective prevention strategy for young children at risk of academic difficulties and is cost-effective. This randomised controlled trial aims to recruit 440 children with low working memory after a school-based screening of 2880 children in Grade one. The authors will approach caregivers of all children from 48 participating primary schools in metropolitan Melbourne for consent. Children with low working memory will be randomised to usual care or the intervention. The intervention will consist of 25 computerised working memory training sessions, which take approximately 35 minutes each to complete. Follow-up of children will be conducted at 6, 12 and 24 months post-randomisation through child face-to-face assessment, parent and teacher surveys and data from government authorities. The primary outcome is academic achievement at 12 and 24 months, and other outcomes include child behaviour, attention, health-related quality of life, working memory, and health and educational service utilisation. A successful start to formal learning in school sets the stage for future academic, psychological and economic well-being. If this preventive intervention can be shown to be efficacious, then we will have the potential to prevent academic underachievement in large numbers of at-risk children, to offer a ready-to-use intervention to the Australian school system and to build international research partnerships along the health-education interface, in order to carry our further studies of effectiveness and generalisability.

]]> John Ainley Student engagement with science in early adolescence : the contribution of enjoyment to students' continuing interest in learning about science http://works.bepress.com/john_ainley/149 http://works.bepress.com/john_ainley/149 Thu, 14 Jun 2012 00:21:03 PDT Recent research has expanded understanding of the contribution of emotions to student engagement and achievement. Achievement emotions can be conceptualized as general ways of responding to achievement settings or specific emotional states aroused during a specific learning activity. Emotion processes can be distinguished as positive or negative, activating or deactivating. Using data from an international survey of science achievement (PISA 2006; N > 400,000 15-year-old students from 57 countries), relations between the positive, activating achievement emotion of enjoyment and a number of variables that combine with enjoyment to define students’ engagement with learning science are examined. Previously, we reported that enjoyment is central to relations between interest in science, value and knowledge, and students’ reported current and future engagement. The embedded attitudinal items from PISA 2006 allow testing of how enjoyment contributes to a more direct measure of engagement with science by assessing students’ interest in finding out more about the specific topics used to measure their science achievement. In this investigation, structural equation modeling is used to test predictions based on Hidi and Renninger’s (2006) four-phase model of interest development, and Pekrun’s (2006) control-value theory of achievement emotions.

]]> John Ainley et al. Enhancing education http://works.bepress.com/john_ainley/147 http://works.bepress.com/john_ainley/147 Sun, 06 May 2012 16:49:04 PDT The ‘Little children are sacred’ report concluded that improving the quality of education provided the key to solving, or at least ameliorating, the incidence of child sexual abuse in Aboriginal communities. The report identified poor educational outcomes as being at the heart of the exclusion of Aboriginal people from confidently participating in either their own culture or mainstream culture.845 A number of specific issues were highlighted as being at the roots of a crisis in educational attainment in Aboriginal communities. They included inadequate access for Aboriginal children to early learning centres, in preparation for later years of schooling, a failure amongst the predominantly non-Aboriginal teaching workforce to communicate effectively with Aboriginal students (many of whom speak English only as a second or third language), and a failure of parents to take responsibility for sending their children to school. To ensure that Indigenous children in the Northern Territory received appropriate support to improve levels of educational achievement, the Australian Government introduced a range of education-related measures through the Northern Territory Emergency Response (NTER). Five sub-measures address these concerns: four under the Enhancing Education measure and one under the Supporting Families measure. Two years after the release of the ‘Little children are sacred’ report and after the introduction of the NTER, data were collected on the health and development of all Australian children. Data from the Australian Early Development Index (AEDI) provide recent support for concerns that many children entering school in the Northern Territory are at a distinct disadvantage when compared to other children across Australia. This chapter examines data on the education-related measures.

]]> Sheldon Rothman et al. A cultural perspective on the structure of student interest in science http://works.bepress.com/john_ainley/146 http://works.bepress.com/john_ainley/146 Sun, 06 May 2012 15:57:37 PDT In this article, the authors examine the nature of interest in science as represented in the Programme for International Student Assessment (PISA) 2006 data. They discuss the interconnections between measures of knowledge, affect, and value as components of interest in science. Working from a perspective acknowledging that many of the models of motivation represented in the literature have been developed in Western countries, the authors investigated whether the ways that knowledge, affect, and value combine in the structure of students' interest in science might vary in line with historical and cultural traditions. Four countries were chosen to represent contrasting cultural values as defined in analyses of the World Values Surveys and the European Values Surveys - Colombia, Estonia, USA, and Sweden. Models are described showing variations in fit across the four countries. Efforts to increase the attractiveness of science to students should take heed of the fact that all models indicated a central role for enjoyment of science in the paths linking personal value, interest, and current science activities with intentions for future participation in science. Differences in the strength of the associations between science knowledge and interest in science support the proposition that the interconnections between knowledge, affect, and value need to be understood in relation to students' broader historical and cultural context.

]]> John Ainley et al. Analysis of Year 12 or Certificate II attainment of Indigenous young people – Stage 1 : a report prepared for the Council of Australian Governments Reform Council http://works.bepress.com/john_ainley/145 http://works.bepress.com/john_ainley/145 Mon, 23 Apr 2012 21:32:56 PDT The purpose of this paper is to assist the Council of Australian Governments (COAG) Reform Council to analyse performance information relating to the Year 12 or Certificate II attainment of Indigenous young people and to understand better variations in performance across jurisdictions. As part of these analyses variations over time within each jurisdiction have been reported. The analyses use existing data sources. No new data have been collected. There are four strands to this work:
  1. a statistical overview of educational attainments and participation;
  2. multivariate analyses of influences on attainment;
  3. a review of literature intended to inform an understanding of trends and patterns in attainment; and
  4. the development of a model of influences on attainment based on the quantitative analyses and informed by perspectives from the literature.

As the first stage of a two-stage project, this paper focuses on quantitative analyses of data but with an orientation to providing a basis for the second stage that will focus upon strategies and interventions associated with improved outcomes.

]]> John Ainley et al. ICCS 2009 Technical Report http://works.bepress.com/john_ainley/144 http://works.bepress.com/john_ainley/144 Mon, 09 Jan 2012 04:21:58 PST The International Civic and Citizenship Education Study (ICCS) studied the ways in which young people are prepared to undertake their roles as citizens. ICCS was based on the premise that preparing students for citizenship roles involves developing relevant knowledge and understanding as well as helping them form positive attitudes toward being a citizen and participating in activities related to civic and citizenship education. It also examined differences among countries in relation to these outcomes of civic and citizenship education, and it explored how differences among countries relate to student characteristics, school and community contexts, and national characteristics. This report is structured so as to provide technical detail about each aspect of ICCS. The chapters cover: test development, questionnaire development, development of regional instruments, translation and national adaptations of ICCS 2009 instruments, sampling design and implementation, sampling weights and participation rates, ICCS survey operations procedures, quality assurance in the ICCS data collection, data management and creation of the ICCS international database, scaling procedures for ICCS test items, scaling procedures for ICCS questionnaire items, and the reporting of ICCS results.

]]> Wolfram Schulz et al. The Contribution of IEA Research Studies to Australian Education http://works.bepress.com/john_ainley/143 http://works.bepress.com/john_ainley/143 Mon, 09 Jan 2012 04:21:56 PST This chapter is concerned with the contribution of the research studies conducted by the International Association for the Evaluation of Educational Achievement (IEA) to Australian education. During a period of 50 years education across the world has undergone a remarkable transformation. Many of the changes that have occurred have been initiated by the United Nations Organization and its agen¬cies, particularly UNESCO. From its origins within the UNESCO Institute in Hamburg, IEA and its programs have evolved to develop the worldwide conduct of research in education both through the undertaking and reporting of studies as well as the informal training of research workers to participate in these studies. Consequently, the large group of people who have been involved in the IEA studies together with their colleagues have built a new world vision of education in schools. This vision that extends beyond the boundaries of western Europe and North America has been presented in the two editions of the International Encyclopaedia of Education with its numerous Handbooks as well as an electronic version named the Complete Encyclopaedia.

]]> John Ainley et al. Navigating the transition from school to work: Implications for the emotional well-being of young people http://works.bepress.com/john_ainley/142 http://works.bepress.com/john_ainley/142 Mon, 09 Jan 2012 04:21:54 PST Julie McMillan et al. Health and well being Educational attainment and transitions from school International Civic and Citizenship Education Study : Assessment Framework http://works.bepress.com/john_ainley/141 http://works.bepress.com/john_ainley/141 Mon, 09 Jan 2012 04:21:52 PST The aim of ICCS is to report on student achievement on a test of conceptual knowledge and understandings in civic and citizenship education. It also intends to collect and analyze data about student dispositions and attitudes relating to civic and citizenship education. This publication contains the ICCS assessment framework, which provides the blueprint for the assessment of the outcomes of civic and citizenship education. An important feature of ICCS is the establishment of regional modules. Regional modules compromise groups of countries from the same geographic region that together administer additional instruments to assess region-specific aspects of civic and citizenship education. Three regional modules have been implemented as part of ICCS for participating countries in the regions of Europe, Latin America, and Asia respectively. The following instruments are administered as part of the ICCS survey: an international cognitive student test; a student questionnaire; a regional student instrument; a teacher questionnaire; a school questionnaire; and an online national contexts survey. The assessment framework provides a conceptual underpinning for the international instrumentation for ICCS and has also been a point of reference for the development of regional instruments.

]]> Wolfram Schulz et al. John Keeves and the Australian Council for Educational Research http://works.bepress.com/john_ainley/140 http://works.bepress.com/john_ainley/140 Mon, 09 Jan 2012 04:21:50 PST This book has been produced as a tribute to the work and contributions of ACER's 3rd chief executive. Professor John Keeves on the occasion of his 85th birthday. Professor Keeves has made substantial contributions to key research studies, especially in mathematics and science education. His work laid the foundations for large scale assessments of student learning that are common place today and for better understanding the complex relationships between students home background, school factors and achievement.

]]> Rebecca Leech et al. The International Civic and Citizenship Education Study (ICCS) http://works.bepress.com/john_ainley/139 http://works.bepress.com/john_ainley/139 Sun, 08 Jan 2012 16:23:02 PST The IEA International Civic and Citizenship Education Study (ICCS) investigated the ways in which young people are prepared for, and consequently ready and able to undertake their roles as citizens. Consequently, it studied student knowledge and understanding of civics and citizenship as well as affective and behavioral aspects of civics and citizenship such as value beliefs, attitudes, intended behaviors and current activities related to civic and citizenship education. Contextual data from education systems and schools were analyzed to help explain variation in these outcome variables. ICCS built on the previous IEA studies of civic education (Arnadeo et. al., 2002; Schulz & Sibberns, 2004; Torney-Purta et. al., 2001). In 1971 the IEA Civic Education Study included a 47 item test for 14 year olds in nine countries (Torney, Oppenheim & Farnen, 1975). In 1999 the IEA CIVED study included a 38 item test for 14 year old students in 28 countries (Torney-Purta et. al., 2001) and a 42 item test for 17-18 year olds in 16 countries (Amadeo et. al., 2002). Another chapter in this volume provides a history of these studies from the late 1960s to 2005 (Torney-Purta & Schwille, 2011). In addition to building on these studies ICCS is a response to the challenge of educating young people in changed contexts of democracy and civic participation.

]]> Wolfram Schulz et al. Longitudinal Study of Australian Children (LSAC)