BACKGROUND

Higher education is increasingly a global business. At present, university ranking schemes are heavily reliant on research indicators, while students are likely looking for an excellent teaching and learning environment. Aware of this discrepancy, the OECD has funded the AHELO project.

PURPOSE

AHELO was designed to test the feasibility of an international assessment of higher education learning outcomes. The two test disciplines are civil engineering and economics, together with an assessment of generic skills. The civil engineering test will be reported in this paper.

More than a ranking, AHELO is a direct evaluation of student performance. It is intended to provide data on the relevance and quality of teaching and learning in higher education. The test aims to be global and valid across diverse cultures, languages and different types of institutions.

DESIGN/METHOD

The AHELO project was developed between 2008 and 2012. Preliminary work focussed on develop-ing the assessment framework, which builds upon the frameworks used for accreditation across the world (EA, Washington Accord, ABET, etc), with outcomes grouped into Basic/Engineering Sciences (both branch-specific and general), Engineering Processes (Analysis, Design and Practice), and Ge-neric Skills. It was decided that the test would focus on the engineering processes (using constructed response tasks) and basic and engineering sciences (using multiple choice questions to test knowledge of engineering fundamentals).

Sample questions were developed and reviewed by the International Reference Panel in Oct 2010. During 2011, a pilot of the chosen constructed response tasks plus MCQs was run around the world with approximately 10 students from each university from each participating country (more than 300 responses from Australia, Japan, Canada, Colombia and the Slovak Republic). These results and stu-dent feedback were used to modify some questions on the basis of Discrimination Factors.

RESULTS

In 2012, a full scale test has been conducted and is reported in this paper. Nine countries have partic-ipated with a total of more than 6,000 students. Results are currently being analysed to examine simi-larities and differences between the participating countries and as indicators of areas that need more attention in engineering curricula.

CONCLUSIONS

AHELO is the first international, standardised test that attempts to measure student outcomes from engineering programs across the world irrespective of language and cultural differences. This is im-portant in an increasingly global engineering job market. The paper reports the procedures taken to ensure a valid and reliable test.