The launch of the My School website and the public reporting of National Assessment Program - Literacy and Numeracy results have invited questions about how schools are 'performing'. There are two broad methods which can be used to measure a school's performance - direct and indirect measures. Direct measures of performance are based on observations and judgements of the quality of what is happening in a school. Indirect measures are based on measures of student performance. While both have their uses, this article argues that the attempt to draw inferences about a school's performance from student test scores is inherently problematic. Indirect measures need to be supplemented by more direct performance measures.