An important decision to be made in the design and implementation of large-scale assessments is the appropriate balance of item types — in particular the balance between objectively coded multiple-choice type items and item types that require coding by trained professionals. Decisions made in this regard depend upon finding an appropriate balance in the light of cost and construct validity concerns. Given the importance of this decision there are surprisingly few international studies that consider the effect of item format on achievement and the results of these studies are not consistent. Some of these studies do, however, illustrate that item format can influence results. This study draws upon PISA 2003 Field Trial data to examine the effect of item type. In particular we examine: Is there any evidence that students from particular countries perform differently depending on item format type? To what extent would the country rankings change if measures were based on particular item format types, and not on others? Is there a relationship between item format type and student ability level? Are different formats equally able to measure ability at different parts of the ability spectrum? What are psychometric characteristics like item discrimination of the different formats? Do students seem to respond differently depending on item format type? For example, are the patterns of missing data different for different item format types? Are any patterns observed consistent across countries? To what extent do girls and boys perform differently depending on item format type? Is any observed gender effect consistent across countries?