Oral performance tests designed to assess candidates' ability to function in a particular professional role typically involve role plays which simulate aspects of professional interaction between a candidate, in his/her professional role, and an interlocutor, in the role of client or consumer of the professional services. This paper reports on an investigation into the role which expert professional informants can play in providing post hoc content validation of tasks used in this kind of occupation-specific test, focusing on both the content of the tasks and the behaviour of the interlocutors. The study draws on data from the oral interaction component of an occupation-specific test, the Occupational English Test, a 4-skills test for overseas-trained health professionals (with attention in this study to nurses). The study involved a two-day workshop where representatives of the nursing profession, in conjunction with the researchers, analysed both written test materials and audiotaped test data, contributing to an analysis of the realism of the tasks and the features of the interlocutor behaviour.