Researchers and teachers agree that evaluation of CALL should ideally inform pedagogical choices about how best to use CALL, but how to go about such an evaluation is not clear. This study offers an example of a context-based evaluation by operationalizing criteria for CALL evaluation and administering the instruments to three groups of stakeholders: the people who developed the content for the CALL materials, the teacher, and the students. The CALL materials were Longman English online (LEO). The setting was a community college English as a second language class in New York. Results, which focused on the agreement among stakeholders and their assessment of factors pertaining to six criteria, indicated good agreement among stakeholders and overall positive evaluations, but also identified some areas for improvement in the materials and the evaluation instruments.