Purpose: Like many libraries, Georgia State University (GSU) Library consistently evaluates most of its services and resources, but the provision of high-quality reference service is assumed more than it is measured. As part of its fiscal year 2010-2011 Institutional Effectiveness Assessment Plan, GSU Library committed to investigating the quality of reference services provided by library employees in order to get a better picture of this core, and under-assessed, function of the library.
Design/Methodology/Approach: In 2009, the library evaluated customer service skills of library employees using the secret, or mystery, shopper approach often used in retail establishments in the private sector. In 2010, the library chose to evaluate reference services using the same method. Volunteers posed as students with reference questions, engaged employees in informally scripted interactions, and then assessed the employees’ reference skills using an evaluation form. The evaluation forms were based on the Reference and User Services Association’s Guidelines for Behavioral Performance of Reference and Information Service Providers (http://www.ala.org/ala/mgrps/divs/rusa/resources/guidelines/guidelinesbehavioral.cfm) and consisted of numeric rating scales. Volunteers rated whether and to what extent employees exhibited certain behaviors during their interactions with volunteers. Additionally, volunteers were asked to provide written comments to justify their numeric scores. The interactions occurred in person at the reference desk, circulation desk, and special collections reading room desk; online via email; and over the phone with reference desk, circulation desk, and special collections reading room desk employees.
Findings: Based on the evaluation results, one identified issue was that employees routinely took patrons’ questions at face value and responded accordingly, without attempting a reference interview. For example, if a patron asked about the location of magazines, the library employee did not attempt to find out whether the patron was looking for a specific title, issue, article, etc., and instead directed the patron to the location of the physical magazines. As another example, if a patron requested information about birds, the library employee did not make an effort to determine what kind of information was needed (a particular species, theme of birds in a work of literature, birds depicted in art, etc.) Another issue of potential concern is that library employees unnecessarily referred patrons. During the course of the project, library employees referred volunteers to other library departments, other campus departments, and, in one situation, to another institution. All of the scenarios used in this study were designed so that any library employee could respond to them using resources in this library. The findings of the study served to inform reference interviewing training; make reference interviewing training a regularly offered workshop; and promote discussions among supervisors and employees about reference interviewing expectations.
Practical Implications/Value: While the secret shopper method is not a novel one, it seems to be underused in libraries, perhaps because it seems challenging to execute. This paper will show that the challenges are minimal, and the results of the effort are quite worthwhile.
Available at: http://works.bepress.com/jenniferlinkjones/11/