Herek’s Attitudes Toward Lesbians and Gay Men-Short Version (ATLG-S) is widely used in research about anti-lesbian/gay bias (formerly called homophobia), yet the scale has not been validated with rigorous methods. This study addresses weaknesses identified in the original validation studies, specifically to verify the instrument’s factor structure in samples of students in social work and other helping professions. We performed an exploratory factor analysis to determine if the factor structure would be reproduced as Herek had hypothesized. We tested the resulting model and a variety of other possible models using confirmatory factor analysis with a different sample. In addition, we tested for evidence of internal consistency reliability, convergent construct, known groups, and predictive validity. The exploratory factor analysis supported the single factor structure but the confirmatory factor analysis did not, requiring excessive and inappropriate error term correlations to obtain acceptable model fit. Despite outdated items, the scale was reliable and otherwise empirically valid. We conclude that the ATLG-S has both conceptual and empirical problems that should be considered before use with a socially progressive sample. We suggest that the development of the ATLG and the ATLG-S is a cautionary tale illustrating the importance of a priori theoretical conceptualization of the latent construct to be measured. Further, we recommend that researchers, practitioners, and educators not assume a measure’s validity, and especially a short version of a validated longer measure, just because the measure appears frequently in the literature.