Objective To address socio-demographic factors associated with adherence to follow-up recommendations in a high-risk population of women referred for follow-up care after an abnormal Pap test. Methods 486 women aged 46–64 served by BCCEDP in two southeastern states between 1999–2002 and referred for follow-up care after an abnormal Pap test were the sampling frame for this cross-sectional study; 204 women completed a phone-based interview in 2004. Cox proportional hazards modeling was used to determine the association of various risk factors with time to adherence. Results Among those completing the phone interview (interview rate = 61.4%) the mean age was 53.3 years, 64.7% were African–American women, 81.9% had low-grade cervical lesions, and all were either uninsured or under insured. Over 95% received follow-up care for an abnormal Pap test within 365 days of referral. When the BCCEDP criteria of follow-up within 60 days were applied, 52.9% were adherent. Rates of self-reported and program documented adherence differed significantly by state. After adjusting for state of residence, women who reported having symptoms of a chronic disease were more likely to be adherent within 365 days (aHR = 1.42; 95% CI = 1.00, 2.04). Neither age, race, lesion severity, education, number of dependent adults or children, self-perceived physical health, nor smoking status was associated with time to adherence. Conclusions Findings suggest that institutional factors may be more important than individual factors in predicting time to adherence for an abnormal Pap test.