Background: This study examined the relationship between physical activity (PA) and academic performance and retention among college students using accelerometer data while controlling for sex and socioeconomic background. Methods: Data were collected from 4643 first-year college students at a private university in the south-central United States who began their studies in the Fall semesters between 2015 and 2022. Daily step counts were collected using accelerometers worn as part of the institutions PA requirements. The expected family contribution was calculated based on information provided on the Free Application for Federal Student Aid. Grade point average (GPA) and retention data were collected directly from official university databases. Results: Female students generally had lower median age and steps count and a higher median GPA than males. The retention rates from fall to spring and fall to fall were 95.9% and 83.8%, respectively, with no significant difference in retention rates between males and females. GPA was significantly positively correlated with steps, expected family contribution, and age, and negatively correlated with being male and having an expected family contribution of zero. The fall to spring retention rate was significantly positively correlated with steps and GPA. Conclusions: The study’s findings suggest that increasing levels of PA, as measured by daily step counts, may be effective in promoting academic performance and retention, even when controlling for sex and socioeconomic background. The use of device-based measures of PA in this study contributes to the literature on this topic, and policymakers and educational institutions should consider interventions focused on academic performance and physical activity to help students persist.