The antiviral drug Paxlovid has been shown to rapidly reduce viral load. Coupled with vaccination, timely administration of safe and effective antivirals could provide a path towards managing COVID-19 without restrictive non-pharmaceutical measures. Here, we estimate the population-level impacts of expanding treatment with Paxlovid in the US using a multi-scale mathematical model of SARS-CoV-2 transmission that incorporates the within-host viral load dynamics of the Omicron variant. We find that, under a low transmission scenario (Re∼1.2) treating 20% of symptomatic cases would be life and cost saving, leading to an estimated 0.26 (95% CrI: 0.03, 0.59) million hospitalizations averted, 30.61 (95% CrI: 1.69, 71.15) thousand deaths averted, and US$52.16 (95% CrI: 2.62, 122.63) billion reduction in health- and treatment-related costs. Rapid and broad use of the antiviral Paxlovid could substantially reduce COVID-19 morbidity and mortality, while averting socioeconomic hardship.