Hong Kong has implemented stringent public health and social measures (PHSMs) to curb each of the four COVID-19 epidemic waves since January 2020. The third wave between July and September 2020 was brought under control within 2 m, while the fourth wave starting from the end of October 2020 has taken longer to bring under control and lasted at least 5 mo. Here, we report the pandemic fatigue as one of the potential reasons for the reduced impact of PHSMs on transmission in the fourth wave. We contacted either 500 or 1,000 local residents through weekly random-digit dialing of landlines and mobile telephones from May 2020 to February 2021. We analyze the epidemiological impact of pandemic fatigue by using the large and detailed cross-sectional telephone surveys to quantify risk perception and self-reported protective behaviors and mathematical models to incorporate population protective behaviors. Our retrospective prediction suggests that an increase of 100 daily new reported cases would lead to 6.60% (95% CI: 4.03, 9.17) more people worrying about being infected, increase 3.77% (95% CI: 2.46, 5.09) more people to avoid social gatherings, and reduce the weekly mean reproduction number by 0.32 (95% CI: 0.20, 0.44). Accordingly, the fourth wave would have been 14% (95% CI%: −53%, 81%) smaller if not for pandemic fatigue. This indicates the important role of mitigating pandemic fatigue in maintaining population protective behaviors for controlling COVID-19.