A lab-scale anaerobic-anoxic-oxic system was used to investigate the nitrogen removal mechanism under low dissolved oxygen (DO) conditions. When DO was decreased from 2 to 0.5 mg L−1, chemical oxygen demand (COD) and NH4+ removals were not influenced, while total nitrogen removal increased from 69% to 79%. Further batch tests indicated that both the specific nitrification rate and denitrification rate greatly increased under low DO conditions. When DO was decreased from 2 to 0.5 mg L−1, the oxygen half saturation constant value for ammonia oxidizing bacteria (AOB) decreased from 0.39 to 0.29 mg-O2 L−1, and for nitrite oxidizing bacteria (NOB), it reduced from 0.29 to 0.09 mg-O2 L−1. Correspondingly, the observed yield coefficients increased from 0.05 to 0.10 mg-cell mg−1-N for AOB, and from 0.02 to 0.06 mg-cell mg−1-N for NOB. High-throughput sequencing revealed that the relative abundances of AOB increased from 6.13% to 6.54%, Nitrospira-like NOB increased from 3.67% to 6.50%, and denitrifiers increased from 2.84% to 7.04%. Improved simultaneous nitrification and denitrification under low DO conditions contributed to the enhanced nitrogen removal.