In this paper, a novel nonlinear output feedback neural network (NN)-based consensus controller is developed for a group of quadrotor unmanned aerial vehicles (UAVs). One UAV in the group tracks a desired trajectory while the rest of the group uses consensus-based formation controllers without knowledge of the desired trajectory. Each UAV estimates its own and its neighbor's velocities through a novel nonlinear NN-based observer by using position and orientation information. Neighboring UAV positions and orientation information is assumed to be available via wireless communication or obtained through local sensors. Since quadrotor UAVs have six degree of freedom with only four control inputs, the UAV's pitch and roll angles are utilized as virtual control inputs to bring all UAVs to consensus points along x and y directions. The Lyapunov stability theorem is utilized to demonstrate that all the position errors, orientation errors, velocity tracking errors, observer estimation errors, and NN weight estimation errors are semi-globally uniformly ultimately bounded (SGUUB) in the presence of bounded disturbances. The effectiveness of our consensus-based output feedback formation control of quadrotor UAVs is demonstrated in simulation validating our theoretical claims.