In this paper, a novel NN-based optimal adaptive consensus-based formation control scheme over finite horizon is presented for networked mobile robots or agents in the presence of uncertain robot/agent dynamics. The uncertain robot formation dynamics are approximated online by using an NN-based identifier and a suitable weight tuning law. In addition, a novel time-varying value function is derived by using the augmented error vector, which consists of the regulation and consensus-based formation errors of each robot. By using the value function approximation and the identified dynamics, the near optimal control input over finite horizon is derived. This finite horizon optimal control leads to a time-varying value function, which becomes the solution of the Hamilton-Jacobi-Bellman equation, and control input is approximated by a second NN with time-varying activation function. A novel weight update law for the NN value function is developed to tune the value function, satisfy the terminal constraint, and relax an initial admissible controller requirement. The Lyapunov stability method is utilized to demonstrate the consensus of the overall formation. Finally, simulation results are given to verify theoretical claims.