A novel reinforcement learning-based adaptive neural network (NN) controller, also referred as the adaptive-critic NN controller, is developed to deliver a desired tracking performance for a class of non-strict feedback nonlinear discrete-time systems in the presence of bounded and unknown disturbances. The adaptive critic NN controller architecture includes a critic NN and two action NNs. The critic NN approximates certain strategic utility function whereas the action neural networks are used to minimize both the strategic utility function and the unknown dynamics estimation errors. The NN weights are tuned online so as to minimize certain performance index. By using gradient descent-based novel weight updating rules, the uniformly ultimate boundedness (UUB) of the closed-loop tracking error and weight estimates is shown.