A multilayer neural network (NN) controller in discrete-time is designed to deliver a desired tracking performance for a class of nonlinear systems with input deadzones. This multilayer NN controller has an adaptive critic NN architecture with two NNs for compensating the deadzone nonlinearity and a third NN for approximating the dynamics of the nonlinear system. A reinforcement learning scheme in discrete-time is proposed for the adaptive critic NN deadzone compensator, where the learning is performed based on a certain performance measure, which is supplied from a critic. The adaptive generating NN rejects the errors induced by the deadzone whereas a second NN based critic generates a signal, which is used to tune the weights of the action generating NN so that the deadzone compensation scheme becomes adaptive whereas a third multilayer NN simultaneously approximate the nonlinear dynamics of the system. Using the Lyapunov approach, the uniform ultimately boundedness (UUB) of the closed-loop tracking error and weight estimates of action generating NN, critic NN and the third NN are shown by using a novel weight update.