This article intends to address an online optimal adaptive regulation of nonlinear discrete-time systems in affine form and with partially uncertain dynamics using a multilayer neural network (MNN). The actor-critic framework estimates both the optimal control input and value function. Instantaneous control input error and temporal difference are used to tune the weights of the critic and actor networks, respectively. The selection of the basis functions and their derivatives are not required in the proposed approach. The state vector, critic, and actor NN weights are proven to be bounded using the Lyapunov method. Our approach can be extended to neural networks with an arbitrary number of hidden layers. We have demonstrated our approach via a simulation example.