A novel reinforcement-learning based output-adaptive neural network (NN) controller, also referred as the adaptive-critic NN controller, is developed to track a desired trajectory for a class of complex feedback nonlinear discrete-time systems in the presence of bounded and unknown disturbances. This nonlinear discrete-time system consists of a second order system in nonstrict form and an affine nonlinear discrete-time system tightly coupled together. Two adaptive critic NN controllers are designed - primary one for the nonstrict system and the secondary one for the affine system. A Lyapunov function shows the uniformly ultimate boundedness (UUB) of the closed-loop tracking error, weight estimates and observer estimates. Separation principle and certainty equivalence principles are relaxed, persistency of excitation condition is not required and linear in the unknown parameter assumption is not needed. The performance of this controller is evaluated on a spark ignition (SI) engine operating with high exhaust gas recirculation (EGR) levels where the objective is to reduce cyclic dispersion in heat release.