Spark ignition (SI) engines running at very lean conditions demonstrate significant nonlinear behavior by exhibiting cycle-to-cycle dispersion of heat release even though such operation can significantly reduce NOx emissions and improve fuel efficiency by as much as 5-10%. A suite of neural network (NN) controller without and with reinforcement learning employing output feedback has shown ability to reduce the nonlinear cyclic dispersion observed under lean operating conditions. The neural network controllers consists of three NN: a) A NN observer to estimate the states of the engine such as total fuel and air; b) a second NN for generating virtual input; and c) a third NN for generating actual control input. For reinforcement learning, an additional NN is used as the critic. The uniform ultimate boundedness of all closed-loop signals is demonstrated by using Lyapunov analysis without using the separation principle. Experimental results on a research engine at an equivalence ratio of 0.77 show a drop in NOx emissions by around 98% from stoichiometric levels. A 30% drop in unburned hydrocarbons from uncontrolled case is observed at this equivalence ratio.