Research has shown substantial reductions in the oxides of nitrogen (NOx) concentrations by using 10% to 25% exhaust gas recirculation (EGR) in spark ignition (SI) engines [1]. However under high EGR levels the engine exhibits strong cyclic dispersion in heat release which may lead to instability and unsatisfactory performance. A suite of neural network (NN)-based output feedback controllers with and without reinforcement learning is developed to control the SI engine at high levels of EGR even when the engine dynamics are unknown by using fuel as the control input. A separate control loop was designed for controlling EGR levels. The neural network controllers consists of three NN: a) A NN observer to estimate the states of the engine such as total fuel and air; b) a second NN for generating virtual input; and c) a third NN for generating actual control input. For reinforcement learning, an additional NN is used as the critic. The stability analysis of the closed loop system is given and the boundedness of all signals is ensured without separation principle. Online training is used for the adaptive NN and no offline training phase is needed. Experimental results obtained by testing the controller on a research engine indicate an 80% drop of NOx from stoichiometric levels using 10% EGR. Moreover, unburned hydrocarbons drop by 25% due to NN control as compared to the uncontrolled scenario.