This paper presents the design of an optimal neurocontroller that replaces the conventional automatic voltage regulator (AVR) and the turbine governor for a turbogenerator connected to the power grid. The neurocontroller design uses a novel technique based on the adaptive critic designs (ACDs), specifically on heuristic dynamic programming (HDP) and dual heuristic programming (DHP). Results show that both neurocontrollers are robust, but that DHP outperforms HDP or conventional controllers, especially when the system conditions and configuration change. This paper also shows how to design optimal neurocontrollers for nonlinear systems, such as turbogenerators, without having to do continually online training of the neural networks, thus avoiding risks of instability.