A method for hybridizing supervised learning with adaptive dynamic programming was developed to increase the speed, quality, and robustness of on-line neural network learning from an imperfect teacher. Reinforcement learning is used to modify and enhance the original supervisory signal before learning occurs. This paper describes the method of hybridization and presents a model problem in which a human supervisor teaches a simulated car to drive around a race track. Simulation results show successful learning and improvements in convergence time, error rate, and stability over either component method alone.