With the increased complexity of today's industrial processes, maintaining equipment by preventing unscheduled downtime using monitoring hardware is a key challenge. Industrial statistics indicate that seal and impeller failures are predominant failure modes in centrifugal pumps and they are not adequately addressed in the literature. In this paper, a neural network (NN) based Nonlinear Autoregressive Moving Average with Exogenous input (NARMAX) model is used to develop fault detection scheme for detecting seal and impeller failures in centrifugal pumps. A rigorous methodology of detecting failures at the incipient stage is introduced. First a nonlinear relationship among the monitored parameters (inlet and outlet pressure, outlet flow, inlet and outlet temperature, and acceleration) where the previous values of the indicative parameters are used as inputs to the NARMAX model and the output being the value at the current instance is captured. The NARMAX modeled outputs are compared with the actual measured values in order to generate residuals. By choosing a suitable threshold, we could minimize false and missed alarms. Mathematical procedure for selection of threshold is derived in this paper. Along with the NARMAX model, an online approximator is used in the fault detection scheme for understanding the faults in the system. Experiments on the centrifugal pump seal and impeller failures were conducted by using a laboratory test bed. Experimental results show that the proposed fault detection scheme is able to successfully detect failures.