We study the characteristics of the leaky-integrator recurrent neural network dynamics and its applications. Our results show that the set of solutions of the dynamical system is positive invariant and attractive for the continuous-time recurrent neural network model. For the discrete-time recurrent neural network model, the stability analysis has been provided. Examples are given to demonstrate how our approaches can be applied to compress the data and perform the global optimization techniques to the nonlinear regression models effectively. The method offers an ideal setting to carry out the recurrent neural network approach to different areas including engineering, business and statistics.