We propose a neural network approach for global optimization with applications to nonlinear least square problems. The center idea is defined by the algorithm that is developed from neural network learning. By searching in the neighborhood of the target trajectory in the state space, the algorithm provides the best feasible solution to the optimization problem. The convergence analysis shows that the convergence of the algorithm to the desired solution is guaranteed. Our examples show that the method is effective and accurate. The simplicity of this new approach would provide a good alternative in addition to statistics methods for power regression models with large data.