In this paper, first a novel decentralized state feedback stabilization controller is introduced for a class of nonlinear interconnected discrete-time systems in affine form with unknown subsystem dynamics, control gain matrix, and interconnection dynamics by employing neural networks (NNs). Subsequently, the optimal control problem of decentralized nonlinear discrete-time system is considered with unknown internal subsystem and interconnection dynamics while assuming that the control gain matrix is known. For the near optimal controller development, the direct neural dynamic programming technique is utilized to solve the Hamilton-Jacobi-Bellman (HJB) equation forward-in-time. The decentralized optimal controller design for each subsystem utilizes the critic-actor structure by using NNs. All NN parameters are tuned online. By using Lyapunov techniques it is shown that all subsystems signals are uniformly ultimately bounded (UUB) for stabilization of such systems.