In this paper, we present a distributed adaptive optimal control scheme for uncertain interconnected linear dynamical system using state and output feedback. The control policies at each of the distributed controllers are synthesized using a multi-player optimization problem via nonzero sum differential game theory. A novel adaptive observer, co-located with the distributed controller at each subsystem, is employed to reconstruct the augmented system internal states of the interconnected system. To accommodate the system uncertainties, an adaptive estimation scheme is proposed and the Nash equilibrium solution to the optimization problem is learned at each controller using the temporal difference error. The proposed distributed control scheme is employed in numerical simulations to regulate a network of ten interconnected linear systems and the results are presented to demonstrate the efficacy of the design.