In this paper, we present a novel representation for cyber-physical systems wherein the states of the cyber system are incorporated into the physical system and vice Next, by using this representation, optimal strategies are derived for the defender and the attacker by using zero-sum game formulation and iterative Q-learning is utilized to obtain the Nash equilibrium. In addition, a Q-learning-based optimal controller is revisited for the physical system with the presence of uncertain dynamics resulting from the cyber system under attacks. The benefit of the learning strategy is that the approach can handle a variety of attacks provided they affect packet losses and delays. Simulation results, on the yaw-channel control of the unmanned aerial vehicle (UAV), show that for the cyber system, both the defender and the attacker gain their largest payoff and for the physical system, the controller maintains the system stable.