The use of intrusion detection systems (IDS) has become crucial for modern networks. To ensure the targeted performance of such networks, diverse techniques were introduced to enhance system reliability. Many network designs have adapted the use of Unmanned Aerial Vehicles (UAVs) to provide wider coverage and meet performance targets. However, the cybersecurity aspect of UAVs has not been fully considered. In this paper, we propose a lightweight intrusion detection and prevention system (IDPS) module for UAVs. The IDPS module is trained using Deep Reinforcement Learning (DRL), specifically Deep Q-learning (DQN), to enable UAVs to autonomously detect suspicious activities and to take necessary action to ensure the security of the network. A customized reward function is used to take into consideration the dataset unbalanced nature, which encourages the IDPS module to detect minor classes. Also, considering the limited availability of resources for UAVs, a periodic offline-learning approach is introduced to ensure that UAVs are capable to learn and adapt to the evolution of intrusion attacks autonomously. Numerical simulations show the efficiency of the proposed IDPS in detecting suspicious activities and corroborating the advantages brought by the periodic offline learning in comparison with similar online learning approaches, in terms of accuracy and energy consumption.