Aerial imagery captured through airborne sensors mounted on Unmanned Aerial Vehicles (UAVs), aircrafts, satellites, etc. in the form of RGB, LiDAR, multispectral or hyperspectral images provide a unique perspective for a variety of applications. These sensors capture high-resolution images that can be used for applications related to mapping, surveying, and monitoring of crops, infrastructure, and natural resources. Deep learning based algorithms are often the forerunners in facilitating practical solutions for such data-centric applications. Deep learning-based landmark detection is one such application which involves the use of deep learning algorithms to accurately identify and locate landmarks of interest in images captured through UAVs. This study proposes an efficient transfer learning method for feature extraction using a ResNet50 architecture, paired with a FasterRCNN object detection for an automated landmark detection framework. Additionally, a novel technique for hierarchical image annotation and synthetic sampling is also introduced to address the issue of class imbalance. Empirical results prove that our proposed approach outperforms other state-of-the-art landmark detection methodologies compared.