Recent studies towards dragline excavation efficiency have focused on incrementally achieving automation of the entire excavation cycle. Initial efforts resulted in the development of an automated dragline swing system, which optimizes the swing phase time. However, the system still requires human operation for collision avoidance. For full dragline autonomy, a machine vision system is needed for collision prevention and big rock handling during the 'swinging' and 'digging' phases of the excavation operation. Previous attempts in this area focused on collision avoidance vision models which estimated the location of the bucket in space in real-time. However, these previous models use image segmentation methods that are neither scalable nor multi-purpose. In this study, a scalable and multi-purpose vision model has been developed for draglines using Convolutional Neural Networks. This vision system averages 82.6% classification accuracy and 91% detection in collision avoidance. It also achieves an 87.32% detection rate in bucket pose estimation tasks. In addition, it averages 80.9% precision and 91.3% recall performance across terrain recognition and oversized rock detection tasks. With minimal modification, the proposed vision system can be adjusted for other automated excavators.