Recent improvements in machine vision algorithms have led to closed-circuit television (CCTV) cameras emerging as an important data source for determining of the state of traffic congestion. In this study we used two different deep learning techniques, you only look once (YOLO) and deep convolution neural network (DCNN), to detect traffic congestion from camera images. The support vector machine (SVM), a shallow algorithm, was also used as a comparison to determine the improvements obtained using deep learning algorithms. Occupancy data from nearby radar sensors were used to label congested images in the dataset and for training the models. YOLO and DCCN achieved 91.5% and 90.2% accuracy, respectively, whereas SVM’s accuracy was 85.2%. Receiver operating characteristic curves were used to determine the sensitivity of the models with regard to different camera configurations, light conditions, and so forth. Although poor camera conditions at night affected the accuracy of the models, the areas under the curve from the deep models were found to be greater than 0.9 for all conditions. This shows that the models can perform well in challenging conditions as well.