Smart and connected traffic intersections are a key component of the smart city idea. These intersections will be equipped with dynamic street and traffic lights, traffic analysis, anomaly detection, and other "smart"features, contributing towards solving various energy, safety, and congestion problems. The objective of this paper is to lay out a process for utilizing deep learning technologies to develop a smart traffic flow for cities. With the increase in accurate deep learning video analytic models, we propose developing multiple object detection and tracking solutions for traffic video analytics. One key challenge is that the detection results from each video frame are not associated with each other, thus causing the traffic tracking to be lack continuity. To solve this problem, we propose a Deep Associated Elastic Tracker (DAE-Tracker) that performs multi-stage object association and tracking for various deeplearning based detectors. We tested our solution with multiple deep learning models under different datasets. We also deployed our system on the roof of our engineering building and contributed our own dataset.