© 2018 IEEE. Automatic understanding of videos is one of the complex problems in machine learning and computer vision. An important area in the field of video analysis is human action recognition (HAR). Though a large number of HAR systems have already been developed, there is plenty of daily life actions that are difficult to recognize, due to several reasons, such as recording on different devices, poor video quality and similarities among actions. Development in the field of deep learning, especially in convolutional neural networks (CNN), has provided us with methods that are well-suited for the tasks of image and video recognition. This work implements a CNN-based hierarchical recognition approach to recognize 20 most difficult-to-recognize actions from the Kinetics dataset. Experimental results have shown that the application of our method significantly improves the quality of recognition for these actions.