Skip to main content
A Novel Feature Set for Video Emotion Recognition
  • Shasha Mo
  • Jianwei Niu
  • Yiming Su
  • Sajal K. Das, Missouri University of Science and Technology

In video recommendation systems, emotions are used along with several other proposed content-based video features. However, such features are independently based on visual or audio signals and the relationship representing the dependencies between the visual and the audio signals is still unexplored. In order to solve this problem, a novel feature set called HHTC features based on the combination of Hilbert—Huang Transform (HHT) based visual features, HHT-based audio features, and cross-correlation features is proposed in this paper. In addition to the dependencies between the visual and the audio signals, the proposed HHTC features have the ability to indicate the time-varying characteristics of these signals. The proposed features are applied to video emotion recognition with the Support Vector Regression (SVR) with potential use in video affective recommendation systems. Experimental results demonstrate that the proposed approach can achieve an improved performance of video affective recognition.

Computer Science
Research Center/Lab(s)
Intelligent Systems Center
We would like to extend our heartfelt thanks to all the reviewers, without whose assistance the accomplishment of this thesis would have been impossible. This work was supported by the National Natural Science Foundation of China (61572060, 61772060) and CERNET Innovation Project (NGII20151004, NGII20160316).
Keywords and Phrases
  • Feature extraction,
  • Recommender systems,
  • Speech recognition,
  • Affective recognition,
  • Content analysis,
  • Cross correlations,
  • Emotion recognition,
  • Empirical Mode Decomposition,
  • Huang transform,
  • Support vector regression (SVR),
  • Time-varying characteristics,
  • Mathematical transformations,
  • Article,
  • Content analysis,
  • Correlation analysis,
  • Emotion,
  • Priority journal,
  • Problem solving,
  • Recognition,
  • Support vector machine,
  • Videorecording,
  • Cross-correlation,
  • Hilbert-Huang transform,
  • Video affective content analysis
Document Type
Article - Journal
Document Version
File Type
© 2018 Elsevier, All rights reserved.
Publication Date
Citation Information
Shasha Mo, Jianwei Niu, Yiming Su and Sajal K. Das. "A Novel Feature Set for Video Emotion Recognition" Neurocomputing Vol. 291 (2018) p. 11 - 20 ISSN: 0925-2312
Available at: