The automatic recognition of human emotions has become a subject of interest in recent years. The need to improve the interaction between human and machine has led researchers to focus on the subject of human emotion recognition as a solution to the minimal level of human-machine interaction nowadays. By being able to recognize emotions, machines such as robots will be able to better interact with humans by reacting according to their emotions, thus enriching the user experience.
In this thesis, two modalities of emotion expression will be analyzed, namely, facial expression and upper body movements. Both these modalities contribute greatly to the communication of a person’s emotions, much more than their words.
To recognize emotions from both modalities, Convolutional Neural Networks will be trained using benchmark datasets of subjects performing different emotions. Later, the results from each modality will be fused to formulate the final bimodal emotion recognition system.