Multimodal Sentiment Analysis based on Video and Audio Inputs

The main objective of this project is to prove the usability of emotion recognition models that take video and audio inputs. Fine-tuned models (e.g., Facebook wav2vec2 and Google vivit) have been used for averaging the decision-making framework. After disparity in the results, if one of the models gets much higher accuracy, another test framework is created. The methods used are the Weighted Average method, the Confidence Level Threshold method, the Dynamic Weighting Based on Confidence method, and the Rule-Based Logic method. This approach gives encouraging results that make future research into these methods viable.