A media digestion tool that cross-analyses verbal and nonverbal cues for presentation, analysis, and summarization of broadcast news.
The main objective of this project was to understand the relationship between content and presentation for any given scene and understand the emotive aspects of the same. We wanted to study the extent to which such a presentation affects audiences and see if we can extract the content from its packaging. We also investigated how the presentation of the same content differs from channel to channel. This analysis aimed to measure if such portrayals affect their audiences and contribute to the formation of potentially dangerous echo chambers.
SuperGlue fuses multiple modalities to create a comprehensive model for the cross-analysis of body language, scene context, and other signals to explore the nature of news on different media outlets. I spearheaded the body language analysis for this project, where we cross-examined hand gestures, facial expressions, and body posture of the newscaster as three dimensions of influence on the overall manner of demonstration. The model is developed in OpenCV and PyTorch using fusion at the decision level for emotion analysis. Its principal components are a Recurrent Neural Network, 3D Convolutional Neural Network, and Azure Media Analytics’ model for posture recognition, hand gesture recognition, and facial emotion recognition respectively.
- Date: Aug 2019
- Details: Project Website