Project Submission for Nebula|MDG
Feal_Beatz is a music app which recommends and plays musics based on your emotions.The app captures emotions from audio and video and suggests you a playlist catered to your emotions.
The uncopy-righted music is provided by NCS Music Library.
The app utilizes a CNN model to categorize facial emotion into seven categories namely : Angry, Disgust, Fear, Happy, Neutral, Sad, Surprise
The dataset used for the model: https://www.kaggle.com/jonathanoheix/face-expression-recognition-dataset
The app utilizes an LSTM model to categorize speech emotion into seven categories namely : Angry, Disgust, Fear, Happy, Neutral, Sad, Surprise
The dataset used for the model: https://www.kaggle.com/datasets/ejlok1/toronto-emotional-speech-set-tess
The App has been deployed on the streamlit community servers and can be accessed on the following url : https://corrosivelogic-feel-beatz-home-xgi8v2.streamlit.app/
Several Problems were encountered with the streamlit library as it's a relatively new library and some features are still having bugs . Major problem occured when the streamlit_webrtc component does not allow variable values to be trasferred out of the event loop thus a seperate picture capture had to be setup.
- To develop a UserAnalytics feature that can graphically show the users the change in moods overa a period of time and songs prefferd.
- To develop a classifier to self classify songs emotions.
- Add a recommendation feature to recommend songs based on past songs, moods and include Topic Modelling.
- https://arxiv.org/pdf/2109.04081.pdf
- https://arxiv.org/pdf/1904.06022v1.pdf
- https://arxiv.org/ftp/arxiv/papers/1906/1906.05681.pdf
- https://www.researchgate.net/publication/261250781_Emotion_Recognition_with_Image_Processing_and_Neural_Networks
- https://towardsdatascience.com/the-ultimate-guide-to-emotion-recognition-from-facial-expressions-using-python-64e58d4324ff