Reads your face and your words at the same time to tell you how you're really feeling.
People say one thing and feel another. MoodSync watches both signals at once — your facial expression through the webcam and the sentiment of what you type — and fuses them into a single read on your mood, live. Type "I'm fine" with a frown and it'll notice the gap.
Built in a weekend at a UMass hackathon.
- 📷 Real-time face emotion — your webcam feed streams into the browser with live emotion detection drawn right on top, frame by frame.
- ⌨️ Text sentiment — type how you're doing and a language model scores it positive vs. negative.
- 🔀 Smart fusion — instead of trusting just one signal, MoodSync blends them (weighted 70% text / 30% face) so your words and your expression both count toward the final call.
- ⚡ Smooth and fast — frame skipping and exponential smoothing keep the video responsive and the labels from flickering.
A Flask app serves the camera feed and the UI. Two models do the thinking:
- Face — FER with an MTCNN detector finds faces and reads emotions (happy, sad, angry, surprise, fear, disgust, neutral) on a live OpenCV stream.
- Text —
distilbert-base-uncased-finetuned-sst-2-english(Hugging Face Transformers) scores the sentiment of whatever you type.
When you submit text, MoodSync grabs your current expression, runs both models, and combines them: positive-leaning faces (happy/surprise) push the score up, negative ones (sad/angry/fear/disgust) push it down, while neutral expressions stay out of the way. The result is one verdict — POSITIVE or NEGATIVE — with a confidence percentage.
Flask · OpenCV · FER (MTCNN) · Hugging Face Transformers (DistilBERT) · PyTorch
cd Hack-Umass
pip install flask fer opencv-python transformers torch
python finalize.pyThen open http://localhost:5000 and allow camera access.
Needs a webcam. First run downloads the DistilBERT and FER model weights.
A hackathon experiment in reading between the lines — and the smiles.