Identifying deepfake videos on Twitter

Our Bot in action
Architecture diagram

Inspiration

Deepfakes have garnered widespread attention for their uses in celebrity pornographic videos, revenge porn, fake news, hoaxes, and financial fraud. This has elicited responses from both industry and government to detect and limit their use. In the recent elections in the states, deepfake videos were spread on the social media which confused the thought process of common people. People viewing the posts/tweets on the social media deserve to know whether the information is true or false.

What it does

Our application classifies the videos in the tweets as deepfake or not.

How we built it

Fetch the tweets using the Twitter API every 10-15 minutes.
Search and get the tweets that have video content.
Check if this tweet has already been analyzed by our bot.
If no, process the video using machine learning .
Return the result in the comments section of the Tweet using Twitter API.

Tech Stack : Python, Data store, Twitter API, ML, Efficientnetb7

Challenges we ran into

We did not have sufficient data to train the model.

Accomplishments that we're proud of

A full working solution that can be tried out by any Twitter user !
Upload a video on Twitter and tag us @rahul_grover99 to see it in action.

What we learned

How to use the Twitter API
Machine Learning
Team Work in remote environment !

What's next for Identifying deepfake videos on Twitter

Optimize Machine Learning Model.
Identify and notify twitter users if they are in any deepfake video using image recognition.
Direct Message Bot to know if any video is deepfake.