DownloadThisVideo

On April 4, 2023, Twitter suspended @this_vid, after nearly 5 years and millions of users. 😢

Easily download videos/GIFs off Twitter. Mention the bot (@this_vid) in a reply to the tweet containing the video, and it'll reply with a download link in a few minutes.

How this works

Stack

• AWS Lambda with the Serverless Framework
• AWS SNS
• Redis
• Node.js

Implementation

The bot consists of several AWS Lambda functions that work in tandem:

fetchTweetsToDownload

This function runs every 4 minutes and checks for new mentions. It publishes these new mentions as a new notification on an SNS topic. The 4-minute interval is so as to not hit Twitter's rate limits and minimize AWS Lambda usage time, while being near-realtime.

sendDownloadLink

This is triggered by new notifications on the SNS topic. It:

• processes the tweets in the message body,
• calls Twitter's API to retrieve media links. Any video links retrieved for a tweet are stored in Redis for faster repeated access (other users requesting the same video).
• adds the download details to the user's store in Redis. The user's store is an entry in Redis where all downloads requested by a user are cached for a certain period (48 hours).
• attempts to reply to the user with a link to the user's download page (see section below). "Attempts" because Twitter enforces tweet limits (2400 per day, counted in 15-minute periods). If the API limits have been reached, the bot will "cool down" (not send any replies) for 10 minutes.

getDownloads

This is triggered by a HTTP request to the <AWS API Gateway URL>/<Twitter-handle> (for instance, http://thisvid.space/jack). It renders a page showing a list of the user's recent downloads.

getHomePage

Renders the homepage 😁. See http://thisvid.space.

retryFailedTasks

This re-publishes failed tasks (stored in Redis) as a new SNS message. For now, it can only be triggered manually.