Malheatmap Save
An extension for tracking your activities on myanimelist.net
MAL Heatmap
|
MAL Heatmap is a tool for tracking your anime/manga consumption based on your myanimelist.net recent history. This app crawls the recent user history daily and generates a full visualization of all activities like Github contributions calendar. It also generates an image of your activities calendar that you can embed into your myanimelist.net profile or forum signature extending your MAL experience. |
| App | Sign in |
|---|---|
Profile examples
Here are some real user profiles to give you an idea of how it looks like in case you don't have an MAL account:
Motivation
At first, I made this experiment to play around with some cool Rails features, like ActiveStorage and ActionCable. Also, I want to take this opportunity to try web scrapping with Ruby and have a pretty much vanilla Rails experience, which is very different from the apps that I have been working on.
Building an app from zero to deployment has been a great experience, and I feel all its pain standing up from the ground. It's taught me a lot about sysadmin tasks, servers, docker, deployment, etc.
Over the years, I've made several upgrades, adding Turbo, SolidQueue, better caching with Russian Doll caching + HTTP Caching, etc.
How it works
The web app displays the processed data, and jobs do the hard and dirty work of crawling, processing, and saving data.
When users subscribe via OAuth, we capture some information based on the token and then start crawling their history. We periodically run the web crawler to keep the user's history current.
The users get a calendar image that can be embedded on their MAL profile or forum signature. The image is generated in the background by taking a screenshot of their profiles using a headless Chrome controlled by Ferrum.
Future
The plan is to consume data solely from MAL's API and drop the web crawling. That will require some deep reorganization of the code and how the activities are calculated since MAL does not have an API for history usage, and they don't seem to have that in their roadmap.
Main Tools
- Ruby on Rails
- SQLite
- SolidQueue
- SolidCache
- Hotwire
- Bulma CSS
Dependencies:
- SQLite
- Redis for action cable
- Ruby
- LibVips for ActiveStorage
Since this app is a classical Rails project, there is nothing special here to set up or run the project.
Deployment
This app is gloriously deployed with Kamal.
Since I keep this app running from my pocket, I use some space to run it on my tiny VPS. To keep costs low, we use the VPS storage to host the ActiveStorage files.
Other Inspirations
Open Source Agenda Badge
