A telegram bot to automatically search for apartments for rent
Clone this repository with --recursive
flag
docker-compose
is required
Modify filter.json
in the root of the repository according to your needs. The format is self-explanatory, there are 3 options:
You can modify the filter at any time while the bot is running, no bot restart is needed for chagnes to be taken into account
Run from the root directory:
export HS_TELEGRAM_BOT_TOKEN="XXX"
export HS_TELEGRAM_CHAT_ID="XXX"
docker-compose up -d
Where the environment variables are:
HS_TELEGRAM_BOT_TOKEN
- telegram bot token
HS_TELEGRAM_CHAT_ID
- telegram chat id where the notifications will be posted
export HS_SCRAPE_INTERVAL_MINS=10
- environment variable that controls the number of minutes between each scrape loop. Default is 10 mins.
Apart from the bot itself there will be pre-configured instances of Prometheus and Grafana started for monitoring purposes.
Grafana is accessible at
admin / admin
It shows the number of notifications sent per source, time it took to collect the information and some other metrics.
To run crawling without monitoring run:
docker-compose up -d crawler
Data is stored in the ~/.hobo-sapiens
directory.
In order to detect duplicate annoucements posted on different websites the service uses Image hashing techniques.
Contribution is welcome. It's best to add more proxy retrievers or/and agencies websites scrappers.
All agency scrappers have the same tructure inheriting
services.abstract_service.AbstractService
In order to add a new one it's best just to copy (and rename) a
services.starter_service.StarterService
and then write an implementations to methods inside of it
To plug it in add it to the array service_classes = [...]
in runner.py