Scrapy Cloudflare Middleware Save
A Scrapy middleware to bypass the CloudFlare's anti-bot protection
Project README
Scrapy "CloudFlare" middleware
A Scrapy middleware to bypass the CloudFlare's anti-bot protection, based on cloudflare-scrape.
Installation
pip install scrapy_cloudflare_middleware
Usage
Add the middleware to your DOWNLOADER_MIDDLEWARES settings
DOWNLOADER_MIDDLEWARES = {
# The priority of 560 is important, because we want this middleware to kick in just before the scrapy built-in `RetryMiddleware`.
'scrapy_cloudflare_middleware.middlewares.CloudFlareMiddleware': 560
}
Done. Happy scraping !
Open Source Agenda is not affiliated with "Scrapy Cloudflare Middleware" Project. README Source: clemfromspace/scrapy-cloudflare-middleware
Stars
103
Open Issues
4
Last Commit
2 years ago
Repository
Open Source Agenda Badge
