This Scrapy project uses Redis and Kafka to create a distributed on demand scraping cluster.
Intermittent release to support changes outside of this project. No major code updates
The next official release of Scrapy Cluster
Release Candidate for Scrapy Cluster 1.2
The next official release of Scrapy Cluster
This is the initial release of Scrapy Cluster 1.0. Please see the documentation at http://scrapy-cluster.readthedocs.org/ for more information.