Scrapy Spider Example Save

This project is a Scrapy spider example collection, Michael Yin create this project to host the source code of Scrapy Tutorial Series: Web Scraping Using Python

You can find Scrapy spider example code which can help you:

1. A simple Scrapy spider shows you how to extract data from the web page.

2. How to handle pagination in Scrapy spider.

3. A simple script which can make your Scrapy shell more powerful.

4. How to define Scrapy item, and how to create a custom Item Pipeline to save the data of Item into Databases such as Mysql or PostgreSQL.

5. All the code can run without problem in Python2 and Python3

Warning

When you run the code in your local env, make sure to edit CONNECTION_STRING in scrapy_spider/settings.py. If you want more detail about this project, just check the Scrapy Tutorial Series: Web Scraping Using Python

Feedback

If you have any problem, feel free to fire issues in Github, I will reply ASAP.

Contact

You can contact me admin#michaelyin.info