Scrapy - Get started !!

Scrapy is a python module for web scraping. If you want to scrape data in the websites in a automated way then this is one of the best option available in the techno-space. writing a python script with scrapy will create a crawler and it will scrape the data from the web sites that you intended to scrap.

Here i am listing few interesting links for setting and using the python scrapy !!

• This is one of the starting place for scrapy, which explains the setup and usage - Check here

• This is another wonderful article on scrapy !! Explains better - See here 

• Most probably you will get confused with xPath selection, If you want to look into it in detail then Check this and this

Use developer tools for xPath

Go to the inspect element of the required field in Chrome developer tool and right click+copy xpath

Right-click on the <tr> and copy the xpath and print it on the developer tool console to make sure that whether you got the correct xpath.

Here you can see that the xpath has selected the entire <tr>. now we can easily scrape the data with the scrapy's methods.