Portia, how to save data to database?

Question

In portia, I want to save the data to the database like Mysql or do something to clean the data, but I don't know how to do that, can you give me some suggestion. I'm new in scrapy, and I'll wait online, thank you very much!

Answer 1

You need to add a new item processing pipeline for storing data to MySQL. To do this you need to go to the Portia project folder, add a new pipelines.py file that can save data to MySQL and edit the settings.py file to use this pipeline.

Here is an example of an item pipeline for storing data in MySQL

https://github.com/darkrho/dirbot-mysql/blob/master/dirbot/pipelines.py#L36

Here is the documentation on how to enable the pipeline and how it works

http://doc.scrapy.org/en/latest/topics/item-pipeline.html