Search code examples
pythonscrapyscrapyd

How to run scrapy server as a daemon


I am trying to run scrapy server as a daemon. I have tried this link but i getting error whenever i run this command:

sudo apt-get install scrapyd-0.14
Reading package lists... Done
Building dependency tree       
Reading state information... Done
E: Unable to locate package scrapyd-0.14
E: Couldn't find any package by regex 'scrapyd-0.14'

I am able to run scrapy server from my scrapy project folder but i want to run it as background service (daemon)..i am using scrapy version : Scrapy 0.16.5

how do i do this...please help me..


Solution

  • You need to add the Ubuntu repository first. Run the following commands under root permission:

    $ curl -s http://archive.scrapy.org/ubuntu/archive.key | sudo apt-key add -
    $ echo "deb http://archive.scrapy.org/ubuntu precise main" >> /etc/apt/sources.list
    $ apt-get update
    $ apt-get install scrapyd-0.14
    

    Also note that the latest version of Scrapyd available up to date is 0.17