I've recently set up my first Ubuntu server and installed scrapy and scrapyd. I've written a few spiders, and I've figured out how to execute the spiders through the API on port 6800. I also noticed there's a web interface there. I've also noticed that pretty much anyone could do the same, especially since the data at the web interface lists all the project and spider names. Is there a way to protect this feature so that only I can manage this?
Thanks, Chad
I figured it out. I couldn't figure out how to get .htaccess to do what I wanted, which was to password-protect a port where a service was listening to requests. I used iptables instead, assuming that the communication to that port was only going to be coming from a few different IPs.