I'm fairly new to Javascript programming, and am working on a web scraping script built using CasperJS.
The issue is that it's fairly slow, but it works. I'm trying to think of a way to make an overlaying script/program that starts this script I created, but I'm not sure the best way to do so. I have experimented with the GNU Parallel command, but I'd prefer something using JS, PHP or Python, as I'm more familiar with those languages.
I am also aware that the CasperJS instances will share the cookies and local storage, but that's not an issue on my use case. If anyone more experienced with this kind of architecture and framework could assist me, I'd appreciate it.
Thanks!
I ended up using NodeJS' child_process: https://nodejs.org/api/child_process.html It was pretty much what I wanted and used the same language I already used in the CasperJS script.
Tutorial and example I used: https://era86.github.io/2012/10/11/quick-and-dirty-nodejs-exec-limit-queue.html