I currently have a script that scrapes data from a website by visiting a url and then scraping a certain piece of information. My only concern is that since there is roughly 30,000 pages to scrape that it might appear to the server as a DDoS attack? On top of this I was recommended to use Typhoeus gem for ruby to complete them all basically simultaneously. So I was just wondering if this should be alright?
A DDoS attack cant be performed by a single machine because DDoS means Distributed Denial of Service. So in the worst case it can be recognised as a DoS attack which isn't likely unless your using a huge amount of threads to perform page calls.
A single thread used by your bot isn't even a problem to a default configured apache because the connections per host are limited to 10 I think. By using 1 thread youre only having one connection established after another so you never even reach the limitation for one host. A normal browser may establishes more connections by default by downloading the assets of the webpage after it receiverd the html file. To sum it up: nope you dont seem to be an attacker by the view of the webserver admin.