I have been working on operations with strings in a recent 100 level CompSci course. I got the very "original" idea that I might write up a simple domain name generator/checker.
So I did a little homework and discovered that the various whois servers understandably limit the number of queries allowed.
So, I decided to first check for a DNS boolean. If no records are found I then check a MySQL database to make sure the same query hasn't been sent recently. If it hasn't I fire off a whois query with PHP using fsockopen. So, I was just getting ready to finish up my little script and upload it from my development server to my production server and I found some sites suggesting that various whois servers limit the queries to only 1,000.
My question:
Am I approaching this appropriately? The simple math suggests that only 10 users each checking out 10 searches each search providing only 10 results (10**3) might result in exceeding the limit and a temporary ban.
Are there any methods of doing bulk queries to the whois server?
Are other sites using some form of client-side javascript query or server-side proxy? I found another similar question here at stackoverflow suggesting that *NIX systems have access to a terminal command that has no limits. Other questions I have found deal with parsing the data - which is not a concern of mine.
I understand that this is a vague question. I do not want to inappropriately burden the whois servers. I do not expect, nor want, a ready-made code solution. A basic discussion of alternative programmatic strategies to go about this would make me a very satisfied friend :) Anyone have a keyword or two with which I can continue my research?
The whois unix command appears to be less limited (https://superuser.com/questions/452751/what-are-the-limits-of-whois-command-on-unix). It might be easiest to do what I assume whois is doing under the covers and open a tcp connection to whois.internic.net on port 43.
<?php
$fp = fsockopen("whois.internic.net", 43);
fwrite($fp, "hello.com\n");
$response = "";
while (!feof($fp)) {
$response .= fread($fp, 8192);
}
fclose($fp);
echo $response;
?>
If that's what you're already doing, then that's probably your best bet. I'm guessing a 1,000 query limit likely refers to the use of somebody's web service that does this for you (e.g. whois.com). I think you can make a lot more queries than that if you're doing what I showed above.
(I've made a lot of guesses and assumptions here.)
P.S. A lot of good info here: http://semmyfun.blogspot.com/2010/08/how-does-whois-work-dirty-guide.html