I'd like to display my page differently for the visitor based on if they came from a search engine or not. I guess I'm detecting search engine referrals? I'd also like to know the search terms used to arrive at my site.
My solution so far is this:
1) Filter on the HTTP request referers which contain common search engine URLs, i.e. http://www.google.com/search, http://www.bing.com/search, or http://search.yahoo.com/search
2) Parse the query string parameter of the referer for the search terms, e.g. "q=search+terms+for+my+website+go+here"
I feel this isn't the most robust solution, because it misses lesser known search engines and each search engine may have it's own query string parameter for the search terms. For example, Google's and Bing's search term parameter are both "q" but Yahoo's is "p" as far as I can tell. And what about special operators like +, -, etc.?
Is there a more general way to do this? Or is there a library that would help me handle more search engines? I'm working with Java running on Tomcat, but ideas from any language and server setup are welcome.
Have a look here: http://www.gdargaud.net/Hack/Searches.html
and here: http://search.cpan.org/perldoc?URI::ParseSearchString
2 slightly different approaches but they cover a much wider range of search engines and their referer strings.