Search code examples
wikipediabingbing-apibing-custom-search

Why does Bing (and Bing Custom Search) return no results for certain Wikipedia pages?

I am trying to use Bing Custom Search over certain Wikipedia pages. However, I am finding that there are many pages that don't return any results in Bing Custom Search (or on Bing Search).

For example, you can search Bing for site:https://en.wikipedia.org/wiki/George_Washington name and get a response back for that one web page. However, if you choose from any of the following dog breed names (really almost any dog breed from this list: https://en.wikipedia.org/wiki/List_of_dog_breeds): 

https://wikipedia.org/wiki/Wire_Fox_Terrier
https://wikipedia.org/wiki/Kishu
https://wikipedia.org/wiki/English_Cocker_Spaniel

as the site, searching site:{site_from_list_above} dog (for example site:https://wikipedia.org/wiki/Wire_Fox_Terrier dog) there are no results. Am I searching wrong, or does Bing just not cache these pages?

Solution

  • To answer my own question: after talking with Bing Support and playing with the URLs myself, I found that there were two issues:

    1. The urls need to not be redirects, as https://wikipedia.org/wiki/Kishu takes you to https://en.wikipedia.org/wiki/Kishu.
    2. The urls that have special characters in them (such as https://en.wikipedia.org/wiki/Chihuahua_(dog) need to be replaced (this means “(“ with “%28” and “)” with “%29”, etc.). More details on escaping can be found here: https://www.w3schools.com/tags/ref_urlencode.asp

    Hope this helps someone else!