Search code examples
phpwordpresshttp-headersseolighthouse

Wordpress > change x-robots-tag

I have a Wordpress site that uses the Astra theme. In Chrome's Lighthouse tab it states:

Page is blocked from indexing 
Blocking Directive Source 
x-robots-tag: noindex, nofollow

However:

  • In Wordpress (Settings > Reading) the box is unchecked for "Search engine visibility: Discourage search engines from indexing this site"
  • The source code of the website does nowhere includes text such as "x-robots", "noindex" or "nofollow".

Does anyone understand why the page is blocked from indexing?

Update:

  1. I checked in the network tab, and under response headers it states x-robots-tag: noindex, nofollow.

  2. I looked at the source code of the page to check the meta tags, but there's nothing in there about x-robots, noindex or nofollow.

  3. The file robots.txt contains the following:

    User-agent: *

    Disallow: /wp-admin/

    Allow: /wp-admin/admin-ajax.php

    Sitemap: https://example.com/wp-sitemap.xml

How can I make the Wordpress site not being blocked from indexing?

Solution

  • I received a response from the hosting agency. The site at the moment is still on a subdomain of the hosting agency. They replied: "It comes from the nginx setup for default domains. Once you will use the domain name this would get resolved." So, once I switch to my own domain it should be resolved and I think the conclusion is that the nginx server configuration is the cause (set up this way on purpose).