Adding X-Robot-Tag to response header in Sitecore 8

Question

I have got a website setup in Sitecore. My sitemap is an item in sitecore which is under the homepage.

I am able to access my sitemap by typing following URL: http://example.com/xmlsitemap Whereas xmlsitemap is the name of the item in Sitecore. Which has the rendering to get the XML sitemap given below:

XmlDocument SiteMap = new XmlDocument();
SiteMap.Load(System.Web.HttpContext.Current.Server.MapPath("~/") + "//SiteMap//Sitemap-" + Sitecore.Context.Site.SiteInfo.Name + ".xml");
return this.Content(SiteMap.InnerXml, "text/xml");

I have got multiple site setup in sitecore. That's why I created sitemap as an Item in sitecore. So that it gets the right sitemap for each website.

The issue is when I submit this sitemap to google using the URL. It is indexing the sitemap URL as well and it is appearing in the actual results.

I know I can stop google from indexing my sitemap by adding X-Robot-Tag: noindex. But I can't do it IIS because it is not an item inside the website directory.

Any ideas on how that can be achieved?

Answer 1

You can specify the header in your web.config by specifying it within a location node.

<configuration>
...
    <location path="xmlsitemap">
        <system.webServer>
            <httpProtocol>
                <customHeaders>
                    <add name="X-Robots-Tag" value="noindex" />
                </customHeaders>
            </httpProtocol>
        </system.webServer>
    </location>
</configuration>

You can add this manually, the file does not need to be physically present in IIS.