Search code examples
asp.net-mvcazuresearch-enginerobots.txtazure-traffic-manager

Hide Traffic Manager Endpoint from Search Engines but Expose the Custom Domain

I'm running a ASP.NET MVC website hosted on Azure and using Traffic Manager as load balancer, thus the users can not only visit the website by custom domain like foobar.yourcompany.com, but also endpoint provided by Azure Traffic Manager like foobar.trafficmanager.net. So do Search Engine crawlers.

Is there any feasible solution to hide the Traffic Manager endpoint from crawlers and expose the custom domain endpoint?

P.S. I know robots.txt.

Solution

  • If you setup your app to return an HTTP 301 permanent redirect to your custom domain when accessed over the Traffic Manager domain, it should stop it from showing up in search results. As far as I know, you can't fully disable the TM domain. Similarly like you can't fully disable the App Service domains.

    So e.g. if your app gets a request for:

    http://foobar.trafficmanager.net/foo/bar

    Redirect it with an HTTP 301 to:

    http://foobar.yourcompany.com/foo/bar