I want to add nofollow
and noindex
to my site whilst it's being built. The client has request I use these rules.
I am aware of
<meta name="robots" content="noindex,nofollow">
But I only have access to the robots.txt
file.
Does anyone know the correct format I can use to apply noindex, nofollow
rules via the robots.txt
file?
noindex
and nofollow
mean that you do not want any search engines like Google to crawl your website.
So, simply put the following code into your robots.txt
file:
User-agent: *
Disallow: /
It means noindex
and nofollow
.