Search code examples
seosearch-enginerobots.txt

Prevent search engines indexing pages with 3 or more forwarslashes in their paths

How can I prevent search engines indexing pages with 3 or more forwarslashes in their paths? for example:

www.example.com/about.html ->ok
www.example.com/1/2/3/4/5/test.html -> no index!!!

How should I write the robots.txt? Thanks!

Solution

  • Try with

    User-Agent: *
Disallow: /*/*/*/*

    Although I'm not sure all bots support the *.