Search code examples
urlpathseorobots.txtslash

What's the meaning of the slash in the end of disallow path?

Here are two lines in robots.txt file:

Disallow: /messages
Disallow: /qanda/edit/

What paths is this /messages point to? And what about /qanda/edit/? Is the / in the end of the second path useless? Or it has a specific meaning?

Or let me ask this way: what's the exact meaning of theses?

Disallow: /messages
Disallow: /messages/
Solution

  • Disallow values represent the beginning of the URL path.

    Disallow: /messages

    would block URLs like these:

    https://example.com/messages
https://example.com/messages.html
https://example.com/messages/
https://example.com/messages/foo

    Disallow: /messages/

    would block URLs like these:

    https://example.com/messages/
https://example.com/messages/foo

    but allow URLs like these:

    https://example.com/messages
https://example.com/messages.html