I am using Metasploit auxiliary/scanner/http modules like dir_listing, http_login, files_dir.... and for some modules cookie is not required, everything can be testing on the root page.
But for some modules, like the scanner, the blind_sql_query you cannot test everything withing the root page scope, if the website requires a logging or a certaing page requires a cookie, or a http_referer.
The crawler module has USER and PASSWORD options but with the login web as the start poing of crawling and the credentials well set, it doesn't happen to work well, it doesn't ask for the name of the field if its a POST login, etc.
Does someone knows how to perform it¿? How to audit with metasploit as if you were a user, the same way in other applications you can set either a cookie or login-in a form.
Because every login mechanism can be implemented a bit differently, you might need a bit more manual interaction. I think that this MSF plugin might not be the right tool for that.
I would recommend using an interception proxy for this task, with already integrated crawler. That way, you can login to the app, get the required token of authority and crawl the site. One of the best - http://portswigger.net/. This task you can do with Free version. Or OWASP Zed Attack Proxy.
If you still need to use MSF, you can chain the plugin through one of these more capable proxies, using PROXIES MSF variable.