Search code examples
ruby-on-railsurllistings

Getting a full list of the URLS in a rails application

How do I get a a complete list of all the urls that my rails application could generate?

I don't want the routes that I get get form rake routes, instead I want to get the actul URLs corrosponding to all the dynmically generated pages in my application...

Is this even possible?

(Background: I'm doing this because I want a complete list of URLs for some load testing I want to do, which has to cover the entire breadth of the application)

Solution

  • I was able to produce useful output with the following command:

    $ wget --spider -r -nv -nd -np http://localhost:3209/ 2>&1 | ack -o '(?<=URL:)\S+'
http://localhost:3209/
http://localhost:3209/robots.txt
http://localhost:3209/agenda/2008/08
http://localhost:3209/agenda/2008/10
http://localhost:3209/agenda/2008/09/01
http://localhost:3209/agenda/2008/09/02
http://localhost:3209/agenda/2008/09/03
^C

    A quick reference of the wget arguments:

    # --spider                  don't download anything.
# -r,  --recursive          specify recursive download.
# -nv, --no-verbose         turn off verboseness, without being quiet.
# -nd, --no-directories     don't create directories.
# -np, --no-parent          don't ascend to the parent directory.

    About ack

    ack is like grep but use perl regexps, which are more complete/powerful.

    -o tells ack to only output the matched substring, and the pattern I used looks for anything non-space preceded by 'URL:'