Search code examples
regexlinuxbashwget

How to download an entire directory and subdirectories using wget?


I am trying to download the files for a project using wget, as the SVN server for that project isn't running anymore and I am only able to access the files through a browser. The base URLs for all the files is the same like

http://abc.tamu.edu/projects/tzivi/repository/revisions/2/raw/tzivi/*

How can I use wget (or any other similar tool) to download all the files in this repository, where the "tzivi" folder is the root folder and there are several files and sub-folders (upto 2 or 3 levels) under it?


Solution

  • You may use this in shell:

    wget -r --no-parent http://abc.tamu.edu/projects/tzivi/repository/revisions/2/raw/tzivi/
    

    The Parameters are:

    -r     //recursive Download
    

    and

    --no-parent // Don´t download something from the parent directory
    

    If you don't want to download the entire content, you may use:

    -l1 just download the directory (tzivi in your case)
    
    -l2 download the directory and all level 1 subfolders ('tzivi/something' but not 'tivizi/somthing/foo')  
    

    And so on. If you insert no -l option, wget will use -l 5 automatically.

    If you insert a -l 0 you´ll download the whole Internet, because wget will follow every link it finds.