Search code examples
pythonhttpdownloadcommand-line-interface

Downloading a file from the command line using Python


I am looking for a quick way to download a file via HTTP, using a Python one-liner from the command line (similar to the functionality of wget or curl). The idea is to enable a quick copy/paste to download distutils on Windows.

I know of one solution (see my answer below). I'm interested in other solutions that consider the following:

  • Concise
  • Most "Pythonic" solution
  • Compatible with both Python 2 and Python 3
  • Cross-platform
  • Can deal with large files efficiently
  • No dependencies (we're fetching distutils here. It's unlikely we'll have access to requests at this stage)
  • Correctly handles various HTTP headers, such as Content-Disposition

Solution

  • The simplest solution I could come up with would be:

    try:
        from urllib.request import urlretrieve
    except ImportError:
        from urllib import urlretrieve
    
    urlretrieve('http://example.org', 'outfile.dat')
    

    urlretrieve takes care of downloading the resource to a local file and can deal with large files.

    It, however, ignores Content-Disposition headers. If you want that to be considered, you'd need to use urlopen and parse the response headers yourself. Content-Disposition isn't a HTTP standard header, so I doubt you will find much support for it in the Python HTTP libraries...