Search code examples
pythonurllib

How to implement the equivalent of urllib.urlretrieve in Python 3?


I would like to download an APK file from http://www.apkmirror.com using a different user agent. The following works in Python 2:

import urllib

class ApkURLopener(urllib.FancyURLopener):
    version = 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.81 Safari/537.36'

urllib._urlopener = ApkURLopener()

download_link = 'https://www.apkmirror.com/wp-content/themes/APKMirror/download.php?id=215041'
download_file = '/tmp/apkmirror_test/youtube.apk'

if __name__ == "__main__":
    urllib.urlretrieve(url=download_link, filename=download_file)

I'm struggling a bit to find the code to do the same in Python 3 without using urllib.request.urlretrieve, which might become deprecated in the future. So far I've come up with

#!/usr/bin/python3
import urllib.request

download_link = 'https://www.apkmirror.com/wp-content/themes/APKMirror/download.php?id=215041'
download_file = '/tmp/apkmirror_test/youtube.apk'

USER_AGENT = 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.81 Safari/537.36'

request = urllib.request.Request(url=download_link, headers={'User-Agent': USER_AGENT})

if __name__ == "__main__":
    response = urllib.request.urlopen(url=request)

I suspect I should use the write method of the response object, but I'm not sure how, or indeed whether this is the way to do it. Any advice?


Solution

  • Based on Alternative of urllib.urlretrieve in Python 3.5, I completed the Python 3 script as follows:

    #!/usr/bin/python3
    import urllib.request
    import contextlib
    
    download_link = 'https://www.apkmirror.com/wp-content/themes/APKMirror/download.php?id=215041'
    download_file = '/tmp/apkmirror_test/youtube2.apk'
    
    USER_AGENT = 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.81 Safari/537.36'
    
    request = urllib.request.Request(url=download_link, headers={'User-Agent': USER_AGENT})
    
    if __name__ == "__main__":
        response = urllib.request.urlopen(url=request)
    
        with open(download_file, 'wb') as out_file:
            with contextlib.closing(response) as fp:
                block_size = 1024 * 8
                while True:
                    block = fp.read(block_size)
                    if not block:
                        break
                    out_file.write(block)