Search code examples
pythonpython-2.7urllib2urlopen

Read and put by chunk with urllib2.urlopen synchronously

I have a simple Python script which should read a file from HTTP source and make a PUT request to another HTTP source.

block_size = 4096
file = urllib2.urlopen('http://path/to/someting.file').read(block_size)    
headers = {'X-Auth-Token': token_id, 'content-type': 'application/octet-stream'} 
response = requests.put(url='http://server/path', data=file, headers=headers)

How can I make synchronous reading and putting this file by block_size (chunk) while the block is not empty?

Solution

  • What you want to do is called "streaming uploads". Try the following.

    Get the file as a stream:

    resp = requests.get(url, stream = True)

    And then post the file like object:

    requests.post(url, data= resp.iter_content(chunk_size= 4096))