Converting cURL POST into urllib2 without Requests

Question

I currently have a cURL command that works in my environment. I need to convert this into a python 2.7 urllib equivalent and need some assistance

cURL command in question:

curl -k -v -XPOST -H "Authorization: Bearer $Token" -F file=@${local_filename} https://interesting_url.com/

I need the ability to support both a Header (-H) and a form/file (-F) and have yet to be successful.

This post includes a discussion on headers, but I have not been able to get the '-F' equivalent working

Answer 1

With python3 you can do it with requests:

import requests

headers = {'Authorization': 'Bearer $Token'}

files = [('file', open('${local_filename}', 'rb'))]

requests.post('https://interesting_url.com/', headers=headers, files=files, verify=False)

Uploading files with urllib2 is quite a complicated task (example). So I suggest you requests.

Long answer without using pip and third-party packages

You can implement custom class MultiPartForm and then use it to encode files:

import itertools
import mimetools
import mimetypes
from cStringIO import StringIO
import urllib
import urllib2

class MultiPartForm(object):
    """Accumulate the data to be used when posting a form."""

    def __init__(self):
        self.form_fields = []
        self.files = []
        self.boundary = mimetools.choose_boundary()
        return

    def get_content_type(self):
        return 'multipart/form-data; boundary=%s' % self.boundary

    def add_field(self, name, value):
        """Add a simple field to the form data."""
        self.form_fields.append((name, value))
        return

    def add_file(self, fieldname, filename, fileHandle, mimetype=None):
        """Add a file to be uploaded."""
        body = fileHandle.read()
        if mimetype is None:
            mimetype = mimetypes.guess_type(filename)[0] or 'application/octet-stream'
        self.files.append((fieldname, filename, mimetype, body))
        return

    def __str__(self):
        """Return a string representing the form data, including attached files."""
        # Build a list of lists, each containing "lines" of the
        # request.  Each part is separated by a boundary string.
        # Once the list is built, return a string where each
        # line is separated by '\r\n'.  
        parts = []
        part_boundary = '--' + self.boundary

        # Add the form fields
        parts.extend(
            [ part_boundary,
              'Content-Disposition: form-data; name="%s"' % name,
              '',
              value,
            ]
            for name, value in self.form_fields
            )

        # Add the files to upload
        parts.extend(
            [ part_boundary,
              'Content-Disposition: file; name="%s"; filename="%s"' % \
                 (field_name, filename),
              'Content-Type: %s' % content_type,
              '',
              body,
            ]
            for field_name, filename, content_type, body in self.files
            )

        # Flatten the list and add closing boundary marker,
        # then return CR+LF separated data
        flattened = list(itertools.chain(*parts))
        flattened.append('--' + self.boundary + '--')
        flattened.append('')
        return '\r\n'.join(flattened)

with open(`local_file.txt`) as f:
    form = MultiPartForm()
    form.add_file('file', `local_file`, 
                  fileHandle=f)

    # Build the request
    request = urllib2.Request('https://interesting_url.com/')
    request.add_header('Authorization', 'Bearer $Token')
    body = str(form)
    request.add_header('Content-type', form.get_content_type())
    request.add_header('Content-length', len(body))
    request.add_data(body)

    print
    print 'OUTGOING DATA:'
    print request.get_data()

    print
    print 'SERVER RESPONSE:'
    print urllib2.urlopen(request).read()

The source