urllib2/pycurl in Django: Fetch XML, check HTTP status, check HTTPS connection

Question

I need to make an API call (of sorts) in Django as a part of the custom authentication system we require. A username and password is sent to a specific URL over SSL (using GET for those parameters) and the response should be an HTTP 200 "OK" response with the body containing XML with the user's info.

On an unsuccessful auth, it will return an HTTP 401 "Unauthorized" response.

For security reasons, I need to check:

1. The request was sent over an HTTPS connection
2. The server certificate's public key matches an expected value (I use 'certificate pinning' to defend against broken CAs)

Is this possible in python/django using pycurl/urllib2 or any other method?

Answer 1

Using M2Crypto:

from M2Crypto import SSL
ctx = SSL.Context('sslv3')
ctx.set_verify(SSL.verify_peer | SSL.verify_fail_if_no_peer_cert, depth=9)
if ctx.load_verify_locations('ca.pem') != 1:
   raise Exception('No CA certs')

c = SSL.Connection(ctx)
c.connect(('www.google.com', 443)) # automatically checks cert matches host
c.send('GET / \n')
c.close()

Using urllib2_ssl (it goes without saying but to be explicit: use it at your own risk):

import urllib2, urllib2_ssl

opener = urllib2.build_opener(urllib2_ssl.HTTPSHandler(ca_certs='ca.pem'))
xml = opener.open('https://example.com/').read()

Related: Making HTTPS Requests secure in Python.

Using pycurl:

c = pycurl.Curl()
c.setopt(pycurl.URL, "https://example.com?param1=val1&param2=val2")
c.setopt(pycurl.HTTPGET, 1)
c.setopt(pycurl.CAINFO, 'ca.pem')
c.setopt(pycurl.SSL_VERIFYPEER, 1)
c.setopt(pycurl.SSL_VERIFYHOST, 2)
c.setopt(pycurl.SSLVERSION,     3)    
c.setopt(pycurl.NOBODY, 1)
c.setopt(pycurl.NOSIGNAL, 1)
c.perform()
c.close()

To implement 'certificate pinning' provide different 'ca.pem' for different domains.