Determine difference in http request between Python2 and Python3

Question

I am attempting to use Python3 to send metrics to Hosted Graphite. The examples given on the site are Python2, and I have successfully ported the TCP and UDP examples to Python3 (despite my inexperience, and have submitted the examples so the docs may be updated), however I have been unable to get the HTTP method to work.

The Python2 example looks like this:

import urllib2, base64

url = "https://hostedgraphite.com/api/v1/sink"
api_key = "YOUR-API-KEY"

request = urllib2.Request(url, "foo 1.2")
request.add_header("Authorization", "Basic %s" % base64.encodestring(api_key).strip())
result = urllib2.urlopen(request)

This works successfully, returning a HTTP 200.

So far I have ported this much to Python3, and while I was (finally) able to get it to make a valid HTTP request (i.e. no syntax errors), the request fails, returning HTTP 400

import urllib.request, base64

url = "https://hostedgraphite.com/api/v1/sink"
api_key = b'YOUR-API-KEY'

metric = "testing.python3.http 1".encode('utf-8')
request = urllib.request.Request(url, metric)
request.add_header("Authorization", "Basic %s" % base64.encodestring(api_key).strip())
result = urllib.request.urlopen(request)

The full result is:

>>> result = urllib.request.urlopen(request)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/Cellar/python3/3.3.1/Frameworks/Python.framework/Versions/3.3/lib/python3.3/urllib/request.py", line 160, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/local/Cellar/python3/3.3.1/Frameworks/Python.framework/Versions/3.3/lib/python3.3/urllib/request.py", line 479, in open
    response = meth(req, response)
  File "/usr/local/Cellar/python3/3.3.1/Frameworks/Python.framework/Versions/3.3/lib/python3.3/urllib/request.py", line 591, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/local/Cellar/python3/3.3.1/Frameworks/Python.framework/Versions/3.3/lib/python3.3/urllib/request.py", line 517, in error
    return self._call_chain(*args)
  File "/usr/local/Cellar/python3/3.3.1/Frameworks/Python.framework/Versions/3.3/lib/python3.3/urllib/request.py", line 451, in _call_chain
    result = func(*args)
  File "/usr/local/Cellar/python3/3.3.1/Frameworks/Python.framework/Versions/3.3/lib/python3.3/urllib/request.py", line 599, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 400: Bad Request

Is it obvious what I am doing wrong? Are there any suggestions on how I might capture and compare what the successful (python2) and failing (python3) requests are actually sending?

Answer 1

Don't mix Unicode strings and bytes:

>>> "abc %s" % b"def"
"abc b'def'"

You could construct the header as follows:

from base64 import b64encode

headers = {'Authorization': b'Basic ' + b64encode(api_key)}

A quick way to see the request is to change the host in the url to localhost:8888 and run before making the request:

$ nc -l 8888

You could also use wireshark to see the requests.