No JSON object could be decoded after retrieving JSON content using POST without Gzip encoding

Question

Using Python 2.7.8 we get "ValueError: No JSON object could be decoded", if running this script:

from urllib2 import urlopen, Request
from json import dumps, loads, load

values = dumps({
    "issueId": 10600,
    "versionId": "10000",
    "cycleId": "16",
    "projectId": 10000
})
headers = {"Content-Type": "application/json"}
request = Request("http://private-anon-491d363a1-getzephyr.apiary-mock.com/jira_server/rest/zapi/latest/execution", data=values, headers=headers)
print request.get_method()
print request.get_header("Content-Encoding")

response = urlopen(request)
print "\n" + response.read()
print response.getcode()
print response.geturl()
print response.info()
json = loads(response.read())  # raises ValueError("No JSON object could be decoded")

Output is:

POST
None

{
    "32": {
        "id": 32,
        "executionStatus": "-1",
        "comment": "",
        "htmlComment": "",
        "cycleId": 16,
        "cycleName": "Audit Test Cycle 3",
        "versionId": 10000,
        "versionName": "v1",
        "projectId": 10000,
        "issueId": 10600,
        "issueKey": "ZFJ-19",
        "summary": "test - check1",
        "label": "",
        "component": ""
    }
}
200
http://private-anon-491d363a1-getzephyr.apiary-mock.com/jira_server/rest/zapi/latest/execution
Server: Cowboy
X-Apiary-Ratelimit-Limit: 120
X-Apiary-Ratelimit-Remaining: 119
Content-Type: application/json
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: OPTIONS,GET,HEAD,POST,PUT,DELETE,TRACE,CONNECT
Access-Control-Max-Age: 10
X-Apiary-Transaction-Id: 55423c1c7996e10300e32acc
Date: Thu, 30 Apr 2015 14:28:45 GMT
X-Cache: MISS from p-proxy.int.hrs.com
X-Cache-Lookup: MISS from p-proxy.int.hrs.com:3128
Via: 1.1 vegur, 1.0 p-proxy.int.hrs.com:3128 (squid/2.6.STABLE6)
Proxy-Connection: close

Traceback (most recent call last):
  File "test2.py", line 20, in <module>
    json = loads(response.read())  # raises ValueError("No JSON object could be decoded")
  File "C:\Program Files (x86)\python27\lib\json\__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "C:\Program Files (x86)\python27\lib\json\decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "C:\Program Files (x86)\python27\lib\json\decoder.py", line 384, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

I read a lot of Q&A involving aspects like Gzip and requests lib but it was not helpful. How can I decode the response using urlib2 and json libs?? If I copy the response into a file and parse the file, it works so it is valid json.

Answer 1

You cannot .read() file-like object (that is what according to the docs urlopen returns) twice. I modified your code, try it:

from urllib2 import urlopen, Request
from json import dumps, loads, load

values = dumps({
    "issueId": 10600,
    "versionId": "10000",
    "cycleId": "16",
    "projectId": 10000
})
headers = {"Content-Type": "application/json"}
request = Request("http://private-anon-491d363a1-getzephyr.apiary-mock.com/jira_server/rest/zapi/latest/execution", data=values, headers=headers)
print request.get_method()
print request.get_header("Content-Encoding")

response = urlopen(request)
text = response.read()         #storing the data
print "\n" + text              #note the usage of the stored string
print response.getcode()
print response.geturl()
print response.info()
json = loads(text)             #note the usage of the stored string