Android ByteArrayOutputStream corrupting HTTP GET JSONArray

Question

I'm using this code to parse a JSON array I'm getting from my server.

    try {
            URL u = new URL("http://54.68.139.250/get_user_likes");
            HttpURLConnection conn = (HttpURLConnection) u.openConnection();
            conn.setRequestMethod("GET");

            conn.connect();
            InputStream is = conn.getInputStream();
            byte[] b = new byte[1024];
            ByteArrayOutputStream baos = new ByteArrayOutputStream();

            while ( is.read(b) != -1)
                baos.write(b);

            String JSONResp = new String(baos.toByteArray());

            JSONArray arr = new JSONArray(JSONResp);
            for (int i=0; i < arr.length(); i++) {
                result.add(convertArticle(arr.getJSONObject(i)));
            }
            return result;
        }
        catch(Throwable t) {
            t.printStackTrace();
        }
        return null;

This code works great on my phone. Unfortunately, when I'm using a Genymotion emulator with the virtual device of Google Nexus 7, the JSON array is slightly altered. 95% of the JSON array is fine, but it is truncated near the very end and is randomly missing about 4 characters of the json array at character 1253 so I'm getting:

org.json.JSONException: Expected ':' after top_id at character 1253 of [{"top_id":6,"top_url":

I'm thinking this is some memory problem with the emulator. Its base memory is 1024. Increasing that amount though doesn't change anything. Any tips as to the reason behind the problem would be greatly appreciated. Also, feel free to comment on my code if you see room for improvement. :)

Answer 1

That's weird.

I can think of two things to try:

• Check the encoding of the server and the encoding of the String constructor.

It's possible that the server is decoding with, say, Latin-1 (ISO-8859-1) and the String is encoding with UTF-8. But JSON is supposed to be sent in Unicode and the default charset for Android is UTF-8. Check the HTTP Content-type response header; it should say: application/json; charset=utf-8. If the charset part isn't there, do some investigation and find out what character set your server is using for decoding to HTTP stream. And check that the default charset on the phone and the emulator is the same; it should be UTF-8.

• Try calling flush() on the ByteArrayOutputStream after you read the data and before you construct the String.

It may be that there is a slight difference between the phone OS and the emulator OS on how stream data is transferred/buffered/flushed.

If flush() doesn't work, try rewriting the code without using ByteArrayOutputStream. You could, for example, wrap the input stream with an InputStreamReader, which reads characters, not bytes, then append the characters using a StringBuilder or StringBuffer.

One way you could make the code better is to use JSONReader instead of JSONArray or JSONObject. The JSONReader would wrap an InputStreamReader which in turn wraps the HTTP input stream. It can be faster and more memory efficient since you don't have to read the entire input stream before starting to parse the data. When the server is sending a LOT of JSON data, that can make a big difference.