I have been using this helper class to parse JSON data for a while now but upon doing updates the app crashes the 2nd time it is used at any point while the app is running. I can't seem to figure out why this is happening.
Update 2
The issue is with redirects and has nothing to do with cookies but I was able to fix my problem. Check my answer for solution.
Update 1
After Aaron's suggestion. I in fact don't get any JSON back after the first request. If this is an issue with a cookie why does it work the first time?
HTTP/1.1 302 Moved Temporarily
Content-Type: text/html; charset=UTF-8
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Pragma: no-cache
Expires: Mon, 01 Jan 1990 00:00:00 GMT
Date: Tue, 03 Jan 2017 13:31:59 GMT
Location: https://docs.google.com/spreadsheets/d/I REMOVED THE KEY/gviz/tq
P3P: CP="This is not a P3P policy! See https://support.google.com/accounts/answer/151657?hl=en for more info."
P3P: CP="This is not a P3P policy! See https://support.google.com/accounts/answer/151657?hl=en for more info."
MainActivity
new DownloadWebpageTask1(new AsyncResult1() {
@Override
public void onResult(JSONObject object) {
processJson(object);
}
}).execute("SOMEWEBSITE");
DownloadWebpageTask1.class
public class DownloadWebpageTask1 extends AsyncTask<String, Void, String> {
com.test.test.AsyncResult1 callback;
public DownloadWebpageTask1(com.test.test.AsyncResult1 callback) {
this.callback = callback;
}
@Override
protected String doInBackground(String... urls) {
// params comes from the execute() call: params[0] is the url.
try {
return downloadUrl(urls[0]);
} catch (IOException e) {
return "Unable to download the requested page.";
}
}
// onPostExecute displays the results of the AsyncTask.
@Override
protected void onPostExecute(String result) {
// remove the unnecessary parts from the response and construct a JSON
int start = result.indexOf("{", result.indexOf("{") + 1);
int end = result.lastIndexOf("}");
String jsonResponse = result.substring(start, end);
Log.i("Error", jsonResponse);
try {
JSONObject table = new JSONObject(jsonResponse);
callback.onResult(table);
} catch (JSONException e) {
e.printStackTrace();
}
}
private String downloadUrl(String urlString) throws IOException {
InputStream is = null;
try {
URL url = new URL(urlString);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setReadTimeout(30000 /* milliseconds */);
conn.setConnectTimeout(30000 /* milliseconds */);
conn.setRequestMethod("GET");
conn.setDoInput(true);
// Starts the query
conn.connect();
int responseCode = conn.getResponseCode();
is = conn.getInputStream();
String contentAsString = convertStreamToString(is);
return contentAsString;
} finally {
if (is != null) {
is.close();
}
}
}
private String convertStreamToString(InputStream is) {
BufferedReader reader = new BufferedReader(new InputStreamReader(is));
StringBuilder sb = new StringBuilder();
String line = null;
try {
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
is.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return sb.toString();
}
}
JSON
/*O_o*/
google.visualization.Query.setResponse({"version":"0.6","reqId":"0","status":"ok","sig":"806967004","table":{"cols":[{"id":"A","label":"Date","type":"date","pattern":"M/d/yyyy"},{"id":"B","label":"Name","type":"string"},{"id":"C","label":"News","type":"string"},{"id":"D","label":"url","type":"string"},{"id":"E","label":"icon","type":"string"},{"id":"F","label":"picture","type":"string"}],"rows":[{"c":[{"v":"Date(2017,0,2)","f":"1/2/2017"},{"v":"Event 1"},{"v":"This is some event data for event 1"},{"v":""},{"v":"/icon1.png"},{"v":"Test"}]},{"c":[{"v":"Date(2017,0,2)","f":"1/2/2017"},{"v":"Event 2"},{"v":"This is some event data for event 2"},{"v":"https://www.google.com/"},{"v":"/icon2.png"},{"v":"test"}]},{"c":[{"v":"Date(2016,11,30)","f":"12/30/2016"},{"v":"Event 3"},{"v":"This is some event data for event 3"},{"v":""},{"v":"/icon3.png"},{"v":"test"}]},{"c":[{"v":"Date(2016,11,30)","f":"12/30/2016"},{"v":"Event 4"},{"v":"This is some event data for event 4"},{"v":""},{"v":"/icon4.png"},{"v":"test"}]},{"c":[{"v":"Date(2016,11,30)","f":"12/30/2016"},{"v":"Event 5"},{"v":"This is some event data for event 5"},{"v":"https://www.google.com/"},{"v":"/icon5.png"},{"v":"test"}]}]}});
Error
W/System.err: java.net.SocketTimeoutException: Read timed out
W/System.err: at org.apache.harmony.xnet.provider.jsse.NativeCrypto.SSL_read(Native Method)
W/System.err: at org.apache.harmony.xnet.provider.jsse.OpenSSLSocketImpl$SSLInputStream.read(OpenSSLSocketImpl.java:664)
W/System.err: at libcore.net.http.UnknownLengthHttpInputStream.read(UnknownLengthHttpInputStream.java:41)
W/System.err: at java.io.InputStreamReader.read(InputStreamReader.java:244)
W/System.err: at java.io.BufferedReader.fillBuf(BufferedReader.java:130)
W/System.err: at java.io.BufferedReader.readLine(BufferedReader.java:354)
W/System.err: at com.test.test.DownloadWebpageTask1.convertStreamToString(DownloadWebpageTask1.java:84)
W/System.err: at com.test.test.DownloadWebpageTask1.downloadUrl(DownloadWebpageTask1.java:69)
W/System.err: at com.test.test.DownloadWebpageTask1.doInBackground(DownloadWebpageTask1.java:32)
W/System.err: at com.test.test.DownloadWebpageTask1.doInBackground(DownloadWebpageTask1.java:20)
W/System.err: at android.os.AsyncTask$2.call(AsyncTask.java:287)
W/System.err: at java.util.concurrent.FutureTask.run(FutureTask.java:234)
W/System.err: at android.os.AsyncTask$SerialExecutor$1.run(AsyncTask.java:230)
W/System.err: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1080)
W/System.err: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:573)
W/System.err: at java.lang.Thread.run(Thread.java:856)
W/dalvikvm: threadid=1: thread exiting with uncaught exception (group=0xa6165908)
E/AndroidRuntime: FATAL EXCEPTION: main
java.lang.StringIndexOutOfBoundsException: length=1566; regionStart=-1; regionLength=0
at java.lang.String.startEndAndLength(String.java:583)
at java.lang.String.substring(String.java:1464)
at com.test.test.DownloadWebpageTask1.onPostExecute(DownloadWebpageTask1.java:44)
at com.test.test.DownloadWebpageTask1.onPostExecute(DownloadWebpageTask1.java:20)
at android.os.AsyncTask.finish(AsyncTask.java:631)
at android.os.AsyncTask.access$600(AsyncTask.java:177)
at android.os.AsyncTask$InternalHandler.handleMessage(AsyncTask.java:644)
at android.os.Handler.dispatchMessage(Handler.java:99)
at android.os.Looper.loop(Looper.java:137)
at android.app.ActivityThread.main(ActivityThread.java:5041)
at java.lang.reflect.Method.invokeNative(Native Method)
at java.lang.reflect.Method.invoke(Method.java:511)
at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:793)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:560)
at dalvik.system.NativeStart.main(Native Method)
This is an issue having to do with redirects. Google changed the URL where JSON data is pulled from.
In your URL request change
https://spreadsheets.google.com/tq?key=YOUR_KEY_HERE
To
https://docs.google.com/spreadsheets/d/YOUR_KEY_HERE/gviz/tq
And some slightly modified code from https://stackoverflow.com/a/40105282/6147989 and everything will be up and running.
private String downloadUrl(String urlString) throws IOException {
String temp = "";
try {
URL obj = new URL(urlString);
HttpURLConnection conn = (HttpURLConnection) obj.openConnection();
conn.setReadTimeout(5000);
conn.addRequestProperty("Accept-Language", "en-US,en;q=0.8");
conn.addRequestProperty("User-Agent", "Mozilla");
conn.addRequestProperty("Referer", "google.com");
System.out.println("Request URL ... " + urlString);
boolean redirect = false;
// normally, 3xx is redirect
int status = conn.getResponseCode();
if (status != HttpURLConnection.HTTP_OK) {
if (status == HttpURLConnection.HTTP_MOVED_TEMP
|| status == HttpURLConnection.HTTP_MOVED_PERM
|| status == HttpURLConnection.HTTP_SEE_OTHER)
redirect = true;
}
System.out.println("Response Code ... " + status);
if (redirect) {
// get redirect url from "location" header field
String newUrl = conn.getHeaderField("Location");
// get the cookie if need, for login
String cookies = conn.getHeaderField("Set-Cookie");
// open the new connnection again
conn = (HttpURLConnection) new URL(newUrl).openConnection();
conn.setRequestProperty("Cookie", cookies);
conn.addRequestProperty("Accept-Language", "en-US,en;q=0.8");
conn.addRequestProperty("User-Agent", "Mozilla");
conn.addRequestProperty("Referer", "google.com");
System.out.println("Redirect to URL : " + newUrl);
}
BufferedReader in = new BufferedReader(
new InputStreamReader(conn.getInputStream()));
String inputLine;
StringBuffer html = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
html.append(inputLine);
}
in.close();
System.out.println("URL Content... \n" + html.toString());
System.out.println("Done");
temp = html.toString();
} catch (Exception e) {
e.printStackTrace();
}
return temp;
}