Slow Download of some Links running HTTPUrlConnection in Java

Question

i´m new to programming. I´m trying to write an App for Android which downloads a List with Links in HTML and uses the given Links to download the content of these Links. I will Post the Code of the Downloader and the Log. My Question is: Why do some Links need around 5 seconds and some of them up to 5 minutes? The cause of this cant be the Website, cause when i´m opening the Link in Chrome or other Browsers the Page is instantly loaded. Can u guys help me?

package com.example.newsreader;

import android.util.Log;

import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;

public class MyRunnable implements Runnable {

    String result = "";
    String urlSource="";

    public MyRunnable(String urlSource){
        this.urlSource = urlSource;
    }

    @Override
    public void run() {
        Thread t1 = new Thread(new Runnable() {
            @Override
            public void run() {
                try {
                    result = "";
                    URL url = new URL(urlSource);
                    HttpURLConnection urlConnection = (HttpURLConnection) url.openConnection();
                    urlConnection.setRequestMethod("GET");
                    InputStream inputStream = urlConnection.getInputStream();
                    InputStreamReader inputStreamReader = new InputStreamReader(inputStream);

                    int data = inputStreamReader.read();

                    while (data !=-1) {
                        char current = (char) data;
                        result += current;
                        data = inputStreamReader.read();
                    }

                    inputStream.close();

                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
        });
        t1.start();
        try{
            t1.join();
        } catch (Exception e){
            e.printStackTrace();
        }
    }

    public String getResult(){
        return result;
    }
}

LOGDATA: Times for Each Link to finish

Answer 1

You are reading data one by one, it's not efficient. Instead you should create char buffer and read data by small chunks.

Also don't concatenate many strings, use StringBuilder, it uses less memory for such things.

 public class MyRunnable implements Runnable {

    private final StringBuilder result = new StringBuilder();
    String urlSource="";
    private final char[] buffer = new char[512]; //allocating buffer

    public MyRunnable(String urlSource){
        this.urlSource = urlSource;
    }

    @Override
    public void run() {
        Thread t1 = new Thread(new Runnable() {
            @Override
            public void run() {
                try {
                    result.setLength(0); //clearing StringBuilder
                    URL url = new URL(urlSource);
                    HttpURLConnection urlConnection = (HttpURLConnection) url.openConnection();
                    urlConnection.setRequestMethod("GET");
                    InputStream inputStream = urlConnection.getInputStream();
                    InputStreamReader inputStreamReader = new InputStreamReader(inputStream);
                    
                    // now reading not by 1 char, but by 512 for once
                    int i;
                    while ((i = inputStreamReader.read(buffer)) != -1) {
                       result.append(buffer, 0, i);
                    }

                    inputStreamReader.close(); //close reader instead of InputStream

                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
        });
        t1.start();
        try{
            t1.join();
        } catch (Exception e){
            e.printStackTrace();
        }
    }

    public String getResult(){
        return result.toString(); //converting StringBuilder to String
    }
}

And use BufferedInputStream, it also increases speed.

InputStreamReader inputStreamReader = new InputStreamReader(new BufferedInputStream(inputStream));

Some tests with "https://www.google.com":

Simple run
time: 838 ms
run with char buffer
time: 264 ms
Run with char buffer and BufferedInputStream
time: 130 ms