Search code examples
javaurlconnection

Java read from URL stream working selectively


Summary: Sample Java code that reads over a URLConnection reads only certain URLs, not others.

Details: I have this sample Java code that I am using to read over a URLConnection. When the URL is "http://www.example.com", the code reads the page content without any issues. However, if the URL is "http://www.cnn.com", the page content is not read

public class StackOverflow {
    public static void main(String[] args) throws Exception {
        BufferedReader inputStream = null;
        try {
            String urlStr = "http://www.cnn.com"; // Does not work
//          urlStr = "http://www.example.com"; // **Works if this line is uncommented**

            URL url = new URL(urlStr);

            inputStream = new BufferedReader(new InputStreamReader(url.openStream()));

            String textLine = null;
            while((textLine = inputStream.readLine()) != null) {
                System.out.println(textLine);
            }
        }
        catch (Exception e) {
            e.printStackTrace();
        }
        finally {
            if(inputStream != null) inputStream.close();
        }
    }
}

Solution

  • CNN redirects from http to https but your call doesn't follow redirects. You are getting a 307 with an empty body so the readline results in a null and your loop is skipped. Try with https for CNN.