How to convert Http url (image URL) to byte buffer or in byte?

I have tried some functions to convert url image to byte buffer but it doesn't work. It shows an IOException error

String imagePath = "https://www.clearias.com/up/UPSC-Civil-Services-Mains-Exam-2018-Timetable-V1.png";

Try 1:

URL u = new URL(imagePath);
int contentLength = u.openConnection().getContentLength();
InputStream openStream = u.openStream();
byte[] binaryData = new byte[contentLength];
openStream.read(binaryData);
ByteBuffer imageBytes = ByteBuffer.wrap(openStream);

This shows error when i wrap openStream to ByteBuffer

"The method wrap(byte[]) in the type ByteBuffer is not applicable for the arguments (InputStream)"

Try 2:

    URL url = new URL(imagePath);
    ByteArrayOutputStream output = new ByteArrayOutputStream();

    try (InputStream inputStream = url.openStream()) {
        int n = 0;    
        byte[] buffer = new byte[1024];    
        while (-1 != (n = inputStream.read(buffer))) {
            output.write(buffer, 0, n);
        }
    }
    byte[] img = output.toByteArray();    
    ByteBuffer imageBytes = ByteBuffer.wrap(img);

I have also tried this function but it shows this error:

java.io.IOException: Server returned HTTP response code: 403 for URL: https://www.clearias.com/up/UPSC-Civil-Services-Mains-Exam-2018-Timetable-V1.png

Try 3: And another is

byte[] img = Base64.encodeBase64(IOUtils.toByteArray((new URL(imagePath)).openStream()), true);

This line also gives me error

Solution

The HTTP error 403 means that the server has fully understood your intention to request this image and download it. However, it estimates that you have no right to do so, not because of a programming error, but because you shouldn't be allowed to do that.

Normally the conclusion should be that it is not possible to obtain this image from just its URL, and maybe you need to provide some form of authentication alongside the request to prove to the server that you should be allowed to get the image.

However it is clear, by just copy/pasting the URL in a browser, that in "normal" conditions the server accepts to give out the image unconditionally. The server only rejects the request when it is made from a Java program (and maybe other technologies too, I did not test.) The question remaining is, how does the server tell that the request is made by a Java program? Or, in more general terms, how does the server decide whether or not you have a right to make this request?

In theory we cannot guess everyone's intentions, but it is common for HTTP servers that want to deny requests from specific technologies, to do that based on the User-Agent HTTP header of the request. So I decided to modify the User-Agent that Java sends by default, and lie and pretend the request is done by Firefox (in a very naive way)

Here's the code:

URL url = new URL(imagePath);
ByteArrayOutputStream output = new ByteArrayOutputStream();
URLConnection conn = url.openConnection();
conn.setRequestProperty("User-Agent", "Firefox");

try (InputStream inputStream = conn.getInputStream()) {
  int n = 0;
  byte[] buffer = new byte[1024];
  while (-1 != (n = inputStream.read(buffer))) {
    output.write(buffer, 0, n);
  }
}
byte[] img = output.toByteArray();
ByteBuffer imageBytes = ByteBuffer.wrap(img);

It works.