Streaming Rest Api - Send data in chunks to the client in a stream from the restful api

Question

I have a requirement where I have to create a streaming rest api to read a file. We are building something similar to cat linux command.

So in case if the user types in ccat filename | tail, then we have to keep the stream open and read out the content in chunks of 256 bytes and wait till the resource is closed from the client.

I have created a POC. The rest api code looks something like this -

@Produces(MediaType.APPLICATION_OCTET_STREAM)
@RequestMapping(method = RequestMethod.GET, value = "stream")
public void hello(@Context HttpServletRequest request, @Context HttpServletResponse response)
  throws InterruptedException {
String content = "This is the text content";
ServletOutputStream outputStream = null;
try {
  for(int i=0;i<100;i++){
    content = content + i;
    final byte[] bytes = content.getBytes();
    outputStream = response.getOutputStream();
    response.setContentType("application/octet-stream");
    response.setStatus(200);
    outputStream.write(bytes);
    Thread.sleep(1000l);
    outputStream.flush();
  }

  outputStream.close();
} catch (IOException e) {
  e.printStackTrace();
}

And the client code is as follows -

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.MalformedURLException;
import java.net.URL;

public class NetClientGet {
public static void main(String[] args) {

try {

  URL url = new URL("http://localhost:6868/api/stream");
  HttpURLConnection conn = (HttpURLConnection) url.openConnection();
  conn.setRequestMethod("GET");
  conn.setRequestProperty("Accept", "application/json");
  if (conn.getResponseCode() != 200) {
    throw new RuntimeException("Failed : HTTP error code : "
        + conn);
  }
  BufferedReader br = new BufferedReader(new InputStreamReader(
      (conn.getInputStream())));
  String output;
  System.out.println("Output from Server .... \n");
  while ((output = br.readLine()) != null) {
    System.out.println(output);
  }
  conn.disconnect();
} catch (MalformedURLException e) {
  e.printStackTrace();
} catch (IOException e) {
  e.printStackTrace();
}
}
}

This is working fine, but the problem is it is not writing the chunks of bytes, rather the entire content in one go. I am calling flush() so I was expecting that it will send the chunks to the client on each call of flush() but that doesn't seem to be happening. It is sending to the client after the call to the close(). How can I fix this?

Answer 1

I think it will be better to use java sockets for this use case.