Streaming data over HTTP with Java

Question

I have two Java Spring Boot applications and let's name them Source and Consumer. Source has a GET endpoint /api/data that returns infinite stream of data. The idea is to call it from Consumer and listen for chunks of data every a few seconds and "never" close this connection. I already made a simple Source that looks working for now:

@RestController
@RequestMapping ("/api")
public class SourceController {

    private final Logger logger = LoggerFactory.getLogger(SourceController.class);

    @GetMapping (value = "/data", produces = MediaType.APPLICATION_JSON_VALUE)
    public ResponseEntity<StreamingResponseBody> data(final HttpServletResponse response) {
        response.setContentType("application/json");
        StreamingResponseBody stream = out -> {
            try {
                for (int i = 0; i < 10; i++) {
                    String content = "{\"counter\":" + i + "}\n";
                    out.write(content.getBytes());
                    logger.info("size: "  + content.getBytes().length);
                    Thread.sleep(1000);
                }
                out.close();
            } catch (final Exception e) {
                logger.error("Exception", e);
            }
        };
        logger.info("steaming response {} ", stream);
        return new ResponseEntity(stream, HttpStatus.OK);
    }
}

I'm not sure is it exactly what I want because when I call it with Postman the response comes after 10 seconds when return is executed.

The consumer reads but reads the whole response, not piece by piece.

@RestController
@RequestMapping("/api")
public class ConsumerController {

    private final Logger logger = LoggerFactory.getLogger(ConsumerController.class);

    @GetMapping(value = "/consume", produces = MediaType.APPLICATION_JSON_VALUE)
    public ResponseEntity<String> consume() throws IOException {
        URL url = new URL("http://localhost:8080/api/data");
        URLConnection connection = url.openConnection();
        connection.setDoOutput(true);

        BufferedReader in = new BufferedReader(
            new InputStreamReader(connection.getInputStream())
        );

        String decodedString;
        while ((decodedString = in.readLine()) != null) {
            logger.info(decodedString);
        }

        in.close();

        return new ResponseEntity("ok", HttpStatus.OK);
    }
}

Answer 1

The source looks correct but you just forget to flush the OutputStream after writing a chunk of data to it , so consumer cannot receive this chunk of data immediately.

So calling flush() after writing some to OutputStream should solve the problem:

 for (int i = 0; i < 10; i++) {
       String content = "{\"counter\":" + i + "}\n";
       out.write(content.getBytes());
       out.flush();
       logger.info("size: "  + content.getBytes().length);
       Thread.sleep(1000);
}