Apache HTTPClient - consume an indefinite HTTP GET Stream

Question

Companies House, in the UK, have recently released a HTTP 'stream' webservice to allow for developers to listen indefinitely for Company changes.

In below is the important section of their help page

Establishing a connection to the streaming APIs involves making a long-running HTTP request, and incrementally processing each response line. Conceptually, you can think of this as downloading an infinitely long file over HTTP.

Using Apache HTTP Client, i can see this 'stream' of company changes in the debug console output of the http client library using the following kotlin code

val httpClient = HttpClients.createDefault()
    val request = HttpGet("https://stream.companieshouse.gov.uk/companies")
    request.addHeader("Authorization", "xxxxxxxxxxxxx");
    httpClient.execute(request).use { response1 ->
        val entity: HttpEntity = response1.entity
        entity.content?.use { inputStream -> println("output-->" + String(inputStream.readAllBytes())) }
    }

however, my console output is never hit (ie, the print string of 'output-->' in the above)

Question: Using Apache HTTP Client, is it possible to consume an indefinite HTTP Get connection? If so, how?

Answer 1

You can achieve this behaviour by consuming the stream returned indefinitely from the http response entity.

Here is a Java example of how you would do that.

    var request = new HttpGet("https://stream.companieshouse.gov.uk/companies");
    request.addHeader(HttpHeaders.AUTHORIZATION, auth);

    try (var stream = client.execute(request).getEntity().getContent()) {
        var buffered = new BufferedReader(new InputStreamReader(new BufferedInputStream(stream)));

        while (true) {

          String value = buffered.readLine();

          if(!value.isBlank()) {
            System.out.printf("Event: %s ", value);
          }
        }
    }

It should be possible to convert the snippet into Kotlin.