Closing InputStream hangs when reading a radio stream

Question

I'm using Apache HttpClient to fetch meta-data from radio stream. What I want to do is make a GET request, read some bytes then close the stream. For some streams, it works properly but for some others, the closing of the stream hangs. It looks like it's still receiving data and not closing the connection while it's happening.

    public SongInfo retrieveMetadata(String streamUrl) {
        HttpClient httpClient = HttpClients.createDefault();
        HttpGet httpGet = new HttpGet(streamUrl);
        httpGet.addHeader("Icy-MetaData", "1");
        httpGet.addHeader("Connection", "close");
        httpGet.addHeader("Accept", "");
        HttpResponse response;
        try {
            response = httpClient.execute(httpGet);
        } catch (IOException e) {
            log.warn("An exception occurred while fetching the stream", e);
            return null;
        }

        if (response.getStatusLine().getStatusCode() != 200) {
            log.warn("Could not fetch stream. Status line: "+ response.getStatusLine());
            return null;
        }

        int metadataOffset = retrieveMetadataOffset(response);

        if (metadataOffset == 0) {
            log.info("Could not find metadata for url:"+ streamUrl);
            return null;
        }

        List<Metadata> metadata = extractMetadata(response, metadataOffset);
        if (metadata == null || metadata.isEmpty()) {
            return null;
        }
        return extractSongInfo(metadata);
    }
    private List<Metadata> extractMetadata(HttpResponse response, int metadataOffset) {
        String metadataStr;
        try(InputStream stream = response.getEntity().getContent()) {
            if (stream.skip(metadataOffset) != metadataOffset) {
                log.warn("Something went wrong while skipping to metadata offset");
                return null;
            }
            int metaDataLength = stream.read() * 16;
            metadataStr = getMetadataStr(stream, metaDataLength);
            if (metadataStr == null) {
                return null;
            }
        } catch (IOException e) {
            log.warn("Something went wrong while reading the stream", e);
            return null;
        } //Hangs here
        //rest of the method
    }

I noticed that the stream returned by response.getEntity().getContent() is of type EofSensorInputStream so I'm wondering if it's waiting for a EOF character which is never coming.

Example of stream for which the code works properly and the stream closes properly: https://icecast.omroep.nl/radio2-bb-mp3, http://live-mp3-128.kexp.org Example of stream for which the code doesn't work properly as the stream never closes and hangs forever: https://kexp-mp3-128.streamguys1.com/kexp128.mp3

Answer 1

This problem occurs because calling close() on the content stream will try to consume the remaining content. This is currently not explicitly mentioned in the documentation (see also HTTPCORE-656), but is mentioned in the tutorials:

The difference between closing the content stream and closing the response is that the former will attempt to keep the underlying connection alive by consuming the entity content while the latter immediately shuts down and discards the connection.
[...]
There can be situations, however, when only a small portion of the entire response content needs to be retrieved and the performance penalty for consuming the remaining content and making the connection reusable is too high, in which case one can terminate the content stream by closing the response.

Therefore in your case it seems appropriate to not close the InputStream returned by getContent() but only close the HttpResponse (which you are apparently not doing yet).