I am constructing a reader using response.body.pipeThrough(new TextDecoderStream()).getReader().
I then loop through the read output and parse what is supposed to be valid JSON.
The complete implementation can be seen here:
It is supposed to produce output similar to:
data: {"id":"chatcmpl-7YkvP6XEACyebJa9awrHeEENrAQNc","object":"chat.completion.chunk","created":1688517887,"model":"gpt-3.5-turbo-0613","choices":[{"index":0,"delta":{"role":"assistant","content":null,"function_call":{"name":"print_tags","arguments":""}},"finish_reason":null}]}
data: {"id":"chatcmpl-7YkvP6XEACyebJa9awrHeEENrAQNc","object":"chat.completion.chunk","created":1688517887,"model":"gpt-3.5-turbo-0613","choices":[{"index":0,"delta":{"function_call":{"arguments":"{\n"}},"finish_reason":null}]}
data: {"id":"chatcmpl-7YkvP6XEACyebJa9awrHeEENrAQNc","object":"chat.completion.chunk","created":168851788
The above is what I get from reader.read() here.
However, notice that the last line got truncated.
I assume this is happening because of some sort of misconfiguration of TextDecoderStream or the reader.
What needs to be done to ensure that I am receiving the complete chunk?
Adding openai-api tag because this might be a bug in their/specific to their API.
Each chunk you receive from a TextDecoderStream is supposed to be valid UTF-8 text not valid JSON.
To be 100% sure you have all the data, you need to wait for it all to be downloaded, at which point you may just use the Response#json() method instead.
Another possible solution would be to use a streaming JSON parser and feed it with each chunk. You can check this Q/A.