Search code examples
javaspring-bootvue.jsstreaming

Java spring boot server sending response all at once instead of streaming depending on how it's run

I am trying to stream a response from a java server. The response is sent to a Vue client. When the server is run in VS Code, it works beautifully. However, when I run the server using java -server -jar xxx.jar, it seems to be streaming on the server side logs but the client receives it all as one batch. It seems to be a configuration issue with the Java server or a problem with the .jar. It's not a code issue as the exact same code is running. Please enlighten me.

The server log shows the response streaming like below.

OpenAiController - =====>> Thread Message Delta:  approved
OpenAiController - =====>> Thread Message Delta:  solely
OpenAiController - =====>> Thread Message Delta:  for
OpenAiController - =====>> Thread Message Delta:  the
OpenAiController - =====>> Thread Message Delta:  treatment
OpenAiController - =====>> Thread Message Delta:  of
OpenAiController - =====>> Thread Message Delta:  fa
OpenAiController - =====>> Thread Message Delta: ec
OpenAiController - =====>> Thread Message Delta: al
OpenAiController - =====>> Thread Message Delta: oma
OpenAiController - =====>> Thread Message Delta: ,
OpenAiController - =====>> Thread Message Delta:  several
OpenAiController - =====>> Thread Message Delta:  medications
OpenAiController - =====>> Thread Message Delta:  are
OpenAiController - =====>> Thread Message Delta:  commonly
OpenAiController - =====>> Thread Message Delta:  used
OpenAiController - =====>> Thread Message Delta:  to
OpenAiController - =====>> Thread Message Delta:  manage
OpenAiController - =====>> Thread Message Delta:  the
OpenAiController - =====>> Thread Message Delta:  underlying
OpenAiController - =====>> Thread Message Delta:  constipation
OpenAiController - =====>> Thread Message Delta:  that
OpenAiController - =====>> Thread Message Delta:  can
OpenAiController - =====>> Thread Message Delta:  lead

However on the client it gets the entire response all at once after everything is finished.

When it works, I am running the server in VS Code using the following command:

& 'C:\Program Files\Java\jdk-21\bin\java.exe' '-agentlib:jdwp=transport=dt_socket,server=n,suspend=y,address=localhost:62203' '@C:\Users\xxx\AppData\Local\Temp\34\cp_396rcp08t5lealrto29q2ek8w.argfile' 'com.xxx.xxx.xxx'

Where it is not working I am running it as:

nohup java -server -Xms4096m -Xmx4096m -Dfile.encoding=UTF-8 -Dsun.jnu.encoding=UTF-8 -jar /mnt/xxx.jar >/dev/null 2>&1 &

Or just as

java -server -jar /mnt/xxx.jar
Solution

  • It turned out the reason it was not working was because the output buffer on the Java server running on the Linux machine was too big. I fixed the problem by adding random characters to the message so that it would fill up the buffer and flush immediately. Not sure how to change the buffer size though; couldn't find where the configuration is.