Search code examples
sqlpostgresqlout-of-memoryheap-memoryquarkus

Quarkus Reactive PG Client OutOfMemory Exception

I have a question regarding Quarkus and the quarkus-reactive-pg-client. I implemented a Quarkus backend service which offers REST endpoints with filter possibilities to requesting data from a PostgreSQL database. It's running as Docker build as legacy-jar currently. Current implementation has the problem, that if somebody only sets a few filters and the query result is huge, that the quarkus application raises an OutOfMemory Heap exception.

Here a few questions raising from my side:

  1. How can I enlarge the heap size settings? Can I just add the JAVA_OPTS parameter when starting the Docker container? e.g. -e JAVA_OPTS='-Xmx16g'

  2. Is there a way to limit the overall query result size? So I don't mean to set a "LIMIT" in the SQL query itself, instead something like, when the query get's executed, and the current size of the query result is bigger than 10GB, end the query and only return back the available 10GB result? Or at least cancel the query and return a response to the client, that query size would be to large?

  3. Or is there at least a possibility to safely handle OutOfMemory exception anyhow in the Quarkus application itself, that I just could return a response to client, that query size would be to large?

Thank you!

Solution

  • How can I enlarge the heap size settings?

    It depends on how your container image is built. If the command ran by the container engine takes into account the JAVA_OPTS environment variable then yes, you can use it to set the maximum heap size.

    Is there a way to limit the overall query result size?

    If you don't want to load all the results at once in memory, you can use the cursors / streaming API of the Reactive Pg Client. With the Mutiny bindings, the RowStream type has a toMulti method that can be helpful.

    It's not clear what the use case is but, generally speaking, it's preferable to limit the query results or use pagination.

    Or is there at least a possibility to safely handle OutOfMemory exception

    There is no safe way to handle an OOM error, because you don't know where it's going to be thrown. The right thing to do is to implement the application in order to avoid it.