I have a Druid Cluster running a router pod and a broker pod. It looks like it is taking too long to process a query, and I get a timeout at the router:
java.util.concurrent.TimeoutException: Total timeout 300000 ms elapsed
32881at org.eclipse.jetty.client.HttpConnection$RequestTimeouts.onExpired(HttpConnection.java:334) ~[jetty-client-9.4.48.v20220622.jar:9.4.48.v20220622]
32882at org.eclipse.jetty.client.HttpConnection$RequestTimeouts.onExpired(HttpConnection.java:314) ~[jetty-client-9.4.48.v20220622.jar:9.4.48.v20220622]
32883at org.eclipse.jetty.io.CyclicTimeouts.onTimeoutExpired(CyclicTimeouts.java:110) ~[jetty-io-9.4.48.v20220622.jar:9.4.48.v20220622]
32884at org.eclipse.jetty.io.CyclicTimeouts.access$100(CyclicTimeouts.java:50) ~[jetty-io-9.4.48.v20220622.jar:9.4.48.v20220622]
32885at org.eclipse.jetty.io.CyclicTimeouts$Timeouts.onTimeoutExpired(CyclicTimeouts.java:196) ~[jetty-io-9.4.48.v20220622.jar:9.4.48.v20220622]
32886at org.eclipse.jetty.io.CyclicTimeout$Wakeup.run(CyclicTimeout.java:298) ~[jetty-io-9.4.48.v20220622.jar:9.4.48.v20220622]
32887at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?]
32888at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
32889at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) ~[?:?]
32890at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
32891at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
32892at java.lang.Thread.run(Thread.java:829) ~[?:?]
According to the documentation (https://druid.apache.org/docs/latest/querying/troubleshooting/) i have to increase druid.server.http.maxIdleTime , but that did not change anything (even the timeout duration) when setting it to PT6H. Is there any other property that i might have missed?
You have to set "druid.router.http.readTimeout". This sets the client setting of the druid router which starts the call to the brokers/historicals.