Hibernate Utils is creating the session factory along with Hikari configuration. Currently we are doing inside @Setup method of ParDo, but it opens way too many connections. So is there any good example to initialize connection pool per worker ?
If you are using @Setup
method inside DoFn
to create a database connection keep in mind that Apache Beam would create connection pool per worker instance thread. This might result in a lot of database connections depending on the number of workers that spin up.
You can use a Singleton class inside a ParDo to create your connection pool and control the number of connections per worker. You can take a look here for a sample implementation