I currently have a service which runs several queries to build data in my database. It then uses the Logstash's JDBC input plugin to send data to Elastic Search. The problem is that Logstash takes quite a lot of time bootstrapping (sometimes more than the time required to send data to Elastic Search).
I have decided to explore having Logstash running as a service to erase this bootstrapping time, however there seem to be no way to have it running as a service and accepting "requests" to run the JDBC statement.
The closest thing seems to use a TCP/HTTP/Websocket plugin to receive data from my service and then send it to Elastic Search.
Would it make sense to try and develop something like this or would it be better to communicate directly to Elastic Search with my service?
I hope my question is clear enough, thanks.
Answering my own question for future reference of anyone bumping in the same issue.
After reading tons of forum discussions I ended up concluding that:
schedule option, to do this periodically or by starting/stopping the service manually, which brings us to