What kind of data is stored in ksqlDB? Only metadata about KStreams and KTables?
From what I understand I can create KStreams and KTables and operations between these using either the Java API or the KSQL language. But what is ksqlDB Server actually used for besides metadata about these objects when I create them using KSQL client?
I also assume that ksqlDB actually runs the streams processors so it is also an execution engine. Does ist scale automatically? Can there be multiple instances of ksqlDB server component that communicate with each other? Is it intended for scenarios that need massive scaling, or is it just some syntactic sugar suitable for people who don't like to write Java code?
EDITED
It is explained in this video: https://www.youtube.com/embed/f3wV8W_zjwE
It does not scale automatically, but you can manually deploy multiple instances of ksqlDB server and make them join the same ksqlDB cluster identified by ksql.server.id
What kind of data is stored in ksqlDB?
Nothing is really stored "in" it. All the metadata of the query history and running processors is stored in Kafka topics and on-disk KTables
I also assume that ksqlDB actually runs the streams processors so it is also an execution engine. Does ist scale automatically? Can there be multiple instances of ksqlDB server component that communicate with each other? Is it intended for scenarios that need massive scaling, or is it just some syntactic sugar suitable for people who don't like to write Java code?
All Yes.