I have a QuestDB instance where I created a table partitioned by month. I imported some data and I can see all the partitions for the previous months, but when I now ingest live/real-time data, it is creating multiple partition within the current month.
Why is this happening? Will this have any impact on query performance?
QuestDB stores data inside each partition sorted by increasing timestamp. If for any reason (latency, business logic, disaster recovery, data reingestion, upserts...) you ingest data out of order, QuestDB needs to make sure data is still stored by increasing timestamp.
To mitigate write amplification due to having to work over the same file over and over, QuestDB uses partition splits, and stores data for the latest partition (the live partition) in small chunks, so whenever data is coming out of order, the impact of rewriting is smaller.
You should see once the partition is rolled over the next month the data will be compacted.
Bottomline: The behaviour is by design and query performance will not be degraded, as this is done specifically to optimize the disk usage.