Search code examples
partitioningazure-data-explorerdatabase-partitioningadx

Partitioning of data in adx table


I was going through the microsoft docs of partition policy here. There it is written that By default, extents are partitioned by time of creation (ingestion), and in most cases there's no need to set a data partitioning policy. Whereas in the later part of document, I found the line - By default, a table's data partitioning policy is null, in which case data in the table won't be repartitioned after it's ingested.

What does these 2 statements mean?

I found out that there is no partitioning policy present in newly created table.

In my use case I want partitioning of data based on the ingestionTime.

How should I create the partition policy if that is not present by default?

  • partition by ingestion/creation time?
  • partition by timestamp(dateTime) column?

I want uniform range partitioning as the hash partition does not make sense here.


Solution

  • By default, extents are partitioned by time of creation (ingestion), and in most cases there's no need to set a data partitioning policy

    this means that when a data partitioning policy isn't defined (is null), data still gets partitioned (into data shards) by time of arrival (i.e. ingestion time). data does not get re-partitioned after it is ingested.

    By default, a table's data partitioning policy is null, in which case data in the table won't be repartitioned after it's ingested

    this means the same as the above.

    In my use case I want partitioning of data based on the ingestion time.

    you don't need to define a partitioning policy. that happens by default (as explained above).