We are working on a solution that is using ADX. We have created some arm templates responsible for creating ADX Cluster, and then we have *.csl scripts used by Adx Azure DevOps Tasks to create tables, functions, policies, etc with the use of the tip:
Right now we have a lot of data in our adx, and we wanted to add materialized view which is using backfill=true
property in order to make sure that the existing data will be included in this view.
Let's say that this is the example query that we want to use:
.create-or-alter materialized-view with (backfill=true) ArgMax on table T
{
T | summarize arg_max(Timestamp, *) by User
}
when we ran this command for the very first time it will create the materialized view, and everything is ok, but if we will ran it again (continuous deployment of IaC) it will return an error:
Unsupported propery in materialized view alter command. Supported properties: DimensionTables,Lookback,Folder,DocString,AutoUpdateSchema.
So it seems that idempotency cannot be maintained within IaC with materialized views.
.create-or-alter materialized-view documentation says that there are some limitations:
The backfill property isn't supported if the materialized view already exists. If the materialized view already exists, it cannot be backfilled.
Our case is that we have backfilled
materialized view, and we cannot run the command that in the end is not changing anything.
We are wondering if that's correct behavior and will appreciate any thoughts or ideas how to achieve idempotency with ADX materialized views and IaC
Would using create ifnotexists
work for you?
.create ifnotexists materialized-view with (backfill=true) ArgMax on table T
{
T | summarize arg_max(Timestamp, *) by User
}