Search code examples
azure-data-explorer

Where is Update Policy failure logged


I had a table with an update policy applied like so:

.create table Foo (
    data: dynamic
)

.create function ParseFoo () {
    Foo
    | project 
        a = tostring(data.a),
        b = tostring(data.b)
}

.create table Bar (
    a: string,
    b: string
)

.alter table Bar policy update
```
[{
    "IsEnabled": true,
    "Source": "Foo",
    "Query": "ParseFoo",
    "IsTransactional": false,
    "PropagateIngestionProperties": false
}]
```

Someone1 changed the definition of ParseFoo to extract another column:

.alter function ParseFoo () {
    Foo
    | project 
        a = tostring(data.a),
        b = tostring(data.b),
        c = tostring(data.c)
}

The difference in schema prevented the update policy from applying and data ingestion was stopped. I was able to figure out the mismatch and correct it, but I would like to proactively monitor for this in future.

From a very cursory glance, I don't see any errors related to the failed ingestion to this table logged in any of the places I've thought to check so far


1. (me)


Solution

  • From the link the this Q&A: https://learn.microsoft.com/en-us/azure/kusto/management/updatepolicy#failures

    .show ingestion failures 
    | where Table == 'Bar'
    | project-reorder Details
    

    The .show ingestion failures command will reveal messages like the following.

    Failed to invoke update policy. Target Table = 'Bar', Query = 'let Foo = 
    __table("Foo", 'All', 'AllButRowStore') 
    | where extent_id() in (guid(29f13c11-e6cf-472a-a1cd-91af4cfb2c44));
    ParseFoo': Query schema does not match table schema