Search code examples
azure-data-explorer

Is it possible to update rows in Azure Data Explorer (Kusto)

  1. Is it possible to update rows in Azure Data Explorer?

  2. Is it a good choice for storing large inventory (100M items) that are updated frequently or it is mostly optimized to append-only data scenarios? My repository is updated around the clock and I need the changes visible immediately (near-real-time).

  3. Assuming I need full-text search, faceting, and basic items queries/lookups and near real-time updates (which flows in around the clock). how does it compare to ElasticSearch? Assuming I consume both as managed service. I'm mostly concerned about the updates part.

Solution

    1. Updating single records isn't possible. You do have the option to update data in batches, by tagging it at ingestion time, and using shard-level commands (such as .replace extents) when you need to perform the 'update'.

    2. While it's perhaps not the "natural" choice, you could still theoretically achieve that, by ingesting a new record for each update, and using arg_max() to always get the latest version per item.

      • There's an upcoming feature, Materialized Views, which can (until it's available, you can orchestrate that materialization periodically using any scheduler that ingests from a query)

    3. A full and meaningful comparison isn't a one-liner (there are too many considerations to list), and in most cases - the result of such comparison would be very specific to your use case, budget, personal preference, performance requirements, etc. That said, all of the features you listed are supported by ADX/Kusto.