Azure Cognitive Search - Doesn't track changes in data
I have an Azure Cognitive Search that scans every night an Azure Storage Blob Container. Within the container is one single CSV file. This file has like 30 columns and maybe 20k rows. The values of some of those rows and columns change. E.g. column a = price, column b = 19,99 USD.
I would expect that every change of information is recognized by the Cognitive Search. But it seems that if a value was tracked once, even if the value changes it won't be reflected in the index. So the index always delivers "old" data. How can I configure an index so it reflects every change?
I run the indexer on Monday, the document says the price for X is 19,99 USD. A day later the CSV states that the price changed to 15,99 USD. But despite that the index runs every night it always stays on 19,99 - not reflecting the change.
Make sure that you have configured your indexer according to Index CSV blobs ( specifically that it has the parsingMode set to delimitedText and the firstLineContainsHeaders property configured properly). If it is not, create a separate indexer and index to test with the suggested configuration and check when changes are made to the CSV and run the test indexer accordingly, make sure that this fixes your issue before making any changes to your current configuration.
Also, make sure that it has a schedule that runs when you need (if you need it to run every night, just make sure that is actually running in the indexer history and that you have no errors or warnings).
Since the indexer may fail and will pick up any changes from any transient errors until the next run, you may want to consider a schedule of more than once per day.