An ItemsControl is inconsistent with its items source - Problem when using Dispatcher.Invoke()

Question

I'm writing a WPF application (MVVM pattern using MVVM Light Toolkit) to read and display a bunch of internal log files my company uses. The goal is to read from multiple files, extract content from each line, put them in a class object and add the said object to an ObservableCollection. I've set the ItemsSource of a DataGrid on my GUI to this list so it displays the data in neat rows and columns. I have a ProgressBar control in a second window, which during the file read and display process will update the progress.

Setup

Note that all these methods are stripped down to the essentials removing all the irrelevant code bits.

Load Button

When the user selects the directory which contains log files and clicks this button, the process begins. I open up the window that contains the ProgressBar at this point. I use a BackgroundWorker for this process.

public void LoadButtonClicked()
{
    _dialogService = new DialogService();
    BackgroundWorker worker = new BackgroundWorker
    {
        WorkerReportsProgress = true
    };
    worker.DoWork += ProcessFiles;
    worker.ProgressChanged += Worker_ProgressChanged;
    worker.RunWorkerAsync();
}

ProcessFiles() Method

This reads all files in the directory selected, and processes them one by one. Here, when launching the progress bar window, I'm using Dispatcher.Invoke().

private void ProcessFiles(object sender, DoWorkEventArgs e)
{
    LogLineList = new ObservableCollection<LogLine>();

    System.Windows.Application.Current.Dispatcher.Invoke(() =>
    {
        _dialogService.ShowProgressBarDialog();
    });

    var fileCount = 0;
    foreach (string file in FileList)
    {
        fileCount++;
        int currProgress = Convert.ToInt32(fileCount / (double)FileList.Length * 100);
        ProcessOneFile(file);
        (sender as BackgroundWorker).ReportProgress(currProgress);
    }
}

ProcessOneFile() Method

This, as the name suggests, reads one file, go through line-by-line, converts the content to my class objects and adds them to the list.

public void ProcessOneFile(string fileName)
{
    if (FileIO.OpenAndReadAllLinesInFile(fileName, out List<string> strLineList))
    {
        foreach (string line in strLineList)
        {
            if (CreateLogLine(line, out LogLine logLine))
            {
                if (logLine.IsRobotLog)
                {
                    LogLineList.Add(logLine);
                }
            }
        }
    }
}

So this works just fine, and displays my logs as I want them.

Problem

However, after displaying them, if I scroll my DataGrid, the GUI hangs and gives me the following exception.

System.InvalidOperationException: 'An ItemsControl is inconsistent with its items source. See the inner exception for more information.'

After reading about this on SO and with the help of Google I have figured out that this is because my LogLineList being inconsistent with the ItemsSource which results in a conflict.

Current Solution

I found out that if I put the line of code in ProcessOneFile where I add a class object to my list inside a second Dispatcher.Invoke() it solves my problem. Like so:

if (logLine.IsRobotLog)
{
    System.Windows.Application.Current.Dispatcher.Invoke(() =>
    {
        LogLineList.Add(logLine);
    });                                
}

Now this again works fine, but the problem is this terribly slows down the processing time. Whereas previously a log file with 10,000 lines took about 1s, now it's taking maybe 5-10 times as longer.

Am I doing something wrong, or is this to be expected? Is there a better way to handle this?

Answer 1

Well observable collection is not thread safe. So it works the second way because all work is being done on the UI thread via dispatcher.

You can use asynchronous operations to make this type of flow easier. By awaiting for the results and updating the collection\progress on the result, you will keep your UI responsive and code clean.

If you cant or don't want to use asynchronous operations, batch the updates to the collection and do the update on the UI thread.

Edit Something like this as an example

private async void Button_Click(object sender, RoutedEventArgs e)
{
    //dir contents
    var files = new string[4] { "file1", "file2", "file3", "file4" };
    //progress bar for each file
    Pg.Value = 0;
    Pg.Maximum = files.Length;
    foreach(var file in files)
    {                
        await ProcessOneFile(file, entries => 
        {
            foreach(var entry in entries)
            {
                LogEntries.Add(entry);
            }
        });
        Pg.Value++;
    }
}

public async Task ProcessOneFile(string fileName, Action<List<string>> onEntryBatch)
{
    //Get the lines
    var lines = await Task.Run(() => GetRandom());
    //the max amount of lines you want to update at once
    var batchBuffer = new List<string>(100);

    //Process lines
    foreach (string line in lines)
    {
        //Create the line
        if (CreateLogLine(line, out object logLine))
        {
            //do your check
            if (logLine != null)
            {
                //add
                batchBuffer.Add($"{fileName} -{logLine.ToString()}");
                //check if we need to flush
                if (batchBuffer.Count != batchBuffer.Capacity)
                    continue;
                //update\flush
                onEntryBatch(batchBuffer);
                //clear 
                batchBuffer.Clear();
            }
        }
    }

    //One last flush
    if(batchBuffer.Count > 0)
        onEntryBatch(batchBuffer);            
}