how to avoid high cpu usage when reading large files

Question

My project loads hugh list of string into the listview, using BackgroundThread,it did very fast performance but takes nearly 100% cpu is there any workaround to avoid high cpu usage without reducing its performance? I use netframework 3.5

The code:

        void readfile() {

        deg_readfile read = new deg_readfile(readfileasync);
        read.BeginInvoke(new AsyncCallback(callback),null);
    }

    void readfileasync(){
        string[] strs = File.ReadAllLines("hughlist.txt");
        List<ListViewItem> _itemlst = new List<ListViewItem>();
        ListViewItem _item;
        int x = 0;
        //problem start here
        foreach (string str in strs) {
            _item = new ListViewItem(str);

            for (int i = 0; i < 6; i++) {
                _item.SubItems.Add("");              
            }              
            _itemlst.Add(_item);
            x++;
            _item = null;
        }
    }

    void callback(IAsyncResult ae) {
        Console.WriteLine("finished");

    }

    delegate void deg_readfile();

    static void Main(string[] args)
    {
        Program prog = new Program();

        Thread t=new Thread(new ThreadStart(prog.readfile))
        {
            IsBackground = true,
            Priority = ThreadPriority.Lowest

        };
        t.SetApartmentState(ApartmentState.MTA);
        t.Start();
        Console.Read();

    }

Thanks in advance.

update:

System configuration :

Processor : Intel(R) Core(TM) i3-2328M CPU @ 2.20GHZ 2.11 GHZ Installed memory : 1.00 GB System type: 32-bit operating system

The file has 35,939 KB size with 1 millions lines

Answer 1

That's not asynchronous: you're just using a different thread to call invoke, and the passed delegate will be run n the main (GUI) thread synchronously (because controls are tied to the thread that created them: the main thread).

So the worker thread does two things:

• Create thread
• Call operation on a threadpool thread

and it is done.

The threadpool thread does all the work, including reading the file synchronously.²

However, it is unlikely¹ that a single file IO will take significant time; and then after the first run it will be cached in memory so reducing the IO to time to dispatch a kernel call.

But performing things asychronously will not reduce the amount of CPU resource needed (it is possible – setting up data structures to keep track of things – it will take more): your CPU will be involved in processing the content of the file.

You might be able to optimise that, but you'll want to start with profiler data to understand where the CPU time is being spent.

---

¹ If the file exists on a network share with a high latency or low bandwidth connection maybe it might be different.

² Strictly speaking it is poor form to perform blocking operations on thread pool threads (they are not available for other activities) but unlikely to be an issue for file IO. (See previous footnote.)