Search code examples
c#memory-leaksoledbvisual-foxpro

VFPOledb Driver Possible Memory Leak?


I'm developing a program in C# which inserts around a 100,000 rows into a version 4 dbase file (*.dbf) using vfpoledb driver.

The method that return the connection string is as follows,

internal string GetDBaseConnectionString(string path)
{            
  return "Provider=vfpoledb;Data Source=" + path + ";Collating Sequence=general;";            
}

The code segment that does the insert is as follows,

        internal long Execute()
        {
            OleDbConnection con = null;
            OleDbCommand cmd = null;

            try
            {
                con = new OleDbConnection(AppSettings.Current.GetDBaseConnectionString(_dbfPath));
                con.Open();

                cmd = new OleDbCommand(_sql);
                cmd.Connection = con;                
                cmd.CommandTimeout = AppSettings.Current.DefaultCommandTimeOutMinutes;

                long rowIndex = 0;
                int countInBatch = 0;

                for (int i = 0; i < _reader.FieldCount; i++)
                    cmd.Parameters.Add(new OleDbParameter());                

                while (_reader.Read())
                {
                    for (int i = 0; i < cmd.Parameters.Count; i++)
                        cmd.Parameters[i].Value = _reader.GetValue(i);

                    cmd.ExecuteNonQuery();

                    rowIndex += 1;

                    if (_progressChangeRowCount > 0)
                    {
                        countInBatch += 1;

                        if (countInBatch >= _progressChangeRowCount)
                        {
                            countInBatch = 0;

                            ProgressChangedEventArgs args = new ProgressChangedEventArgs(rowIndex);
                            this.OnProgressChanged(args);
                        }
                    }
                }

                _reader.Close();

                con.Close();
                con.Dispose();
                cmd.Dispose();

                return rowIndex;
            }
            catch (Exception ex)
            {                
                if (con != null)
                {
                    con.Close();
                    con.Close();
                }

                if (cmd != null)
                    cmd.Dispose();

                if(_reader!= null)
                    _reader.Close();

                throw ex;
            }
        }

This segment is run in three threads concurrently. So data is inserted into three dbase files simultaneously from 3 SqlDataReaders.

My problem is that my program eats up around 50-100 MB per minute and it only increases until I close the program. Because of this System.OutOfMemoryExceptions are raised in the program and the OS soon shuts it down. I can see the Page File Usage in Task Manager go from 540 MB to 2.2 GB.

I have narrowed it down to the line cmd.ExecuteNonQuery(); If I comment out this line, the program executes with only about 1 or 2 MB of increase in memory.

Hence

  1. Can this be due to a memory leak in the VFPOledb driver? I'm using the latest version 9
  2. If so what can I do to cope with it? (Wrapping this as a seperate process so the OS cleans up any memory leaks on exit sounds tempting but should be a last resort)
  3. Do you see or know of any other fault which might be causing this?
  4. Changing the driver may help but the Jet driver is slow painstakingly slow. Is there any other option for dbase bulk inserts? I have also asked this question here Efficient way to bulk insert into Dbase (.dbf) files

Thanks in advance.


Solution

  • After many tries I wrapped the piece of code in a separate process so that the OS cleaned up after it exits. This was the best solution I could find.