Proper way to copy a file line by line to another file?

Question

I need to open a file, read a line and if that line respect some conditions write it to another file, the line that I'm reading is a normal ASCII string representig HEX values and I need to paste it into a new file as HEX values and not as ASCII string.

What I have is this:

private void Button_Click(object sender, RoutedEventArgs e)
{
    byte[] arrayByte = { 0x00 };
    var linesToKeep = File.ReadLines(fileName).Where(l => l.Contains(":10"));
    foreach (string line in linesToKeep)
    {

        string partialA = line.Substring(9);
        string partialB = partialA.Remove(partialA.Length - 2);
        arrayByte = ToByteArray(partialB);
        using (var stream = new FileStream(fileName+"_gugggu", FileMode.OpenOrCreate))
        {
            FileInfo file = null;
            file = new FileInfo(fileName + "_gugggu");
            stream.Position = file.Length;
            stream.Write(arrayByte, 0, arrayByte.Length);
        }
    }
}
public static byte[] ToByteArray(String HexString)
{
    int NumberChars = HexString.Length;
    byte[] bytes = new byte[NumberChars / 2];
    for (int i = 0; i < NumberChars; i += 2)
    {
        bytes[i / 2] = Convert.ToByte(HexString.Substring(i, 2), 16);
    }
    return bytes;
}

This method is doing what I need but it takes ages to finish, the original files have roughly 70000 lines... Is there a better way to do that in order to increase speed?

Answer 1

As stated in the comments, you practically keep opening and closing the destination file. You do this for each line in the source file, this causes a huge and unnecessary overhead.

There's no need to close, re-open and re-set the stream position each time you read a line from the source file.

I created a file with 316240 lines. The code below reads, determines which lines to keep and then writes them to the destination file. Since I didn't have access to your source file, I uses some kind of string-to-byte conversion mock-up.

Result:

As you see it completes in 1/3 of a second.

Code:

private Stopwatch sw = new Stopwatch();
private FileInfo _sourceFile = new FileInfo(@"D:\source.txt");
private FileInfo _destinationFile = new FileInfo(@"D:\destination.hex");

private void ConvertFile()
{
    sw.Start();
    using (var streamReader = new StreamReader(_sourceFile.OpenRead()))
    {
        using (var streamWrite = _destinationFile.OpenWrite())
        {
            while (!streamReader.EndOfStream)
            {
                string line = streamReader.ReadLine();
                if (line.Contains(" x"))
                {
                    var array = ToByteArray(line);
                    streamWrite.Write(array, 0, array.Length);
                }
            }
        }
    }
    sw.Stop();
    MessageBox.Show("Done in " + sw.Elapsed);
}

private byte[] ToByteArray(string hexString)
{
    return hexString.ToList().ConvertAll(c => Convert.ToByte(c)).ToArray();
}

private void Button_Click(object sender, RoutedEventArgs e)
{
    ConvertFile();
}

HTH