Search code examples
vb.netperformancefilestreambinarywriter

How to operate with binary files faster?


What I'm trying to do:

  1. Open two binary files, each 64 MB
  2. Take first half of each byte of each file and combine those halves in 1 bytes same with second half, for example: first byte in first file is 0x51, in second file its 0xA2, so I need write two bytes in third file which are 0x5A and 0x12, same for whole bytes, therefore final length in third file will be 128 MB.
Private Sub Button1_Click(sender As Object, e As EventArgs) Handles 
       Button1.Click
      Try                                                                             
    ' choosing first file
        OpenFileDialog1.FileName = "First file"
        OpenFileDialog1.Title = "Choose the Address.bin file"
        OpenFileDialog1.Filter = "bin files (*.bin)|*.bin|All files 
         (*.*)|*.*"
        If OpenFileDialog1.ShowDialog() = Windows.Forms.DialogResult.OK 
         Then
            Label1.Text = 
         System.IO.Path.GetFullPath(OpenFileDialog1.FileName)

        Else
            Exit Sub
        End If
    Catch ex As Exception
    End Try
    Try                                                                               ' choosing first file
        OpenFileDialog1.FileName = "Second FIle"
        OpenFileDialog1.Title = "Choose the Flash.bin file"
        OpenFileDialog1.Filter = "bin files (*.bin)|*.bin|All files (*.*)|*.*"
        If OpenFileDialog1.ShowDialog() = Windows.Forms.DialogResult.OK Then
            Label2.Text = System.IO.Path.GetFullPath(OpenFileDialog1.FileName)

        Else
            Exit Sub
        End If
    Catch ex As Exception
    End Try
    Dim firstFileByte(1) As Byte
    Dim SecondFileByte(1) As Byte
    Dim Result As String
    Dim Result2 As String
    Dim Final(1) As Byte

    For i = 0 To FileLen(Label1.Text) - 1



        Using FirstFile As New FileStream(Label1.Text, FileMode.Open) 'save

            'FIRST DIGIT********************************************************************************************
            FirstFile.Seek(i, SeekOrigin.Begin)
            FirstFile.Read(firstFileByte, 0, 1)

            'TextBox1.Text = final(0).ToString("X")
            Using SecFile As New FileStream(Label2.Text, FileMode.Open) 'save
                SecFile.Seek(i, SeekOrigin.Begin)
                SecFile.Read(SecondFileByte, 0, 1)
            End Using
            Result = firstFileByte(0).ToString("X2").Substring(0, 1) & SecondFileByte(0).ToString("X2").Substring(0, 1)   ' comobining frist half of the first file and second file 
            Result2 = firstFileByte(0).ToString("X2").Substring(1, 1) & SecondFileByte(0).ToString("X2").Substring(1, 1) ' comobining second half of the first file and second file  

        End Using
        Using vFs As New FileStream(Environment.GetFolderPath(Environment.SpecialFolder.Desktop) & "\Result.bin", FileMode.Append) '  save
            TextBox1.Text = Result2
            'Dim FileLenVar As UInt32 = FileLen(Environment.GetFolderPath(Environment.SpecialFolder.Desktop) & "\Result.bin") - 1

            Final(0) = Convert.ToByte(Result, 16)    'converting result to the byte
            Final(1) = Convert.ToByte(Result2, 16)

            vFs.Write(Final, 0, 1)

            vFs.Write(Final, 1, 1)

        End Using
    Next
End Sub

It works, but takes a lot of time: it writes 1 MB in 1 minute. How can I optimize it?


Solution

  • The files are small enough to load into RAM for processing. Processing the data in RAM can minimise the disk I/O needed, the latter very often being the slowest part of a program, especially if it is done in very small pieces like individual bytes. As also noted by Ben Voigt, string operations are somewhat slower than numeric operations.

    Here's a simple demonstration of what could be done:

    Imports System.IO
    
    Module Module1
        Dim f1 As String = "C:\temp\A.bin"
        Dim f2 As String = "C:\temp\B.bin"
        Dim outFile As String = "C:\temp\C.bin"
    
        Sub CombineFiles()
            Dim a1 = File.ReadAllBytes(f1)
            Dim a2 = File.ReadAllBytes(f2)
            Dim c(a1.Length + a2.Length - 1) As Byte ' c for combined
    
            Dim highBits As Byte = &HF0
            Dim lowBits As Byte = &HF
    
            For i = 0 To c.Length - 1 Step 2
                c(i) = a1(i \ 2) And highBits Or a2(i \ 2) >> 4
                c(i + 1) = a1(i \ 2) << 4 Or a2(i \ 2) And lowBits
            Next
    
            File.WriteAllBytes(outFile, c)
    
        End Sub
    
        Sub CreateTestFiles()
            'TODO: be more creative with the generated data.
    
            Dim nBytes = 64
            Dim a(nBytes - 1) As Byte
    
            For i = 0 To nBytes - 1
                a(i) = &H12
            Next
    
            File.WriteAllBytes(f1, a)
    
            For i = 0 To nBytes - 1
                a(i) = &HAB
            Next
    
            File.WriteAllBytes(f2, a)
    
        End Sub
    
        Sub Main()
            'CreateTestFiles()
            CombineFiles()
    
        End Sub
    
    End Module
    

    Of course, you'd check that the input files were of equal length, and check for any other possible problems ;)