Search code examples
vb.netftpzipunziplarge-files

FTP download files larger than 2GB (VB.net)


Missing last few bytes and file gets corrupted - bounty

I now added a bounty to solve this problem. I changed the integer types to int64 which seem to have solved part of the problem, but now when ever it finishes the download it sometimes misses the last 1-5 bytes, which in return corrupts the file, so it can't be unzipped. Is there another way of closing the stream so it ensures the files are fully downloaded, and avoid getting corrupted? I've since tried this simple code, but same problem happens.

Imports System.ComponentModel
Imports System.Net

Public Class Form1
    Private Sub Form1_Load(sender As Object, e As EventArgs) Handles MyBase.Load
        Control.CheckForIllegalCrossThreadCalls = False
    End Sub
    Dim WithEvents WC As New WebClient
    Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
        WC.DownloadFileAsync(New Uri("ftp://dmr-ftp-user:[email protected]/ESStatistikListeModtag/ESStatistikListeModtag-20160327-094743.zip"), "C:\XML\ESStatistikListeModtag-20160327-094743.zip.zip")
    End Sub
    Private Sub WC_DownloadProgressChanged(ByVal sender As Object, ByVal e As DownloadProgressChangedEventArgs) Handles WC.DownloadProgressChanged
        ProgressBar1.Value = e.ProgressPercentage
        If e.ProgressPercentage = 100 Then
            MsgBox("File download - 100%") 'This message box does trigger once the download is complete, but file is still corrupted.
        End If
    End Sub

    Private Sub WC_DownloadFileCompleted(sender As Object, e As AsyncCompletedEventArgs) Handles WC.DownloadFileCompleted
        MsgBox("Complete") ' This message box doesn't trigger!
    End Sub
End Class

OLD QUESTION:

I'm trying to download a zip file from an FTP server with my vb.net application. My current source code is posted below. This works fine for smaller files, but when I exceed a limit of 2GB I get the following exception:

"Arithmetic operation resulted in an overflow"

It's a file with the size of about 2.5 GB and increasing slightly each weak (about 20 MB), so I need a solution which can handle large files, hopefully with no limit. Eventually I would like to unzip the file with the program too, so if you have any ideas for doing this, you can post this as well. Thanks!

 Private Sub Download(ByVal filePath As String, ByVal fileName As String)
        FTPSettings.IP = "0.0.0.0"
        FTPSettings.UserID = "ftp-user"
        FTPSettings.Password = "ftp-pass"
        Dim reqFTP As FtpWebRequest = Nothing
        Dim ftpStream As Stream = Nothing
        Try
            Dim outputStream As New FileStream(filePath + "\" + fileName, FileMode.Create)
            reqFTP = DirectCast(FtpWebRequest.Create(New Uri("ftp://" + FTPSettings.IP + "/" + fileName)), FtpWebRequest)
            reqFTP.Method = WebRequestMethods.Ftp.DownloadFile
            reqFTP.UseBinary = True
            reqFTP.Credentials = New NetworkCredential(FTPSettings.UserID, FTPSettings.Password)
            Dim response As FtpWebResponse = DirectCast(reqFTP.GetResponse(), FtpWebResponse)
            ftpStream = response.GetResponseStream()
            Dim cl As Long = response.ContentLength
            Dim bufferSize As Integer = 2048
            Dim readCount As Int64
            Dim buffer As Byte() = New Byte(bufferSize - 1) {}
            Dim size As Int64

            readCount = ftpStream.Read(buffer, 0, bufferSize)
            While readCount > 0
                outputStream.Write(buffer, 0, readCount)
                readCount = ftpStream.Read(buffer, 0, bufferSize)

                If readCount = bufferSize Then
                    size += readCount
                    Label1.Text = size
                    Label1.Refresh()
                End If

              End While

            ftpStream.Close()
            outputStream.Close()
            response.Close()
        Catch ex As Exception
            MsgBox(ex.Message)
            If ftpStream IsNot Nothing Then
                ftpStream.Close()
                ftpStream.Dispose()
            End If
            Throw New Exception(ex.Message.ToString())
        End Try
    End Sub
    Public NotInheritable Class FTPSettings
        Private Sub New()
        End Sub
        Public Shared Property IP() As String
            Get
                Return m_IP
            End Get
            Set(ByVal value As String)
                m_IP = value
            End Set
        End Property
        Private Shared m_IP As String
        Public Shared Property UserID() As String
            Get
                Return m_UserID
            End Get
            Set(ByVal value As String)
                m_UserID = value
            End Set
        End Property
        Private Shared m_UserID As String
        Public Shared Property Password() As String
            Get
                Return m_Password
            End Get
            Set(ByVal value As String)
                m_Password = value
            End Set
        End Property
        Private Shared m_Password As String
    End Class
End Class

Solution

  • I've had similar problems with WebClient before, specially if using it with the WithEvents statement. See if re-writing your code like this solves the problem:

    Imports System.ComponentModel
    Imports System.Net
    
    Public Class Form1
        Private wc As New WebClient()
    
        Private Sub Form1_Load(sender As Object, e As EventArgs) Handles MyBase.Load
            wc = New WebClient()
    
            AddHandler wc.DownloadProgressChanged, Sub(s As Object, ByVal e As DownloadProgressChangedEventArgs)
                                                    Me.Invoke(New MethodInvoker(Sub() ProgressBar1.Value = e.ProgressPercentage))
                                                End Sub
    
            AddHandler wc.DownloadFileCompleted, Sub(s As Object, e As ComponentModel.AsyncCompletedEventArgs)
                                                    MsgBox("Complete")
                                                End Sub
        End Sub
    
        Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
            wc.DownloadFileAsync(New Uri("ftp://dmr-ftp-user:[email protected]/ESStatistikListeModtag/ESStatistikListeModtag-20160327-094743.zip"), "C:\XML\ESStatistikListeModtag-20160327-094743.zip.zip")
        End Sub
    End Class