Download is failing in .Net

Question

Hi I am having issues in this code:

// Function will return the number of bytes processed
// to the caller. Initialize to 0 here.
int bytesProcessed = 0;

// Assign values to these objects here so that they can
// be referenced in the finally block
Stream remoteStream = null;
Stream localStream = null;
WebResponse response = null;

// Use a try/catch/finally block as both the WebRequest and Stream
// classes throw exceptions upon error
try
{
    // Create a request for the specified remote file name
    WebRequest request = WebRequest.Create(remoteFilename);
    request.Method = "GET";
    string credentials = Convert.ToBase64String(Encoding.ASCII.GetBytes(uName + ":" + pwd));
    request.Headers[HttpRequestHeader.Authorization] = "Basic " + credentials;

    if (request != null)
    {
        // Send the request to the server and retrieve the
        // WebResponse object 
        response = request.GetResponse();
        if (response != null)
        {
            // Once the WebResponse object has been retrieved,
            // get the stream object associated with the response's data
            remoteStream = response.GetResponseStream();

            // Create the local file
            localStream = File.Create(localFilename);

            // Allocate a 1k buffer
            byte[] buffer = new byte[1024];
            int bytesRead;
            long totalBytesToProcess = response.ContentLength;

            // Simple do/while loop to read from stream until
            // no bytes are returned
            do
            {
                // Read data (up to 1k) from the stream
                bytesRead = remoteStream.Read(buffer, 0, buffer.Length);
                // Write the data to the local file
                localStream.Write(buffer, 0, bytesRead);

                // Increment total bytes processed
                bytesProcessed += bytesRead;
                log(resourcesPath + "/BytesRecieved.txt", bytesProcessed.ToString()+"/"+ totalBytesToProcess.ToString(), false);
            } while (bytesRead > 0);
        }
    }
}
catch (Exception ex)
{
    Response.Write(ex);
   // log(resourcesPath +"/Logs.txt",);
}
finally
{
    // Close the response and streams objects here 
    // to make sure they're closed even if an exception
    // is thrown at some point
    if (response != null) response.Close();
    if (remoteStream != null) remoteStream.Close();
    if (localStream != null) localStream.Close();
}

// Return total bytes processed to caller.
return bytesProcessed;

This was able to download small files ranging up to 200 mb of file unfortunately it's failing when the file size soar high like up to more than 1gb. I have tried downloadfileAsyc of web client but it's failing too. is there any other way to handle large file for this matter ?

Answer 1

Allocate buffer size bigger than expected file size .

byte[] byteBuffer = new byte[65536];

so that , if the file is 1GiB in size, you allocate a 1 GiB buffer, and then you try to fill the whole buffer in one call. This filling may return fewer bytes but you've still allocated the whole buffer. Note that the maximum length of a single array in .NET is a 32-bit number which means that even if you recompile your program for 64bit and actually have enough memory available.

For your reference visit this link :

How to change this code to download file bigger than 2GB?