Search code examples
c#httpstreamhttpwebrequesthttpwebresponse

The number of bytes return from HttpWebResponse's Stream doesn't equal to ContentLength


I'm creating download manager by using HttpWebRequest, HttpWebResponse and Stream. First, I get ContentLength from HttpWebResponse before start read content of file from stream and use while loop to stop reading, when it completed(Stream.Read return 0, which mean end of stream), I noticed that I didn't receive all of bytes(I compared to ContentLength)

This is some of code that I'm using to read bytes from stream, Calculate content length to megabytes, number of bytes that I received(in megabyte) and percent of bytes that I received.

                int maxReadSize = 102400;
                int readByte = 0;
                long downloadedSize = 0;
                int roundCount = 0;
                int byteCalRound = 0;
                byte[] buffer = new byte[maxReadSize];
                _currentFile.Status = DownloadStatus.Downloading;
                DateTime lastUpdate = DateTime.Now;

                do
                {
                    readByte = await _inStream.ReadAsync(buffer, 0, maxReadSize, _ct);
                    byteCalRound += readByte;
                    downloadedSize += readByte;
                    roundCount++;

                    if (roundCount == 5)
                    {
                        var now = DateTime.Now;
                        var interval = (now - lastUpdate).TotalSeconds;
                        var speed = (int)Math.Floor(byteCalRound / interval);
                        lastUpdate = now;
                        _currentFile.RecievedSize = downloadedSize;
                        _currentFile.Throughput = speed;
                        _currentFile.PercentProgress = downloadedSize;

                        byteCalRound = 0;
                        roundCount = 0;
                    }

                    await stream.WriteAsync(buffer, 0, readByte);
                } while (readByte != 0);
            }

            // Download completed
            _currentFile.Status = DownloadStatus.Complete;
            _currentFile.CompleteDate = DateTime.Now;

            ...

            // Calculate file size
            public double FileSize
            {
                get { return _fileSize; }
                set
                {
                    _fileSize = value / 1048576;
                }
            }

            // Calculate receive bytes
            public double RecievedSize
            {
                get { return _recievedSize; }
                set
                {
                    _recievedSize = value / 1048576;
                }
            }

            // Calculate percent
            public double PercentProgress
            {
                get { return _percentProgress; }
                set
                {
                    _percentProgress = value / 1048576 * 100 / _fileSize;
                }
            }

The result(the number of bytes that I downloaded) that I've been testing are sometime I received all of bytes(I checked from PercentProgress and RecievedSize) and
sometime I received 99.6-99.9% of file(again, I checked from PercentProgress and RecievedSize) So, This is the problem that I faced.

So, the question is what cause this problem?

Please note that I've been testing by download video in only one website because I don't think this problem occurred to only this website(I usually download via Chrome and the result is I receive 100% of file.)


Solution

  • The numbers you're reading (PercentProgress and ReceivedSize) are only updated once every 5 iterations of the loop. If there were 12 iterations, they would reflect the sizes of the first 10 iterations.

    You could update them a last time afterwards, ie:

            ...
        } while (readByte != 0);
    }
    
    // Download completed
    _currentFile.Status = DownloadStatus.Complete;
    _currentFile.CompleteDate = DateTime.Now;
    _currentFile.RecievedSize = downloadedSize;
    _currentFile.PercentProgress = downloadedSize;
    

    PS. If you're working in a team, remember that most people expect that the get method of a property returns exactly what it was last set to.