Search code examples
c#webclientdotnet-httpclienttls1.3system.net.httpwebrequest

How to download a file from TLS1.3 enabled web site on Windows 10 using webclient or httpclient in C#


I am trying to download a file from UK government official site using WebClient and HttpClient in C# Following is the site which i guess is using TLS1.3

https://www.gov.uk/government/publications/the-uk-sanctions-list

enter image description here

and i want to download following file https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/1075160/UK_Sanctions_list.ods

I have used following code (which used to work well recently)

private string Download_https_HMT(string url, string filePath, string fileName)
    {
        // The stream of data retrieved from the web server
        Stream strResponse;
        // The stream of data that we write to the harddrive
        Stream strLocal;
        // The request to the web server for file information
        HttpWebRequest webRequest;
        // The response from the web server containing information about the file
        HttpWebResponse webResponse;
        string path, filename;
        path = filePath;// ConfigurationSettings.AppSettings["PATH"];
        filename = fileName;// ConfigurationSettings.AppSettings["FILENAME"];
        //ServicePointManager.SecurityProtocol = System.Net.SecurityProtocolType.SystemDefault; //
        //ServicePointManager.SecurityProtocol = System.Net.SecurityProtocolType.Tls13 | System.Net.SecurityProtocolType.Tls12 | System.Net.SecurityProtocolType.Tls11 | System.Net.SecurityProtocolType.Tls;/// (SecurityProtocolType)3072;
        //ServicePointManager.SecurityProtocol = (SecurityProtocolType)3072 | (SecurityProtocolType)192 | (SecurityProtocolType)768 | (SecurityProtocolType)12288;
        //ServicePointManager.SecurityProtocol=(SecurityProtocolType)12288;
        WebClient webClient = new WebClient();
        webClient.Headers.Add(HttpRequestHeader.Accept, "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet");

        webClient.Headers.Add(HttpRequestHeader.ContentType, "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet");
        webClient.Credentials = CredentialCache.DefaultCredentials; //new NetworkCredential("username", "password", "domain");
        ServicePointManager.Expect100Continue = true;
        //ServicePointManager.SecurityProtocol =SecurityProtocolType.Tls12|SecurityProtocolType.Tls13;// (SecurityProtocolType)12288;// System.Net.SecurityProtocolType.Tls12;/// (SecurityProtocolType)3072;
       
        webClient.DownloadFile(url, filePath + fileName);
        ////using (WebClient wcDownload = new WebClient())
        ////{
        ////    //WebClient wc_ = new WebClient();
        ////    //////wcDownload.Headers.Add(HttpRequestHeader.UserAgent, "Other");
        ////    //////wcDownload.Headers.Add(HttpRequestHeader.Accept, "application/xls");
        ////    //////wcDownload.DownloadFile(url, filePath+fileName);
        ////    ////ServicePointManager.Expect100Continue = true;
        ////    ServicePointManager.SecurityProtocol = (SecurityProtocolType)3072;
        ////    ////ServicePointManager.DefaultConnectionLimit = 9999;
        ////    //ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls |
        ////    //                       SecurityProtocolType.Tls11 |
        ////    //                       SecurityProtocolType.Tls12;

        ////    // Create a request to the file we are downloading
        ////    webRequest = (HttpWebRequest)WebRequest.Create(url);
        ////    // Set default authentication for retrieving the file
        ////    webRequest.Credentials = CredentialCache.DefaultCredentials;
        ////    // Retrieve the response from the server
        ////    webResponse = (HttpWebResponse)webRequest.GetResponse();
        ////    // Ask the server for the file size and store it
        ////    Int64 fileSize = webResponse.ContentLength;

        ////    // Open the URL for download
        ////    strResponse = wcDownload.OpenRead(url);
        ////    // Create a new file stream where we will be saving the data (local drive)
        ////    DirectoryInfo dirInfo = new DirectoryInfo(path);
        ////    if (!dirInfo.Exists)
        ////        dirInfo.Create();

        ////    strLocal = new FileStream(path + filename, FileMode.Create, FileAccess.Write, FileShare.None);

        ////    // It will store the current number of bytes we retrieved from the server
        ////    int bytesSize = 0;
        ////    // A buffer for storing and writing the data retrieved from the server
        ////    byte[] downBuffer = new byte[2048];

        ////    // Loop through the buffer until the buffer is empty
        ////    while ((bytesSize = strResponse.Read(downBuffer, 0, downBuffer.Length)) > 0)
        ////    {
        ////        // Write the data from the buffer to the local hard drive
        ////        strLocal.Write(downBuffer, 0, bytesSize);
        ////    }
        ////    strResponse.Close();
        ////    strLocal.Close();

        ////    return path + filename;
        ////}
        ///
        return path + filename;
    }

I tried different things but some times, I get the errors like:

The underlying connection was closed. Error on Send

or

Algorithm Mismatch

Sorry for posting code with comments, but i have sent it to show different number of things i have tried.


Solution

  • Finally I was successful in doing so, using following HttpClient but it is not applicable to all sites.

    var httpClient = new System.Net.Http.HttpClient();
            var httpResult = httpClient.GetAsync(fileUrl);
            httpResult.Wait();
            using (var resultStream = httpResult.Result.Content.ReadAsStreamAsync())
            {
    
                using (var fileStream = File.Create(filePath +"/"+ fileName))
                {
                    resultStream.Result.Seek(0, SeekOrigin.Begin);
                    resultStream.Result.CopyTo(fileStream);
                    fileStream.Close();
                }
            }
            return pathHMT + fileName2;
    

    Although it work for the above link, its not working for another link Australian Sanctions List