Multi-threading with WebRequest and StreamWriter in VB.NET

Question

I need to make it work in threads for example, thread1 makes call to url with 'order_id=1' and thread2 makes call to url with 'order_id=2' and so on, and the result is then written into the file.

Here's the code:

Public Sub download()
    Dim address As String = "http://www.example.com/sales.php&order_id="
    Dim FILE_NAME As String = "D:\Folder\Licenses.txt"
    Dim index As Integer = 0

    Do While index <= 100
            If index > 100 Then
                Exit Do
            End If
        Try
            Dim request As WebRequest = WebRequest.Create(address & index.ToString)
            Dim response As WebResponse = request.GetResponse()                
            If CType(response, HttpWebResponse).StatusDescription = "OK" Then
                Dim dataStream As Stream = response.GetResponseStream()
                Dim reader As New StreamReader(dataStream)
                Dim responseFromServer As String = reader.ReadToEnd()
                If Not File.Exists(FILE_NAME) Then
                    Using sw As StreamWriter = File.CreateText(FILE_NAME)
                        sw.WriteLine(responseFromServer)
                        index += 1
                    End Using
                ElseIf File.Exists(FILE_NAME) Then
                    Using sw As StreamWriter = File.AppendText(FILE_NAME)
                        sw.WriteLine(responseFromServer)
                        index += 1
                    End Using
                End If
            End If
        Catch ex As Exception
        End Try
        Loop
End Sub

Answer 1

A very simple approach would be using Parallel.For(). You just have to make sure that you don't write different things to the output file at the same time. This is solved with the Monitor, which ensures that there will be only one thread in the critical section. Exception handling is omitted:

Dim address As String = "http://www.example.com/sales.php&order_id="
Dim FILE_NAME As String = "D:\Folder\Licenses.txt"

Using fstream As New StreamWriter(FILE_NAME)
    Parallel.For(0, 100, Sub(i As Integer)
                           Dim client As New WebClient
                           Dim content = client.DownloadString(address + i.ToString())
                           Monitor.Enter(fstream)
                           fstream.WriteLine(content)
                           Monitor.Exit(fstream)
                       End Sub)
End Using

This sample could be improved by creating the web clients as thread-local objects. This would avoid creating new clients for every iteration.