C# WebClient with fidler2 <open and closed> (Separate Result)

Question

So I am just trying to create a basic Stack Overflow Client using WebClient. When I run the program as is, I get an empty string result, even if I sleep and wait. However when I open up Fiddler2 the program works... All I have to do is open Fiddler... Here is the relevant code.

public partial class MainWindow : Window
{
    public  ObservableCollection<question>  questions { get; set; }

    public MainWindow() 
    {
        questions = new ObservableCollection<question>();
        this.DataContext = this;

        InitializeComponent();
    }

    void wc_DownloadStringCompleted(object sender, DownloadStringCompletedEventArgs e)
    {
        MessageBox.Show(e.Result); //Right here is the difference. When 

        <BREAK POINT HERE OR IT BREAKS>

        string data = data = e.Result.Substring(e.Result.IndexOf("class=\"question-summary narrow\"") + 31);
        string content = data.Substring(0, data.IndexOf("class=\"question-summary narrow\""));
        string v, a, t, b, tgs, link;
        questions.Add(new question
            {
                //votes = v,
                //answers = a,
                //title = t.ToUpper(),
                //body = b,
                ////tags = tgs
                //href = link
             });
    }

    private void button1_Click(object sender, RoutedEventArgs e)
    {
        WebClient wc = new WebClient();
        wc.DownloadStringAsync(new Uri(@"http://api.stackoverflow.com/1.1/questions"));
        wc.DownloadStringCompleted += new DownloadStringCompletedEventHandler(wc_DownloadStringCompleted);
    }
}
public class question
{
    public string votes { get; set; }
    public string answers { get; set; }
    public string title { get; set; }
    public string body { get; set; }
    public string tags { get; set; }
    public string href { get; set; }
}

Also worth noting is the fidler results When I load http://api.stackoverflow.com/1.1/questions in the browser fiddler shows

GET http://api.stackoverflow.com/1.1/questions 200 OK (application/json)

and

GET http://api.stackoverflow.com/favicon.ico 503 Service Unavailable (text/html)

When I load it in my program though only this shows

GET http://api.stackoverflow.com/1.1/questions 200 OK (application/json)

Answer 1

Looks like the problem is with the API itself. Even though you are not telling it that you accept GZipped content, it's GZipping it anyway, and apparently Fiddler deals with that and unzips it for you. In your app, you have to deal with this by unzipping the content. Here's a simple example of how to do that:

var wc = new WebClient();
var bytes = wc.DownloadData(new Uri(@"http://api.stackoverflow.com/1.1/questions"));
string responseText;
using (var outputStream = new MemoryStream())
{
    using (var memoryStream = new MemoryStream(bytes))
    {
        using (var gzip = new GZipStream(memoryStream, CompressionMode.Decompress))
        {
            byte[] buffer = new byte[1024];
            int numBytes;
            while ((numBytes = gzip.Read(buffer, 0, buffer.Length)) > 0)
            {
                outputStream.Write(buffer, 0, numBytes);
            }
        }
        responseText = Encoding.UTF8.GetString(outputStream.ToArray());
    }
}

Console.WriteLine(responseText);

Whether or not it will always be GZipped, who knows - you can check the Content-Encoding HTTP header to see if it's gzip, and if so, then run this code, and if not, then you can convert the bytes directly into text.