C# HttpWebRequest POST strange response

Question

Good morning, I am writing a program that makes a POST webrequest to a website (websta.me). The request works, but the problem is the response. I analyzed the http requests using Charles and this is the normal response:

{"meta":{"code":200},"data":{"created_time":"xs","text":"xxxx <3","from":{"username":"xxxx","profile_picture":"xxxx","id":"xxxx","full_name":"xxxx"},"id":"xxxx"},"status":"OK","called_count":x}

My program is returning this:

Any ideas on how to solve this? Thanks in advance for any help :) This is my code if it can help:

byte[] bytes = ASCIIEncoding.UTF8.GetBytes("comment=+xxx&media_id=xxx");
HttpWebRequest postReq = (HttpWebRequest)WebRequest.Create("http://websta.me/api/comments/xxx);
WebHeaderCollection postHeaders = postReq.Headers;
postReq.Method = "POST";
postReq.Timeout = 10000;
postReq.Host = "websta.me";
postReq.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Firefox/35.0";
postReq.Accept = "application/json, text/javascript, */*; q=0.01";
postHeaders.Add("Accept-Language", "it-IT,it;q=0.8,en-US;q=0.5,en;q=0.3");
postHeaders.Add("Accept-Encoding", "gzip, deflate");
postReq.ContentType = "application/x-www-form-urlencoded; charset=UTF-8";
postHeaders.Add("X-Requested-With", "XMLHttpRequest");
postReq.Referer = "http://websta.me/feed";
postReq.ContentLength = bytes.Length;

postReq.CookieContainer = cookies;

postReq.KeepAlive = true;

postHeaders.Add("Pragma", "no-cache");
postHeaders.Add("Cache-Control", "no-cache");
Stream postStream = postReq.GetRequestStream();
postStream.Write(bytes, 0, bytes.Length);
postStream.Close();
HttpWebResponse postResponse;
postResponse = (HttpWebResponse)postReq.GetResponse();
StreamReader reader = new StreamReader(postResponse.GetResponseStream());
MessageBox.Show(reader.ReadToEnd());

Answer 1

The result you are receiving from your request is GZip compressed data (the first three bytes are 0x1f8b08).

To get the text from it, just decompress it using GZipStream. Something like this:

private byte[] Decompress(byte[] data)
{
    using (GZipStream stream = new GZipStream(new MemoryStream(data), CompressionMode.Decompress))
    {
        const int size = 4096;
        byte[] buffer = new byte[size];
        using (MemoryStream memory = new MemoryStream())
        {
            int count = 0;
            do
            {
                count = stream.Read(buffer, 0, size);
                if (count > 0)
                {
                    memory.Write(buffer, 0, count);
                }
            }
            while (count > 0);
            return memory.ToArray();
        }
    }
}

Then modify the end of your code (from where you get your response) to:

postResponse = (HttpWebResponse)postReq.GetResponse();
Stream Reader = postResponse.GetResponseStream();
byte[] Data = new byte[postResponse.ContentLength];
Reader.Read(Data, 0, Data.Length);
Data = Decompress(Data);
string Result = System.Text.Encoding.UTF8.GetString(Data);
MessageBox.Show(Result);

This reads the response data (which is compressed) into a byte array, then decompresses it before converting it to the appropriate text (which I believe is utf-8 from the content-type description).

Please note that the GZip decompression routine is taken from http://www.dotnetperls.com/decompress (it is not mine).