C# WebRequest but not cURL Gives Error 403

Question

I have a WebRequest in C# that I am trying to use to retrieve data from Instagram. The WebRequest throws The remote server returned an error: (403) Forbidden., but a cURL command returns HTML. In practice, my POST form data will be longer and return JSON.

C#

        String uri = "https://www.instagram.com/query/";
        var request = (HttpWebRequest)WebRequest.Create(uri);
        request.Method = "POST";

        string postData = "q=ig_user(1118028333)";
        ASCIIEncoding encoding = new ASCIIEncoding();
        byte[] byte1 = encoding.GetBytes(postData);

        // Set the content type of the data being posted.
        request.ContentType = "application/x-www-form-urlencoded";

        // Set the content length of the string being posted.
        request.ContentLength = byte1.Length;


        using (var dataStream = request.GetRequestStream())
        {
            dataStream.Write(byte1, 0, byte1.Length);
        }

        try
        {
            var x = (HttpWebResponse)request.GetResponse();
        }
        catch (WebException wex)
        {
            String wMessage = wex.Message;
        }

Throws error 403.

cURL (in Windows)

curl "https://www.instagram.com/query/" --data "q=ig_user(1118028333)"

Returns HTML.

FireFox Request Body, Method = POST, no headers

q=ig_user(1118028333)

Returns HTML

Why would the WebRequest throw error 403, but not cURL or FireFox? What else can I do in C# to get data?

Answer 1

Why would the WebRequest throw error 403, but not cURL or FireFox?

I think you are getting confused. The reason I assume so, it's because I just tried doing the same with Postman and while I do get an HTML response, I ALSO get 403 response status code. I think you might not be paying attention to cUrl's response code. See below

What else can I do in C# to get data?

Normally, I try to use the System.Net.Http.HttpClient class, so I can check the status code first before an exception is thrown and I can even retrieve the response content (if any) even if the response code is greater than 400 (error response)

try
{
    var client = new HttpClient();
    var response = await client.GetAsync(url);

    if (response.IsSuccessStatusCode)
    {
         var json = await response.Content.ReadAsStringAsync();
    }
    else
    {
        string content = null;

        if (response.Content != null)
        {
            content = await response.Content.ReadAsStringAsync();
        }
    }

}
catch (Exception ex){}