httpclient post method returns StatusCode: 403, ReasonPhrase: 'Forbidden'

Question

I'm trying to get the data from the website programatically using c# httpclient, but i'm unable to fetch the data. I have provided the link below https://ngodarpan.gov.in/index.php/home/statewise_ngo/5972/33/1

there will be a list of data shown in the table format, if you click on the any of the link there will be a popup with a full set of details, which i require to get it programatically for each record.

I have tried generating the csrf_token everytime by hitting the below link https://ngodarpan.gov.in/index.php/ajaxcontroller/get_csrf

and try to pass the csrf token & id to the following link https://ngodarpan.gov.in/index.php/ajaxcontroller/show_ngo_info

but this throws an error 403 forbidden.

private void sample1()
{
    string str = 
       "https://ngodarpan.gov.in/index.php/ajaxcontroller/show_ngo_info";
    var client = new HttpClient();

    var pairs = new List<KeyValuePair<string, string>>
    {
        new KeyValuePair<string, string>("id", "169486"),
        new KeyValuePair<string, string>("csrf_cookie_name", 
        "decab99c17a84a9040a03c362317289c")
    };

    var content = new FormUrlEncodedContent(pairs);

    var response = client.PostAsync(str, content).Result;
}

{StatusCode: 403, ReasonPhrase: 'Forbidden', Version: 1.1, Content: System.Net.Http.StreamContent, Headers: { Date: Sun, 09 Jun 2019 07:01:09 GMT Set-Cookie: csrf_cookie_name=2e39ed6c9bb142836d81233ba1a94732; expires=Sun, 09-Jun-2019 07:01:11 GMT; Max-Age=2; path=/; httponly Server: Apache/2.4.6 Server: (Red Hat Enterprise Linux) Server: OpenSSL/1.0.1e-fips Server: mod_fcgid/2.3.9 Server: PHP/5.6.30 Server: mod_wsgi/3.4 Server: Python/2.7.5 X-Powered-By: PHP/5.6.30 Content-Length: 1131 Content-Type: text/html; charset=UTF-8 }}

Answer 1

when you get csrf_token you should set its value to two things. csrf_test_name in request body and csrf_cookie_name in cookies. you can see network tab details in browser for more details.

private async Task sample1()
{
    var url = "https://ngodarpan.gov.in";
    var uri = new Uri(url);
    string str = $"{url}/index.php/ajaxcontroller/show_ngo_info";
    var csrf_token = "80c719c60ac281c34f2f7720fbd28be9";
    HttpClientHandler handler = new HttpClientHandler();
    handler.CookieContainer = new CookieContainer();
    handler.CookieContainer.Add(uri, new Cookie("csrf_cookie_name",csrf_token)); // Adding a Cookie
    var client = new HttpClient(handler);

    client.DefaultRequestHeaders.Add("X-Requested-With", "XMLHttpRequest");

    var pairs = new List<KeyValuePair<string, string>>
    {
        new KeyValuePair<string, string>("id", "169486"),
        new KeyValuePair<string, string>("csrf_test_name", csrf_token)
    };

    var content = new FormUrlEncodedContent(pairs);

    var response = await client.PostAsync(str, content);
    using (FileStream fS = File.Create("result.json"))
    {
        await response.Content.CopyToAsync(fS);
    }
    Console.WriteLine(response);
}