I'm doing some web scraping using the following methods
This is successful for most websites.
var content = new FormUrlEncodedContent(new[]
{
new KeyValuePair<string, string>("v1", "value1"),
new KeyValuePair<string, string>("v2", "value2"),
new KeyValuePair<string, string>("v3", "value3"),
});
var response = await client.PostAsync("http://url.com", content);
html = await response.Content.ReadAsStringAsync();
Sometimes certain website needs this way to get response.
var url = "http://url.com?v1=value1&v2=value2&v3=value3";
var response = await client.PostAsync(url, null);
html = await response.Content.ReadAsStringAsync();
And sometimes needs this to get response.
var query = "v1=value1&v2=value2&v3=value3";
var content = new ByteArrayContent(Encoding.UTF8.GetBytes(query));
var response = await client.PostAsync("http://url.com", content);
html = await response.Content.ReadAsStringAsync();
I really have no idea what is the difference.
If you use FormUrlEncodedContent
your parameters will be sent in request body and formatted as query string.
POST http://url.com/ HTTP/1.1
Host: url.com
Content-Length: 29
Expect: 100-continue
Connection: Keep-Alive
v1=value1&v2=value2&v3=value3
In the second case parameters will be sent as query part of URL.
POST http://url.com/?v1=value1&v2=value2&v3=value3 HTTP/1.1
Host: url.com
Content-Length: 0
In the third case you sent a content of query
in the request body.
In your sample it has the same effect as in the first case, but you made formatting by hand.