Search code examples
arrayshttpwebrequesthttpwebresponse

Char Array of Byte Array of HttpWebResponse of HttpWebRequest returns "System.Net.HttpWebRequest"


I'm trying to request a webpage using WebRequest.GetResponse(); and convert that response to a chararray, so I can sort through the array and get any HREF tags that are on the page. The problem is, somewhere in my code the response turns into "System.Net.HttpWebRequest", instead of the HTML that should be retrieved from the page.

The code to get the char array:

HttpWebRequest request = (HttpWebRequest)WebRequest.Create(urlTextBox.Text);
            request.UserAgent = "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.7 (KHTML, like Gecko) Chrome/7.0.517.44 Safari/534.7";
            HttpWebResponse response = (HttpWebResponse)request.GetResponse();
            System.Text.UTF8Encoding encoding = new System.Text.UTF8Encoding();
            bytearray = encoding.GetBytes(Convert.ToString(response));
            chararray = encoding.GetChars(bytearray);

The code to search for links (commented for troubleshooting):

for (int i = 0; i < chararray.Length; i++)
{
    // Get all HREFs
    if (i < 500 & chararray[i] == 'h' & chararray[i + 1] == 'r' & chararray[i + 2] == 'e' & chararray[i + 3] == '=' & chararray[i + 4] == '"')
    {
        for (int tempi = 0; bytearray[i + 4 + tempi] != '"';)
        {
             tempstring = tempstring + chararray[i + 4 + tempi].ToString();
        }
        urlarray[urlarray.Length + 1] = tempstring;
        i = i + 4;
    }
}
scrapeLink1.Text = urlarray[1];

If I missed something, or more information is needed, let me know.


Solution

  • The response is stream that you must first read from.

    HttpWebRequest request = WebRequest.Create(urlTextBox.Text) as HttpWebRequest;
    if (request != null)
    {        
        request.UserAgent = "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.7 (KHTML, like Gecko) Chrome/7.0.517.44 Safari/534.7";
        using (HttpWebRepsonse response = request.GetResponse() as HttpWebResponse)
        using (StreamReader rdr = new StreamReader(response.GetResponseStream())
        {
            string result = rdr.ReadToEnd();
        }
    }