XDocument.Load(url) Error: Data at the root level is invalid. Line 1 position 1

Question

I am trying to read an xml document served on a webpage. Let's say that the url is "http://myfirsturl.com". The xml document at that url seems fine.

        try
        {
            string url = "http://myfirsturl.com";

            HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);

            HttpWebResponse response = (HttpWebResponse)request.GetResponse();

            Stream stream = response.GetResponseStream();

            using (XmlReader reader = 
                 XmlReader.Create(new StreamReader(stream))
            {
                var doc = XDocument.Load(reader);
                Console.WriteLine(doc);
            }
        }
        catch (Exception ex)
        {
            Console.WriteLine(ex);
        }

I keep getting the following error:

   System.Xml.XmlException: Data at the root level is invalid. Line 1, position 1.
   at System.Xml.XmlTextReaderImpl.Throw(Exception e)
   at System.Xml.XmlTextReaderImpl.Throw(String res, String arg)
   at System.Xml.XmlTextReaderImpl.ParseRootLevelWhitespace()
   at System.Xml.XmlTextReaderImpl.ParseDocumentContent()
   at System.Xml.XmlTextReaderImpl.Read()
   at System.Xml.Linq.XDocument.Load(XmlReader reader, LoadOptions options)
   at System.Xml.Linq.XDocument.Load(XmlReader reader)

I have tried the exact same code with a different url and it works for example on: "http://mysecondurl.com".

I need help for steps on what to do next...

I have looked into the error and found two possible directions for a solution:

1. The encoding of the XML returns extra characters (I don't know how to check this)
2. The webpage is blocking the request. (I don't know how to tackle this)

Thanks for your time and help :)

Answer 1

All I had to do was to set the headers to accept xml like so:

        try
        {
            string url = "http://myfirsturl.com";
            var request = (HttpWebRequest)WebRequest.Create(url);
            request.Accept = "application/xml"; // <== THIS FIXED IT

            using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
            {
                using (Stream stream = response.GetResponseStream())
                {
                    XDocument doc = XDocument.Load(stream);
                    Console.WriteLine(doc);
                }
            }
        }
        catch (Exception ex)
        {
            Console.WriteLine(ex);
        }

Thanks for the comments and help!