Search code examples
c#unicodestreamreader

C# StreamReader Encoding.UTF8 not working


I have a C# Project in Visual studio which download and parse XML file that contains Korean, Chinese and another unicode characters. For example for korean artist named Taeyang it produce XML like this :

<name>태양</name>

but it returns

<name>??</name>

I have tried StreamReader Encoding.Default but result is

<name>태양</name>

The code:

string address = String.Format("http://musicbrainz.org/ws/2/artist/{0}?inc=url-rels", mbids[ord]);
HttpWebRequest newRequest = WebRequest.Create(address) as HttpWebRequest;
               newRequest.Headers["If-None-Match"] = etagProf;
               newRequest.Headers[HttpRequestHeader.AcceptEncoding] = "gzip";
var response = newRequest.GetResponse();
// Reader
Stream stream = response.GetResponseStream();
StreamReader reader = new StreamReader(stream, Encoding.UTF-8);
string data = reader.ReadToEnd();

and the xml source:

<?xml version="1.0" encoding="UTF-8"?>
<metadata xmlns="http://musicbrainz.org/ns/mmd-2.0#">
    <artist type="Person" id="d84e5667-3cbe-4556-b551-9d7e4be95d71">   
        <name>태양</name>
        <sort-name>Taeyang</sort-name><gender>Male</gender>
        <country>KR</country>
        ...........
    </artist>
</metadata>

I'm confused, why it happens ? Any idea dude ?


Solution

  • using the code below (notice I comment out 2 of your lines)

    //newRequest.Headers["If-None-Match"] = "d84e5667-3cbe-4556-b551-9d7e4be95d71";
    //newRequest.Headers[HttpRequestHeader.AcceptEncoding] = "gzip";
    

    and changed your line: StreamReader(stream, Encoding.UTF-8);

    to : StreamReader(stream, Encoding.UTF8);

    I got a good result characters wise: enter image description here

    string address = String.Format("http://musicbrainz.org/ws/2/artist/{0}?inc=url-rels","d84e5667-3cbe-4556-b551-9d7e4be95d71");
    HttpWebRequest newRequest = WebRequest.Create(address) as HttpWebRequest;
    //newRequest.Headers["If-None-Match"] = "d84e5667-3cbe-4556-b551-9d7e4be95d71";
    //newRequest.Headers[HttpRequestHeader.AcceptEncoding] = "gzip";
    var response = newRequest.GetResponse();
    // Reader
    Stream stream = response.GetResponseStream();
    StreamReader reader = new StreamReader(stream, Encoding.UTF8);
    string data = reader.ReadToEnd();
    MessageBox.Show(data);