How do I read Little Endian encoded files in C#?

Question

I have a stream coming in from a HttpWebResponse and when I am using the code below to read and basically re-create the file on my local machine.

string sx = "";
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
    using (Stream resStream = response.GetResponseStream())
    {
        StreamReader reader = new StreamReader(resStream, Encoding.Default); // I have also tried using Encoding.UTF8/7/ASCII etc.
        sx = reader.ReadToEnd();
        // sourceItem.Name would have the file name along with the extension.
        using (StreamWriter sw = System.IO.File.AppendText(sourceItem.Name))
        {
            sw.Write(sx);
        }
    }
    response.Close();
}

But when I open the file on my local machine, I get all weird garbage symbols.

Picture:

Garbage characters and symbols

I believe this is a problem with encoding of the files. I used file * on git bash to check the encoding and it turns out, it's Little Endian. Little Endian Screenshot

But unfortunately, I cannot see Little Endian in System.Text.Encoding. How do I fix my problem?

Thanks for the help.

Answer 1

How do I fix my problem?

Are you sure your http endpoint returns a text? My guess would be that this returns some binary file, and not a text file, as you expect. If you want to write a binary response directly to a file, you can do it like this, if I remember correctly:

using (Stream input = response.GetResponseStream())
using (Stream output = File.Open(sourceItem.Name, FileMode.Append))
{
    input.CopyTo(output);
}

I wonder why you're appending to the file though. If you want to "basically re-create the file on my local machine" I would expect writing the full response as one file, would be better. Like:

using (Stream input = response.GetResponseStream())
using (Stream output = File.OpenWrite(sourceItem.Name))
{
    input.CopyTo(output);
}

If this is not the problem, I would go about it in these steps:

1. Can the request be issued in a browser or other http client like Postman? Then I would do that to see if the response is what you expect it to be.
2. But if all's well at step 1, I'd set a breakpoint after sx = reader.ReadToEnd(); and inspect sx to see if the resulting string looks right. If it doesn't you would need to try different encodings. Maybe check the headers in Postman or your browser's debugging tools, to see if they contain a hint about the encoding.
3. If you now know step 1 and 2 are as expected, and you write the string to the file in UTF-8 encoding as AppendText does, and your file is still not shown correctly, then I'd question the text viewer you use - though that seems a bit odd for text viewer not being able to handle UTF-8 correctly.