Convert To Unicode from CP1251 in C#

Question

I have big data that contains russian text

"Ìèíàñÿí Ðóäèê Ñàðêèñîâ"

i need to convert it in to UNICODE in C#

"Минасян Рудик Саркисов"

how to convert it?

Answer 1

If it is really 1251, then the code is:

var enc1251 = Encoding.GetEncoding(1251);
var enc8859 = Encoding.GetEncoding("iso-8859-1");
string str = "Ìèíàñÿí Ðóäèê Ñàðêèñîâ";
byte[] bytes = enc8859.GetBytes(str);
string str2 = enc1251.GetString(bytes);

The Encoding.GetEncoding("iso-8859-1").GetBytes(str) returns the "original" (unprocessed) byte[] array, that I then decode with the CP1251.

I'm adding a little program to "solve" this type of problems. Note that this is a .NET Framework program, not a .NET Core program, because .NET Core has a problem with Encoding.GetEncodings. This program will search for encodings that can be used to fix problems with wrongly encoded text. It will show a list of candidate encodings that can encode/decode the full text given, and then will try to match them.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

class Program
{
    static void Main(string[] args)
    {
        Console.OutputEncoding = System.Text.Encoding.UTF8;

        var encodings = Encoding.GetEncodings();

        Console.WriteLine($"Testing {encodings.Length} encodings");
        Console.WriteLine();

        string from = "Ìèíàñÿí Ðóäèê Ñàðêèñîâ"; // "Єюээ";
        string to = "Минасян Рудик Саркисов"; // "тонн";

        var encodingsFrom = new List<(string Hex, Encoding Encoding)>();
        var encodingsTo = new List<(string Hex, Encoding Encoding)>();

        var stringPlusEncodings = new[]
        {
            new { Str = from, Encodings = encodingsFrom },
            new { Str = to, Encodings = encodingsTo },
        };

        foreach (var stringPlusEncoding in stringPlusEncodings)
        {
            Console.WriteLine(stringPlusEncoding.Str);

            foreach (var info in encodings)
            {
                var enc = info.GetEncoding();

                bool unicodeEncoding = enc.BodyName.StartsWith("utf-");

                if (!enc.IsSingleByte && !unicodeEncoding)
                {
                    Console.WriteLine($"Skipped {enc.BodyName}");
                    continue;
                }

                enc = (Encoding)enc.Clone();

                // We replace unknown characters with easy-to-find code 0
                // Note that this is useless for encodings that map all the characters
                // and use the 0 for something else (like utf-16 and utf-32)
                enc.EncoderFallback = new EncoderReplacementFallback("\0");

                var bytes = enc.GetBytes(stringPlusEncoding.Str);

                if (!unicodeEncoding && (bytes.Length == 0 || bytes.Any(x => x == 0)))
                {
                    continue;
                }

                // Write in hex format
                string encodedHex = string.Join(" ", bytes.Select(x => x.ToString("x2")));

                Console.WriteLine($"{encodedHex} {enc.HeaderName}");

                stringPlusEncoding.Encodings.Add((encodedHex, enc));
            }

            Console.WriteLine(string.Empty);
        }

        Console.WriteLine("Candidates:");

        foreach (var encodingFrom in encodingsFrom)
        {
            var encodingsTo2 = encodingsTo.Where(x => encodingFrom.Hex == x.Hex).ToArray();

            foreach (var encodingTo in encodingsTo2)
            {
                Console.WriteLine($"{encodingFrom.Encoding.HeaderName} -> {encodingTo.Encoding.HeaderName}");
            }
        }
    }
}