Search code examples
c++visual-c++utf-8wstring

Cannot convert character array to wstring with utf-8 characters


When I tried to convert char* to wstring using below function in Visual C++.The function is able to convert strings in normal english language, but when I use characters from other language, it is not converting all the characters.

std::wstring s2ws(const char* utf8Bytes)
{
    const std::string& str(utf8Bytes);
    int size_needed = MultiByteToWideChar(CP_UTF8, 0, &str[0], (int)str.size(), NULL, 0);
    std::wstring wstrTo(size_needed, 0);
    MultiByteToWideChar(CP_UTF8, 0, &str[0], (int)str.size(), &wstrTo[0], size_needed);
    return wstrTo;
}

Example: When I'm printing the converted value into MessageBox, Grüßen is shown as Gr??en

I'm using this converted wstring for obtaining the contents of my directory like below:

map<wstring, wstring> getAllFiles(wstring folder, wstring filter) {  
    wstring directory = folder + L"/" + filter;
    WCHAR szBuf[MAX_PATH];
    WIN32_FIND_DATA d;
    HANDLE hFindFile = FindFirstFile(directory.c_str(), &d);
    .....
}

Here I'm not getting an expected output. i.e., the contents of the directory. But getting it when utf8bytes array is normal English characters.


Solution

  • I think you should change code to below:

    std::wstring s2ws(const char* utf8Bytes)
    {
        const std::string& str(utf8Bytes);
        int size_needed = MultiByteToWideChar(CP_ACP, 0, &str[0], (int)str.size(), NULL, 0);
        std::wstring wstrTo(size_needed, 0);
        MultiByteToWideChar(CP_ACP, 0, &str[0], (int)str.size(), &wstrTo[0], size_needed);
        return wstrTo;
    }
    

    Difference between two flags is listed here.