iconv() – how to detect offending character?

Question

I use iconv() to convert CSV data from UTF-8 to Windows-1252.

$converted = iconv("UTF-8", "Windows-1252", $csvData);

In some cases, iconv() failed quietly, returning false.

I also tried using //TRANSLIT but `iconv()´ returns false here as well.

When i add the //IGNORE statement to the target charset, the conversion succeeds, but that means one or more character(s) got lost.

I can stick to //IGNORE but i would like to find out which character(s) are causing the problem.

How can i do this?

Answer 1

It was bad idea to work with string as char array (see question comments) because php string type

Internally, PHP strings are byte arrays. As a result, accessing or modifying a string using array brackets is not multi-byte safe, and should only be done with strings that are in a single-byte encoding such as ISO-8859-1.

So we can use mb_substr for utf-8 and work with symbols not bytes

error_reporting('E_ALL & !E_NOTICE');
$yourString = "test bad ☺ string";
$convertString = '';
$badChars = [];

if (iconv("UTF-8", "Windows-1252", $yourString) === false) {       
    for($i = 0, $stringLength = mb_strlen($yourString); $i < $stringLength; $i++) {
        $char = mb_substr($yourString, $i, 1);
        $convertChar = iconv("UTF-8", "Windows-1252", $char);

        if ($convertChar === false) {
            $badChars[$i] = $char;
        } else {
            $convertString .= $convertChar;
        }   
    }
} else {
    $convertString = iconv("UTF-8", "Windows-1252", $yourString);
}

var_dump($badChars, $convertString);

Result array(1) { [9]=> string(3) "☺" } string(16) "test bad string"

P.S. The next time I will give a more detailed answer with the code. My mistake