Search code examples
phpgulpcommand-line-interfacelaravel-artisan

Issues with printing extended Ascii "table" codes (e.g. ─, ┬) to console from Php script

Update

I have added a solution to the problem in an answer below. Also the title has been updated from the old to a more fitting one, since it seems that all characters from 128 to 255 seem to be causing an issue (the extended ASCII codes, as seen here: Ascii table).

Update II

The problem seems to have gone away after updating PHP to 7.1.30 (lower version might work as well). Worth noting is that my fix below instead caused problematic output in the newer version: Issue with fix after problem fixed itself Also using the helper function in the answer clearly causes problems, as it generates gibberish:

for($i=128; $i<256; $i++) {echo "'" . chr($i) . "', ";}
'�', '�', '�', '�', '�', '�',...

I have tested switching back and forth between 5.6.25 and 7.1.30 and the problem again re-appears in the 5.6. and the fix works fine in the 5.6., so it is clearly version-related.

Conclusion

It seems that the problem was caused by PHP version – previously it was 5.6.25, now it is 7.1.30 and the fix is no longer needed (and not desireable, as it causes issues). So, if you run into the ÔöÇ issue and you run lower PHP version, the fix might help you. But if you run into �, the issue lies elsewhere.

The old description:

I am trying to output results of a gulp task run from Laravel Artisan command (I need data from PHP to compile and then export assets from multiple modules and then compile them all together in the main application).

Everything works fine but for some reason, the "table" Ascii characters are output incorrectly as gibberish (Ôöî) instead of proper characters (─).

Details

See this image: Gulp output from artisan and compare it to the desired result of the same command output directly from gulp to console: The desired output .

So I was tinkering with this and the funny part is that I get the wrong result even if I just use echo "─┬──────────────"; inside my php file (which is run by Laravel artisan as php artisan my-command). The wrong result is ÔöÇÔöČÔöÇÔöÇÔöÇÔöÇÔöÇÔöÇÔöÇÔöÇÔöÇÔöÇÔöÇÔöÇÔöÇÔöÇ.

I have tried to convert the encoding (mb_convert_encoding("─┬──────────────", "UTF-8")), detect the encoding (php states "UTF-8"), setting config values ( ini_set output_encoding, internal_encoding, ... ) and tried other various methods. Up to no avail...

So I went deeper and tried to use ord() and chr() in my script and in command line to determine what is really going on. So: when i run this in the command line:

$php -a
Interactive mode enabled

<?php
$var = "─┬──────────────";
for($i = 0; $i < strlen($var); $i++) {    echo ord($var[$i])."<br/>"; }
?>

I get 196<br/>194<br/>196<br/>196<br/>196<br/>196<br/>196<br/>196<br/>196<br/>196<br/>196<br/>196<br/>196<br/>196<br/>196<br/>196<br/> (<br/> was copy-pasted from another thread and it is an OK visual separator in this case).

But when I run the exact same code from a file (run via the php artisan command), I get 226<br/>148<br/>128<br/>226<br/>148<br/>172<br/>226<br/>148<br/>128<br/>226<br/>148<br/>128<br/>226<br/>148<br/>128<br/>226<br/>148<br/>128<br/>226<br/>148<br/>128<br/>226<br/>148<br/>128<br/>226<br/>148<br/>128<br/>226<br/>148<br/>128<br/>226<br/>148<br/>128<br/>226<br/>148<br/>128<br/>226<br/>148<br/>128<br/>226<br/>148<br/>128<br/>226<br/>148<br/>128<br/>226<br/>148<br/>128<br/>

So I get 226 148 128 (incorrect) instead of the desired 196. And when I try echo chr(196), it prints the correct char, even from the script file run via Artisan (it prints the ─).

Also, when logging the output from php script file to a log file (via Laravel's \Log::info() ) I do get the correct result.

Only when running the script from a file and printing to console the issues do occur.

Conclusion

Do you have any idea what is going on?

Why do all other characters get printed out correctly and even the wrong ones do get printed correctly in the log, but incorrectly into console?

I beleive that the issue is not related to Gulp and most probably not related to Artisan. It is most probably some kind of encoding issue.

Any help would be much appreciated.

I am out of ideas...

More info

The file with the script is encoded in UTF-8.

I use the following code to run the gulp task, but I am fairly certain that is of no relevance to the real issue:

    $proc = popen($command, 'r');
    while (!feof($proc))
    {
        $fread = mb_convert_encoding(fread($proc, 4096), 'utf-8');

        \Log::info($fread);

        print($fread);
        @ flush();
    }

    pclose($proc);

I use cmder as a console emulator on Windows 10 OS.

Edit I:

I got the codes from ord() in terminal.

I.e. ord("─") returns 196 if run in the terminal via:

php -a
Interactive mode enabled

<?php
echo (ord("─"));
?>
^Z
196

I can output the correct characted by explicitly using echo chr(196) in the script file, but echo "─" prints garbage.

Edit II:

If I run chcp, I get Active code page: 852.

If I use

    $string = "─┬──────────────";
    $string = mb_convert_encoding($string, "windows-1252");

I get ???????????????? as an output of the script.

And I get the same result if I use $string = mb_convert_encoding($string, "ISO-8859-1");.

Why are all the other characters printed correctly? (See my first image above ↑). All letters, all colors, all formating is OK except for those few special characters.

I am considering either running str_replace() on each line that I am printing to replace what I get with what I want.

Or just ignoring the output's formating. It is ugly but it is readable...

Solution

  • In case anyone runs into the same issue, this function fixes the issue:

     /**
 * Fixes the extended ASCII characters incorrectly displayed into console.
 *
 * @param string $text
 *
 * @return string
 */
function convertExtendedAsciiCharacters($text)
{
    return str_replace(
        ['Ç', 'ü', 'é', 'â', 'ä', 'ů', 'ć', 'ç', 'ł', 'ë', 'Ő', 'ő', 'î', 'Ź', 'Ä', 'Ć', 'É', 'Ĺ', 'ĺ', 'ô', 'ö', 'Ľ', 'ľ', 'Ś', 'ś', 'Ö', 'Ü', 'Ť', 'ť', 'Ł', '×', 'č', 'á', 'í', 'ó', 'ú', 'Ą', 'ą', 'Ž', 'ž', 'Ę', 'ę', '¬', 'ź', 'Č', 'ş', '«', '»', '░', '▒', '▓', '│', '┤', 'Á', 'Â', 'Ě', 'Ş', '╣', '║', '╗', '╝', 'Ż', 'ż', '┐', '└', '┴', '┬', '├', '─', '┼', 'Ă', 'ă', '╚', '╔', '╩', '╦', '╠', '═', '╬', '¤', 'đ', 'Đ', 'Ď', 'Ë', 'ď', 'Ň', 'Í', 'Î', 'ě', '┘', '┌', '█', '▄', 'Ţ', 'Ů', '▀', 'Ó', 'ß', 'Ô', 'Ń', 'ń', 'ň', 'Š', 'š', 'Ŕ', 'Ú', 'ŕ', 'Ű', 'ý', 'Ý', 'ţ', '´', '­', '˝', '˛', 'ˇ', '˘', '§', '÷', '¸', '°', '¨', '˙', 'ű', 'Ř', 'ř', '■', ' '],
        [chr(128), chr(129), chr(130), chr(131), chr(132), chr(133), chr(134), chr(135), chr(136), chr(137), chr(138), chr(139), chr(140), chr(141), chr(142), chr(143), chr(144), chr(145), chr(146), chr(147), chr(148), chr(149), chr(150), chr(151), chr(152), chr(153), chr(154), chr(155), chr(156), chr(157), chr(158), chr(159), chr(160), chr(161), chr(162), chr(163), chr(164), chr(165), chr(166), chr(167), chr(168), chr(169), chr(170), chr(171), chr(172), chr(173), chr(174), chr(175), chr(176), chr(177), chr(178), chr(179), chr(180), chr(181), chr(182), chr(183), chr(184), chr(185), chr(186), chr(187), chr(188), chr(189), chr(190), chr(191), chr(192), chr(193), chr(194), chr(195), chr(196), chr(197), chr(198), chr(199), chr(200), chr(201), chr(202), chr(203), chr(204), chr(205), chr(206), chr(207), chr(208), chr(209), chr(210), chr(211), chr(212), chr(213), chr(214), chr(215), chr(216), chr(217), chr(218), chr(219), chr(220), chr(221), chr(222), chr(223), chr(224), chr(225), chr(226), chr(227), chr(228), chr(229), chr(230), chr(231), chr(232), chr(233), chr(234), chr(235), chr(236), chr(237), chr(238), chr(239), chr(240), chr(241), chr(242), chr(243), chr(244), chr(245), chr(246), chr(247), chr(248), chr(249), chr(250), chr(251), chr(252), chr(253), chr(254), chr(255)],
        $text);
}

    The function was generated using the following commands:

    for($i=128; $i<256; $i++) {echo "'" . chr($i) . "', ";}
for($i=128; $i<256; $i++) {echo "chr($i), ";}

    You might have to generate your own one if some characters are missing for you. (See the end of the answer).

    Conclusion:

    This seems to be an acceptable solution to me as it should be reasonably fast and it fixes the issue.

    More details:

    Since the issue was annoying me, I have been tinkering with the output and with my development setup some more. And it seems that the issue is somewhere between PHP output and the console. No characters from the extended ASCII table seem to work (Ascii table). Therefore I have changed the name of the question to better describe the issue.

    No changes in php.ini helped, so I resolved to this simple and quick replace.

    In case anyone has a better idea how to fix the issue, I would be interested in hearing it.

    It is worth noticing that some of the characters printed by my command line are different than those in the aforementioned Ascii table. That might be raleted to the issue, but still gives me no clue on how to solve it.