Search code examples
javascripthtmlnon-ascii-characters

Why are non-ASCII characters displayed as weird symbols?


I have two cases here:

My database contains a lot of information which I want to fetch to the page. Some of this information is name which contain non-ASCII characters like Uwe Rülke

- Old solution which works well:

I fetch the data from the database and populate the page directly from a VB while loop. In this case all the chars are displaying correctly Uwe Rülke.

- New solution which doesn't work properly:

The VB While loop doesn't throw the data directly to the page, rather in a JavaScript strings (to enhance performance by not calling the database each now and then). But when I used the information stored in the JavaScript variables, I got something like this: Uwe R�lke.

In both cases, the page's encoding is:

<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">

Where did I go wrong?

This is the code used to fetch (from the database) and then save to JavaScript strings.

I'm using AJAX LOAD from a page called ISEquery to build a specific request and query it from the database. It is used to either fetch data as an Excel file or as plain HTML. At this point the characters are well represented.

Then the magic happens, and the characters get mis-represented. I checked it in the exctractFields function:

$("<div></div>").load("ISEquery.asp?isExcel=0&" + info, function(){
                            // Extracting the fields into an array
                            var rows = "";
                            var options = "";
                            $(this).children().each(function(index){
                                var fieldsArray = exctractFields($(this).html());
                                rows += createISELine(fieldsArray);
                                options += createISELine_ComboBox(fieldsArray);
                            });
                        });

Solution

  • I followed the string from server to the page and found that it is gets misrepresented after the AJAX LOAD, so I found this answer which resolved my problem. Although I had to use the charset="iso-8859-1" for it to work rather than charset="UTF-8".

    So the final answer is:

    -Encoding in the HTML page:

    <meta http-equiv="Content-Type" content="text/html"; charset="iso-8859-1">
    

    -Encoding the Ajax data:

     $.ajaxSetup({
              'beforeSend' : function(xhr) {
               xhr.overrideMimeType('text/html; charset=iso-8859-1');
            },
        });
    

    And now characters are displayed correctly.

    (The lead was from Aaron Digulla's answer.)