Search code examples
jqueryjqgridturkishfree-jqgrid

Case insensitive search by jqGrid filterToolbar can't find special Turkish character

I have a problem when I'm using jqGrid filterToolbar. Toolbar make a search but cant find the character which is contain "ı". Forexample I can search the "yapi" word but search toolbar can't find the "yapı".

jQuery("#grid-table").jqGrid('filterToolbar',
    { stringResult: false, searchOperators: false, defaultSearch: "cn" });

My page encoding is; 

<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />

and my ajax post is here

$.ajax({ type: "Post", url: "page/get.aspx, contentType: "application/json; charset=utf-8", dataType: "json", data: "{}", success: function () { // }, error: function () { // } });

Solution

  • I'm sure that the problem is in the encoding of the HTML page which you use. I tried to reproduced the problem and opened an old demo saved in ANSI encoding. After I inserted the test yapı in the data and saved I could reproduces the problem, but verification of the code shows that the string yapı was saved as yapi because of ANSI encoding. Then I opened the same demo using Notepad (I work on Windows computer) repeat the same and I used SaveAs to be able to choose UTF-8 encoding. Now one could see really yapı string displayed in the grid instead of yapi before and I could successfully filter for the string. Of cause I had <meta charset="utf-8"> during both experiments.

    So you should validate that not only <meta charset="utf-8"> exist in the <head> of your HTML page, but the data are in UTF-8 encoding too. In case of embedded data (like in my experiment), the file need be saved in UTF-8 format.

    UPDATED: The discussion in comments shows that the main problem was case insensitive filtering of Turkish text.

    The problem was absolutely new for me, but Turkish language have two i: one with point over i and another one without point ı. Both the i have the corresponding capital I: İ and I. All the information is not different from many other languages. The main problem is in the choice of Unicode representation of the 4 characters: the Turkish characters i and I use the same codes like Latin characters: U+0069 and U+0049. Only the characters ı and İ will be mapped on U+0131 and U+0130 (see here). Such mapping makes impossible to implement case insensitive compare or JavaScript functions .toUpperCase() and .toLowerCase(). If the input text contains Latin letter i then the function .toUpperCase() should convert it to I, but it's wrong for Turkish and it should be İ instead. In the same way .toLowerCase() should produce ı for the Turkish text and i for the English text.

    Thus the first important information: it's impossible to implement one universal version of case insensitive compare without the knowledge of input language.

    OK. Now back to the problem. How to implement case insensitive searching inside of Turkish texts? After changing of licence agreement of jqGrid in version 4.7.1 I continue developing of free version of it (under MIT and GPL v2 licence) under the name free jqGrid. I implemented many new features in the first release of free jqGrid: version 4.8. The "custom filtering" feature described in the wiki article can help in the implementation.

    Based on the feature I created the following demo. I made some small bug fixes in the code of free jqGrid during the implementation. So I use the latest sources from GitHub (http://rawgit.com/free-jqgrid/jqGrid/master/js/jquery.jqgrid.src.js) in the demo (read wiki about the URLs).

    I used the following options in jqGrid

    ignoreCase: false,
customSortOperations: {
    teq: {
        operand: "==",
        text: "Turkish insensitive \"equal\"",
        filter: function (options) {
            var fieldData = String(options.item[options.cmName]).replace(/i/g,'İ').toUpperCase(),
                searchValue = options.searchValue.replace(/i/g,'İ').toUpperCase();
            return fieldData === searchValue;
        }
    },
    tne: {
        operand: "!=",
        text: "Turkish insensitive \"not equal\"",
        filter: function (options) {
            var fieldData = String(options.item[options.cmName]).replace(/i/g,'İ').toUpperCase(),
                searchValue = options.searchValue.replace(/i/g,'İ').toUpperCase();
            return fieldData !== searchValue;
        }
    },
    tbw: {
        operand: "^",
        text: "Turkish insensitive \"begins with\"",
        filter: function (options) {
            var fieldData = String(options.item[options.cmName]).replace(/i/g,'İ').toUpperCase(),
                searchValue = options.searchValue.replace(/i/g,'İ').toUpperCase();
            return fieldData.substr(0,searchValue.length) === searchValue;
        }
    },
    tbn: {
        operand: "!^",
        text: "Turkish insensitive \"does not begin with\"",
        filter: function (options) {
            var fieldData = String(options.item[options.cmName]).replace(/i/g,'İ').toUpperCase(),
                searchValue = options.searchValue.replace(/i/g,'İ').toUpperCase();
            return fieldData.substr(0,searchValue.length) !== searchValue;
        }
    },
    tew: {
        operand: "|",
        text: "Turkish insensitive \"end with\"",
        filter: function (options) {
            var fieldData = String(options.item[options.cmName]).replace(/i/g,'İ').toUpperCase(),
                searchValue = options.searchValue.replace(/i/g,'İ').toUpperCase(),
                searchLength = searchValue.length;

            return fieldData.substr(fieldData.length-searchLength,searchLength) === searchValue;
        }
    },
    ten: {
        operand: "!@",
        text: "Turkish insensitive \"does not end with\"",
        filter: function (options) {
            var fieldData = String(options.item[options.cmName]).replace(/i/g,'İ').toUpperCase(),
                searchValue = options.searchValue.replace(/i/g,'İ').toUpperCase(),
                searchLength = searchValue.length;

            return fieldData.substr(fieldData.length-searchLength,searchLength) !== searchValue;
        }
    },
    tcn: {
        operand: "~",
        text: "Turkish insensitive \"contains\"",
        filter: function (options) {
            var fieldData = String(options.item[options.cmName]).replace(/i/g,'İ').toUpperCase(),
                searchValue = options.searchValue.replace(/i/g,'İ').toUpperCase();
            return fieldData.indexOf(searchValue,0) >= 0;
        }
    },
    tnc: {
        operand: "!~",
        text: "Turkish insensitive \"does not contain\"",
        filter: function (options) {
            var fieldData = String(options.item[options.cmName]).replace(/i/g,'İ').toUpperCase(),
                searchValue = options.searchValue.replace(/i/g,'İ').toUpperCase();
            return fieldData.indexOf(searchValue,0) < 0;
        }
    }
}

    The option customSortOperations defined new custom operation for case-insensitive compare of Turkish texts. To use the option one need just specify the operations in searchoptions for the columns which contains Turkish texts:

    searchoptions: { sopt: ["tcn", "tnc", "teq", "tne", "tbw", "tbn", "tew", "ten"] }

    As the result the filtering uses "tcn" (Turkish insensitive "contains") as the default filtering operation. If one uses searchOperators: true option of filterToolbar then another searching operations could be chosen. I hope that all above custom compare operations are correct and there can be used in Turkish grids.

    UPDATED 2: I have found one more interest implementation option: the method localeCompare which supports parameters. I tested that in Google Chrome

    "i".localeCompare("İ", "tr", { sensitivity: "base" }) === 0
"i".localeCompare("I", "tr", { sensitivity: "base" }) === 1
"ı".localeCompare("I", "tr", { sensitivity: "base" }) === 0
"ı".localeCompare("İ", "tr", { sensitivity: "base" }) === -1

    or

    "i".localeCompare("İ", "tr", { sensitivity: "accent" }) === 0
"i".localeCompare("I", "tr", { sensitivity: "accent" }) === 1
"ı".localeCompare("I", "tr", { sensitivity: "accent" }) === 0
"ı".localeCompare("İ", "tr", { sensitivity: "accent" }) === -1

    but the same tests in IE11 failed opposite to the information about the browser compatibility. All the above calls of localeCompare returns 0 in IE11. It can be that one can use some another value of sensitivity to have the expected results. IE9 instead returns 1 or -1 for above calls of localeCompare. I suppose that it take in consideration only the first parameter and ignores "tr", { sensitivity: "base" } part. The results in Chrome looks so

    enter image description here

    One have the same results in Firefox

    enter image description here

    but not in IE11

    enter image description here

    One more options would be to use The ECMAScript Internationalization API class Intl.Collator (see ecma-402 and here), like

    new Intl.Collator("tr", { sensitivity: "base" }).compare("i", "İ")

    for example, but IE seems be not much better in the case.

    In any way I suppose that one can improve the above solution by including browser detection part which choose the closure for the implementation of compare and the usage of the best implementation inside of customSortOperations later. Nevertheless the above code works safe, but it's of cause not so elegant probably.