Search code examples
indexingredisfull-text-searchqueryingredisearch

Why Redis JSON index creation/querying with non-English languages doesn't work?


I create the index:

 const searchSchema: any = {
      "$.Id": { type: SchemaFieldTypes.TEXT, AS: "Id", NOSTEM: true },
      "$.FirstName": {
        type: SchemaFieldTypes.TEXT,
        AS: "FirstName",
        LANGUAGE: RedisSearchLanguages.TURKISH,
      },
      "$.LastName": {
        type: SchemaFieldTypes.TEXT,
        AS: "LastName",
        LANGUAGE: RedisSearchLanguages.TURKISH,
      },
      "$.LicenseId": {
        type: SchemaFieldTypes.TEXT,
        AS: "LicenseId",
        NOSTEM: true,
      },
      "$.Specialties[*]": { type: SchemaFieldTypes.TAG, AS: "Specialties" },
      "$.SubSpecialties[*]": {
        type: SchemaFieldTypes.TAG,
        AS: "SubSpecialties",
      },
    };

    // Create a new index for the Doctor type
    await client.ft.create(REDIS_JSON_INDEX, searchSchema, {
      ON: "JSON",
      PREFIX: REDIS_JSON_PREFIX,
      LANGUAGE: RedisSearchLanguages.TURKISH,
    });

Then when I query for prefix/postfix expression with a Turkish character in it, it returns nothing however if I do the same query without prefix/postfix, it returns multiple items. I have tried passing using RedisSearchLanguages.TURKISH as above, tried 'turkish', 'tr'. Tried setting LANGUAGE on createOptions and on each individual field, still no luck.

FT.SEARCH 'doctors-index' "@FirstName:OĞUZ*" does not work

FT.SEARCH 'doctors-index' "@FirstName:OĞUZ" works


Solution

  • I ended up contacting Redis support and they told me that my Redisearch module was behind a version. Apparently they roll out updates in phases. There was no Unicode search in version 2.10.12. Once they bumped up the version to 2.10.13, the queries started to return results.