Another problem caused by moving to Opensearch (1.3.1, Mac homebrew) from Elasticsearch for a Rails 6 app.
Loading test environment (Rails 6.1.4.7)
>> r = Record.find(638205575)
Record Load (0.4ms) SELECT "records".* FROM "records" WHERE "records"."id" = $1 LIMIT $2 [["id", 638205575], ["LIMIT", 1]]
=> #<Record id: 638205575, created_at: "2022-04-22 08:47:01.931397000 +0000", updated_at: "2022-04-22 08:47:01.931397000 +0000", metadata: {...
>> Record.search(r.name)
Record Search (12.2ms) records_test/_search {"query":{"bool":{"should":[{"dis_max":{"queries":[{"multi_match":{"query":"Eighth Record","boost":10,"operator":"and","analyzer":"searchkick_search","fields":["*.analyzed"],"type":"best_fields"}},{"multi_match":{"query":"Eighth Record","boost":10,"operator":"and","analyzer":"searchkick_search2","fields":["*.analyzed"],"type":"best_fields"}},{"multi_match":{"query":"Eighth Record","boost":1,"operator":"and","analyzer":"searchkick_search","fuzziness":1,"prefix_length":0,"max_expansions":3,"fuzzy_transpositions":true,"fields":["*.analyzed"],"type":"best_fields"}},{"multi_match":{"query":"Eighth Record","boost":1,"operator":"and","analyzer":"searchkick_search2","fuzziness":1,"prefix_length":0,"max_expansions":3,"fuzzy_transpositions":true,"fields":["*.analyzed"],"type":"best_fields"}}]}}]}},"timeout":"11s","_source":false,"size":10000}
Traceback (most recent call last):
Searchkick::InvalidQueryError (Bad mapping - run Record.reindex)
I can't see anything in that output that indicates in what way the query is invalid or the mapping is bad. Actually running Record.reindex doesn't show any errors and simply responds with true in the console.
Can anyone see anything that might offer any clues?
The cause of this problem was related to Text fields are not optimised for operations that require per-document field data and, ultimately, to Searchkick memory leak.
Substituting a custom rake task for rake searchkick:reindex:all caused the indexes not to be built correctly. For example, here's an sdiff of the two different outcomes (correct index from the searchkick task on the right):
"records_development" : { | "records_development_20220425162622176" : {
"aliases" : { }, | "aliases" : {
> "records_development" : { }
> },
"mappings" : { "mappings" : {
> "dynamic_templates" : [
> {
> "string_template" : {
> "match" : "*",
> "match_mapping_type" : "string",
> "mapping" : {
> "fields" : {
> "analyzed" : {
> "analyzer" : "searchkick_index",
> "index" : true,
> "type" : "text"
> }
> },
> "ignore_above" : 30000,
> "type" : "keyword"
> }
> }
> }
> ],
"properties" : { "properties" : {
"abbreviation" : { "abbreviation" : {
"type" : "text", | "type" : "keyword",
> "ignore_above" : 30000,
"fields" : { "fields" : {
"keyword" : { | "analyzed" : {
"type" : "keyword", | "type" : "text",
"ignore_above" : 256 | "analyzer" : "searchkick_index"
> },
> "word_middle" : {
> "type" : "text",