Search code examples
c#elasticsearchnest

Elasticsearch multimatch filtered with list of Ids


I'm rather new to ES, and i'm trying to solve the following issue.

I've created an index in my Elasticsearch with the following config:

client.Indices.Create(lineItemIndex,
                c => c
                    .Settings(s => s
                        .Setting("max_ngram_diff", 13)
                        .Analysis(a => a
                            .Tokenizers(tf => tf
                                .NGram("mynGram", td => td
                                    .MaxGram(15).MinGram(2)))
                            .Analyzers(aa => aa
                                .Custom("mynGram_analyzer", ca => ca
                                    .Filters(new List<string> {"lowercase"})
                                    .Tokenizer("mynGram")))))
                    .Map<ElasticSearchLineItem>(m => m
                        .Properties(ps => ps
                            .Text(ss => ss
                                .Name(na => na.LineItemName)
                                .Fields(ff => ff
                                    .Keyword(k => k
                                        .Name("keyword"))
                                    .Text(tx => tx
                                        .Name("fulltext")
                                        .Analyzer("whitespace")
                                        .Boost(10.0))
                                    .Text(tx => tx
                                        .Name("partial")
                                        .Analyzer("mynGram_analyzer")
                                        .Boost(1.0)))))
                        .Properties(ps => ps
                            .Keyword(kw => kw
                                .Name(na => na.LineItemId)
                                .Index(false)))
                        .Properties(ps => ps
                            .Keyword(kw => kw
                                .Name(na => na.Id)
                                .Index(false)))
                        .Properties(ps => ps
                            .Text(ss => ss
                                .Name(na => na.LineItemNumber)
                                .Fields(ff => ff
                                    .Keyword(k => k
                                        .Name("keyword"))
                                    .Text(tx => tx
                                        .Name("fulltext")
                                        .Analyzer("whitespace")
                                        .Boost(10.0))
                                    .Text(tx => tx
                                        .Name("partial")
                                        .Analyzer("mynGram_analyzer")
                                        .Boost(1.0)))))
                        .Properties(ps => ps
                            .Keyword(ss => ss
                                .Name(na => na.SupplierName)
                                .Index(false)))
                        .Properties(ps => ps
                            .Keyword(ss => ss
                                .Name(na => na.Unit)
                                .Index(false)))
                        .Properties(ps => ps
                            .Number(ss => ss
                                .Name(na => na.PriceAmount)
                                .Type(NumberType.ScaledFloat).ScalingFactor(100)
                                .Index(false)))
                        .Properties(ps => ps
                            .Keyword(ss => ss
                                .Name(na => na.Currency)
                                .Index(false)))
                        .Properties(ps => ps
                            .Keyword(ss => ss
                                .Name(na => na.SupplierId)
                                .Index(false)))
                        .Properties(ps => ps
                            .Text(ss => ss
                                .Name(na => na.ImageUrl)
                                .Index(false)))
                        .Properties(ps => ps
                            .Text(ss => ss
                                .Name(na => na.SupplierPriceListId)
                                .Index(false)))));

I my solution we have a searchbox, for searching.

However, we are also suppose to be able to filter the search based on SupplierId. So someone doing a search could have multiple SupplierId they would want to only see results from.

I've tried to create the following query:

var esSearch2 = new SearchDescriptor<ElasticSearchLineItem>()
            .From(0)
            .Take(250)
            .Query(q => q
                .Bool(b => b
                    .Must(mu => mu
                        .MultiMatch(m => m
                            .Fields(f => f
                                .Field(ff => ff
                                    .LineItemName.Suffix("fulltext"))
                                .Field(ff => ff
                                    .LineItemName.Suffix("partial"))
                                .Field(ff => ff
                                    .LineItemNumber.Suffix("fulltext"))
                                .Field(ff => ff
                                    .LineItemNumber.Suffix("partial")))
                        .Query(request.SearchWord)
                        .Fuzziness(Fuzziness.Auto)
                        ))                        
                    .Filter(f => f
                        .Terms(t => t
                            .Verbatim()
                            .Field(p => p
                                .SupplierId.Suffix("keyword"))
                            .Terms(request.ListOfFavorites.ToArray())))));

This returns nothing regardless if request.ListOfFavorites is empty or not. But if i remove my filter, it will correctly return results.

I guess i'm missing something, or my ordering is messed up. Can anyone help mere here?

Note: i'm using ES 7.5.1 and NEST 7.5.1

EDIT:

I made a change to my index, and removed Index(false) from my supplierId field.

Here is the mapping as shown in kibana after the update

{
"mapping": {
"properties": {
  "currency": {
    "type": "keyword",
    "index": false
  },
  "id": {
    "type": "keyword",
    "index": false
  },
  "imageUrl": {
    "type": "text",
    "index": false
  },
  "lineItemId": {
    "type": "keyword",
    "index": false
  },
  "lineItemName": {
    "type": "text",
    "fields": {
      "fulltext": {
        "type": "text",
        "boost": 10,
        "analyzer": "whitespace"
      },
      "keyword": {
        "type": "keyword"
      },
      "partial": {
        "type": "text",
        "analyzer": "mynGram_analyzer"
      }
    }
  },
  "lineItemNumber": {
    "type": "text",
    "fields": {
      "fulltext": {
        "type": "text",
        "boost": 10,
        "analyzer": "whitespace"
      },
      "keyword": {
        "type": "keyword"
      },
      "partial": {
        "type": "text",
        "analyzer": "mynGram_analyzer"
      }
    }
  },
  "priceAmount": {
    "type": "scaled_float",
    "index": false,
    "scaling_factor": 100
  },
  "supplierId": {
    "type": "keyword"
  },
  "supplierName": {
    "type": "keyword",
    "index": false
  },
  "supplierPriceListId": {
    "type": "text",
    "index": false
  },
  "unit": {
    "type": "keyword",
    "index": false
  }
}
}
}

Solution

  • In you mapping you specified SupplierId to be excluded from the index, so you won't be able to search on it.

    .Properties(ps => ps
        .Keyword(ss => ss
            .Name(na => na.SupplierId)
            .Index(false)))
    

    also, you don't need to specify suffix for your field as it's not defined as multifield, so simply

    .Filter(f => f
        .Terms(t => t
            .Verbatim()
            .Field(p => p.SupplierId)
            .Terms(request.ListOfFavorites.ToArray())))));
    

    is enough.

    Hope that helps.