Search code examples
azure-cognitive-search

Document size adjusting Search.Score - virtually reducing Scoring profile score


We are using scoring profile for driving the relevance and adjusting scores i.e. boost the relevance for a attribute isActive is 1 by 50 using function in scoring profile, While searching for a specific fields on the Index by passing &searchFields=******

however Search.Score seems highly squeezed by size of the document , smaller the size high score probably due to TF-IDF…..

And this is defeating the purpose of using scoring profile , however in our case we don’t want score to be impact due to size of document since we are passing searchFields.

Cases where searchFields are not passed we want scores to be adjusted by size i.e. free form search in all searchable fields.

example search query -

agency temps&$count=true&$top=30&$skip=0&searchMode=All&$filter=(CompanyCode eq '13453' and VNumber eq '00023232312016') &scoringProfile=BusinessProfile1&searchFields=VCategory


Solution

  • For now I adopted the approach to adjust the parameters for algorithm BM25 as advised by Liam, and added b as 0.0 in index creation json, so that document size is not used during TF-IDF while calculating score for the document,

       "similarity": {
        "@odata.type": "#Microsoft.Azure.Search.BM25Similarity",
        "b" : 0.0,
        "k1" : 1.3
    }
    

    however same time identified another field on the index having a correlation with size of the record on the index i.e. larger the size higher the value of that field and using that in scoring profile for the case where document size should be considered in scoring.