Search code examples
elasticsearchnest

How to view the analyzed text when a custom analyzer was used?


I can't figure out how to test my custom analyzer/view the analyzed data.

Normally I would add my custom analyzer to the "index settings" when creating the index. The problem I'm having in this case is that I'm not using an index or at least I think that I'm not and I don't know how to add my custom analyzer to the Elasticsearch client?

This is the method which I'm currently using for testing the "analysis" part:

public async Task AnalizeField(string analyzer, string textToAnalyze)
{
    var elasticClient = ElasticsearchHelper.DatabaseConnection();

    var analyzeResponse = await elasticClient.AnalyzeAsync(a => a
            .Analyzer(analyzer)
            .Text(textToAnalyze)
        );

    var result = "";

    if (analyzeResponse != null && analyzeResponse.Tokens.Count > 0)
    {
        foreach (var token in analyzeResponse.Tokens)
        {
            result += token.Token + " ";
        }

    }

    Console.WriteLine("Analyzing text \"" + textToAnalyze + "\" using the \"" + analyzer + "\" analyzer: " + result);
}

Solution

  • Found it: https://www.elastic.co/guide/en/elasticsearch/client/net-api/current/testing-analyzers.html#_testing_a_custom_analyzer_in_an_index

    Testing a custom analyzer in an index

    In this example, we’ll add a custom analyzer to an existing index. First, we need to close the index

    client.CloseIndex("analysis-index");
    

    Now, we can update the settings to add the analyzer

    client.UpdateIndexSettings("analysis-index", i => i
        .IndexSettings(s => s
            .Analysis(a => a
                .CharFilters(cf => cf
                    .Mapping("my_char_filter", m => m
                        .Mappings("F# => FSharp")
                    )
                )
                .TokenFilters(tf => tf
                    .Synonym("my_synonym", sf => sf
                        .Synonyms("superior, great")
    
                    )
                )
                .Analyzers(an => an
                    .Custom("my_analyzer", ca => ca
                        .Tokenizer("standard")
                        .CharFilters("my_char_filter")
                        .Filters("lowercase", "stop", "my_synonym")
                    )
                )
    
            )
        )
    );
    

    And open the index again. Here, we also wait up to five seconds for the status of the index to become green

    client.OpenIndex("analysis-index");
    client.ClusterHealth(h => h
        .WaitForStatus(WaitForStatus.Green)
        .Index("analysis-index")
        .Timeout(TimeSpan.FromSeconds(5))
    );
    

    With the index open and ready, let’s test the analyzer

    var analyzeResponse = client.Analyze(a => a
        .Index("analysis-index") 
        .Analyzer("my_analyzer")
        .Text("F# is THE SUPERIOR language :)")
    );