Search code examples
elasticsearchindexingnestreindex

Reindexing using NEST V5.4 - ElasticSearch


I'm quite new to ElasticSearch. I'm trying to reindex a index in order to rename it. I'm using NEST API v5.4. I saw this example:

var reindex =
    elasticClient.Reindex<Customer>(r =>
        r.FromIndex("customers-v1")
            .ToIndex("customers-v2")
            .Query(q => q.MatchAll())
            .Scroll("10s")
            .CreateIndex(i =>
                i.AddMapping<Customer>(m =>
                    m.Properties(p =>
                        p.String(n => n.Name(name => name.Zipcode).Index(FieldIndexOption.not_analyzed))))));

Source: http://thomasardal.com/elasticsearch-migrations-with-c-and-nest/

However, I can't reproduce this using NEST 5.4. I think that is to version 2.4. I check the breaking changes of ElasticSearch and try reindexing using this:

Source: https://www.elastic.co/guide/en/elasticsearch/client/net-api/current/nest-breaking-changes.html

public method Nest.ReindexDescriptor..ctor Declaration changed (Breaking)
2.x: public .ctor(IndexName from, IndexName to) 5.x: public .ctor()

var reindex = new client.Reindex(oldIndexName, newIndexName);

But this did not work too. I also search for documentation but i didn't find any code on c#, just JSON Source: https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-reindex.html)

Can someone give me a example how to reindex using NEST 5.4 on C#?

Thanks in advance! :slight_smile:


Solution

  • After search for 2 long days I found out the solution to reindex a index. In order to solve future problems, I'll provide my solution.

    Nest Version - 5.4

    var reindex = client.Reindex<object>(r => r
                  .BackPressureFactor(10)
                  // ScrollAll - Scroll all the documents of the index and store it for 1minute 
                  .ScrollAll("1m", 2, s => s
                      .Search(ss => ss
                          .Index(oldIndexName)
                              .AllTypes())
                          // there needs to be some degree of parallelism for this to work
                          .MaxDegreeOfParallelism(4))
                  .CreateIndex(c => c
                      // New index here
                      .Index(newIndexName)
                      .Settings(
                          // settings goes here)
                      .Mappings(
                          // mappings goes here))
                  .BulkAll(b => b
                      // New index here!
                      .Index(newIndexName)
                      .Size(100)
                      .MaxDegreeOfParallelism(2)
                      .RefreshOnCompleted()));
    

    the ReIndex method returns a cold IObservable on which you have to call .Subscribe() to kick off everything.

    So, you need to add it to your code:

    var o = new ReindexObserver(
                onError: (e) => { //do something },
                onCompleted: () => { //do something });
    reindex.Subscribe(o);
    

    Useful links to check this are:

    Documentation

    Issue 2660 on GitHub

    Issue 2771 on GitHub