Nest - Reindexing

Question

Elasticsearch released their new Reindex API in Elasticsearch 2.3.0, does the current version of NEST (2.1.1) make use of this api yet? If not, are there plans to do so? I am aware that the current version has a reindex method, but it forces you to create the new index. For my use case, the index already exists.

Any feedback/insights will be greately appricated. Thnx!

Answer 1

This kind of question is best asked on the github issues for NEST since the committers on the project will be able to best answer :)

A commit went in on 6 April to map the new Reindex API available in Elasticsearch 2.3.0, along with other features like the Task Management API and Update By Query. This made its way into NEST 2.3.0

NEST 2.x already contains a helper for doing reindexing that uses scan/scroll under the covers and returns an IObservable<IReindexResponse<T>> that can be used to observe progress

public class Document {}

var observable = client.Reindex<Document>("from-index", "to-index", r => r
    // settings to use when creating to-index
    .CreateIndex(c => c
        .Settings(s => s
            .NumberOfShards(5)
            .NumberOfReplicas(2)
        )
    )
    // query to optionally limit documents re-indexed from from-index to to-index
    .Query(q => q.MatchAll())
    // the number of documents to reindex in each request.
    // NOTE: The number of documents in each request will actually be
    //     NUMBER * NUMBER OF SHARDS IN from-index
    // since reindex uses scan/scroll
    .Size(100)
);

ExceptionDispatchInfo e = null;
var waitHandle = new ManualResetEvent(false);

var observer = new ReindexObserver<Document>(
    onNext: reindexResponse =>
    {
        // do something with notification. Maybe log total progress
    },
    onError: exception =>
    {
        e = ExceptionDispatchInfo.Capture(exception);
        waitHandle.Set();
    },
    completed: () =>
    {
        // Maybe log completion, refresh the index, etc..
        waitHandle.Set();
    }
);

observable.Subscribe(observer);  

// wait for the handle to be signalled
waitHandle.Wait();

// throw the exception if one was captured
e?.Throw();

Take a look at the ReIndex API tests for some ideas.

The new Reindex API is named client.ReIndexOnServer() in the client to differentiate it from the existing observable implementation.