Using PIT (Point in Time) ID and Search_After API -NEST ElasticSearch

Question

I have been reading up on Point in time API and wanted to implement it using NEST in my .net application. However, when reading that article (.net application hyperlink), I saw the Fluent DSL example as shown below. Is there a way to find that ID without having to go to kibana on the console and doing an query search to get the id to then later place that id inside of "a-point-in-time-id"? or does "a-point-in-time-id" do that for you like is that mapped to the ID?

s => s
.PointInTime("a-point-in-time-id", p => p
.KeepAlive("1m"))

I know in the kibana cli console if you do:

POST /app-logs*/_pit?keep_alive=5m it will give you a PIT (point in time) id. How do you go about retrieving that in NEST?

and when reading up on search_after and attempting to implement it using the search after usage for the .net client using the Fluent DSL Example. I noticed that they had they word "Project" but it does not say what "Project is in the example. What would that be exactly?

s => s
.Sort(srt => srt
    .Descending(p => p.NumberOfCommits)
    .Descending(p => p.Name)
)
.SearchAfter(
    Project.First.NumberOfCommits,
    Project.First.Name
)

Here I attempted to implement .PointInTime() and .Sort() and .SearchAfter() but got stuck.

var response = await _elasticClient.SearchAsync<Source>(s => s
                  .Size(3000) // must see about this
                  .Source(src => src.Includes(i => i
                                    .Fields(f => f.timestamp,
                                            fields => fields.messageTemplate,
                                            fields => fields.message)))
                  .Index("app-logs*"
                  .Query(q => +q
                      .DateRange(dr => dr
                          .Field("@timestamp")
                              .GreaterThanOrEquals("2021-06-12T16:39:32.727-05:00")
                              .LessThanOrEquals(DateTime.Now))))
                  .PointInTime("", p => p
                                   .KeepAlive("5m"))
                  .Sort(srt => srt
                               .Ascending(p => p.timestamp))
                  .SearchAfter()

I know when you are using PIT ID you do not need the index in the search, but in the hyperlink example it does not show how you would go about implementing that. So just a bit lost on how to do so. Any pointers/guidance/tutorials would be great!

---

But just trying to see how I can do this in NEST but if you are saying its apart of the XPACK, I would understand.

Answer 1

I was able to combine PIT and SearchAfter with the code sample below:

var pit = string.IsNullOrEmpty(pitId) ? client.OpenPointInTime(new OpenPointInTimeDescriptor().KeepAlive(ServerDetail.TTL).Index(Indices.Index(Name))) : null;

            var request = new SearchRequest(Name)
            {
                SearchAfter = string.IsNullOrEmpty(pitId) ? null : new string[] { last },
                Size = ServerDetail.PageSize,
                PointInTime = new PointInTime(pit == null ? pitId : pit.Id)
            };
 List<FieldSort> sorts = new List<FieldSort>();
            foreach (var item in sortBy)
            {
                sorts.Add(new FieldSort() { Field = item.Field, Order = item.Direction == SortDirection.Asc ? SortOrder.Ascending : SortOrder.Descending });
            }
            request.Sort = sorts.ToArray();

Your SearchAfter value should be values from your sort fields for the last object of your previous search result.

For the 1st search pitId is null so a new one is created, for subsequent requests, pitId is passed.

Hope that works for you?