Search code examples
elasticsearchnestmetricbeat

How to read latest index in ElasticSearch


In my ElasticSearch, a new index for metricbeat is created with date suffix. i.e 'metricbeat-7.1-2019.12.20'. Now I am trying to read system parameter from the latest created index but I am not sure how to do that.

 var nodes = new Uri[]
            {
                new Uri(_options.Value.ElasticSearchUrl),
            };
            connectionPool = new StaticConnectionPool(nodes);
            string indexName = "metricbeat*";
            connectionSettings = new ConnectionSettings(connectionPool).DefaultIndex(indexName);
            elasticClient = new ElasticClient(connectionSettings);

            string[] systemFields = new string[]
            {
                "system.memory.*"
            };
            var elasticResponse = elasticClient.Search<object>(s => s
                .DocValueFields(dvf => dvf.Fields(systemFields)));

elasticResponse gives me data but it shows the older date in timestamp.

Please suggest.

  string[] systemFields = new string[]
            {
                "system.memory.actual.used.pct",
                "system.cpu.total.norm.pct",
                "system.load.5",
                "docker.diskio.summary.bytes"
            };
            var elasticResponse = elasticClient.Search<object>(s => s
                .DocValueFields(dvf => dvf.Fields(systemFields))
                .Aggregations(ag => ag.Max("last_process_time", sa => sa
                    .Field("@timestamp")))
                );

Solution

  • Cat indices API will help you find index you are looking for by using creation date as sort parameter.

        await client.Indices.CreateAsync("metricbeat-7.1-2019.12.20", create => create);
        await client.Indices.CreateAsync("metricbeat-7.1-2019.12.21", create => create);
    
        var response = await client.Cat.IndicesAsync(i => i.Index("metricbeat-7.1-*")
            .Headers("index", "creation.date")
            .SortByColumns("creation.date"));
    
        Console.WriteLine(response.Records.LastOrDefault()?.Index);
    

    Output:

    metricbeat-7.1-2019.12.21
    

    Hopefully, you defined ILM policy for your indices and my solution won't retrieve thousands of indices information from elasticsearch.

    Hope that helps.