Search code examples
c#jsonnestelasticsearch-plugin

Using C# Nest API to get nested json data is not retrieving data


I have the following json coming through the elasticsearch:

{
  "_index": "data-2016-01-14",
  "_type": "type-data",
  "_id": "AVJBBNG-TE8FYIA1rf1p",
  "_score": 1,
  "_source": {
    "@message": {
      "timestamp": 1452789770326461200,
      "eventID": 1452789770326461200,
      "eventName": "New",
      "Price": "38.34",
      "Qty": 100,
      "statistic_LatencyValue_ns": 1142470,
      "statistic_LatencyViolation": false,
      "statistic_LossViolation": false
    },
    "@timestamp": "2016-01-14T16:42:50.326Z",
    "@fields": {
      "timestamp": "1452789770326"
    }
  },
  "fields": {
    "@timestamp": [
      1452789770326
    ]
  }
}

I'm using Nest to try to get the eventName data i created the class and marked the property:

public class ElasticTest
{
    [ElasticProperty(Type = FieldType.Nested)] 
    public string eventName { get; set; }
}

But the following query is returning 0 results, what am i doing wrong?

var result = client.Search<CorvilTest>(s => s
                    .From(0)
                    .Size(10000)
                    .Query(x => x
                    .Term(e => e.eventName,"New"))
                    );
                var r = result.Documents;

Mapping definition:

{
   "data-2016-01-14": {
      "mappings": {
         "type-data": {
            "properties": {
               "@fields": {
                  "properties": {
                     "timestamp": {
                        "type": "string"
                     }
                  }
               },
               "@message": {
                  "properties": {
                     "OrderQty": {
                        "type": "long"
                     },
                     "Price": {
                        "type": "string"
                     },
                     "eventID": {
                        "type": "long"
                     },
                     "eventName": {
                        "type": "string"
                     },
                     "statistic_LatencyValue_ns": {
                        "type": "long"
                     },
                     "statistic_LatencyViolation": {
                        "type": "boolean"
                     },
                     "statistic_LossViolation": {
                        "type": "boolean"
                     },
                     "timestamp": {
                        "type": "long"
                     }
                  }
               },
               "@timestamp": {
                  "type": "date",
                  "format": "dateOptionalTime"
               }
            }
         }
      }
   }
}

Solution

  • I see that the field @message.eventName is using a standard analyzer which means that its value is lower-cased and split at word boundaries before indexing. Hence the value "new" is indexed and not "New". Read more about it here. You need to be mindful about this fact when using a Term Query. Another thing is that the field eventName is not of nested type. So the code below should work for you.

    var result = client.Search<CorvilTest>(s => s
        .From(0)
        .Size(10000)
        .Query(x => x
            .Term(e => e.Message.EventName, "new"))); // Notice I've used "new" and not "New"
    
    var r = result.Documents;
    

    For the above code to work the definition of CorvilTest class should be something like below:

    public class CorvilTest
    {
        [ElasticProperty(Name = "@message")]
        public Message Message { get; set; }
    
        /* Other properties if any */
    }
    
    public class Message
    {
        [ElasticProperty(Name = "eventName")]
        public string EventName { get; set; }
    }