Do elasticsearch documents within a type have to have the same field structure

Question

I am working through the example in the elasticsearch guide. For a type with objects like these:

{
    "first_name":  "John",
    "last_name":   "Smith",
    "age":         25,
    "about":       "I love to go rock climbing",
    "interests": [ "sports", "music" ]
}

Can I index an object like this:

{
    "middle_name": "lee",
    "age": 36
}

So it has missing fields and one extra field not in the others.

Are all these fields still indexed? Or does there have to be the same structure for each document?

Answer 1

Elasticsearch uses dynamic mapping by default which means that the first time it sees a new field it will create a mapping for it.

In your case, the field middle_name is new, so Elasticsearch will create a mapping for the field for you automatically:

{
"middle_name": {
  "type":"string"
  }
}

So yes, the extra field will be indexed. Not all documents have to have all the fields.

However, all the same field must have the same type in all documents. That is, if you try to index the doc {"age":26} and then the doc {"age":"old"} you will get an error because in the first case the field was a number, in the second case a string:

{
   "error": "MapperParsingException[failed to parse [age]]; nested: NumberFormatException[For input string: \"adsf\"]; ",
   "status": 400
}

All this is very easy to try yourself by playing with Marvel Sense. Install it by running bin/plugin -i elasticsearch/marvel/latest in your Elasticsearch folder and then go to http://localhost:9200/_plugin/marvel/sense/index.html and try to run the following commands to try for yourself:

put typetest
put typetest/doc/2
{
  "age":25
}
put typetest/doc/2
{
  "age":"old"
}