Program map all fields instead of ones I choose, when I use JsonNetSerializer

Question

I have some problems with mapping. Instead of default, I use JsonNetSerializer with following properties:

var connectionSettings =
                new ConnectionSettings(pool, sourceSerializer: (builtin, settings) => new JsonNetSerializer(
                    builtin, settings,
                    () => new JsonSerializerSettings { NullValueHandling = NullValueHandling.Ignore,
                        ReferenceLoopHandling = ReferenceLoopHandling.Ignore},
                    resolver => resolver.NamingStrategy = new CamelCaseNamingStrategy()
                ))
                .BasicAuthentication(userName, password);

            client = new ElasticClient(connectionSettings);

I map Lecturer like that:

private static CreateIndexDescriptor GetLecturerMap(string indexName)
        {
            CreateIndexDescriptor map = new CreateIndexDescriptor(indexName);
            map.Mappings(M => M
                .Map<Lecturer>(m => m
                    .Properties(prop => prop
                        .Text(s => s
                            .Name(n => n.FullName)
                        )
                        .Boolean(o => o
                            .Name(s => s.IsActive)
                         )
                        .Number(s => s
                            .Name(n => n.Id)
                            .Type(NumberType.Integer)
                        )

                        .Date(d => d
                            .Name(n => n.User.LastLogin)
                        )
                        .Object<User>(u=>u
                            .Name(n=>n.User)
                            .Properties(pr => pr
                                .Text(t=>t
                                    .Name(n=>n.SkypeContact)
                                    )
                                )
                            )
                    )
                )
             )
           ;
            return map;
        }

And call it like that:

public int InitializeLecturers()
    {
        string lecturersIndexName = LECUTRERS_INDEX_NAME;
        client.Indices.Create(GetLecturerMap(lecturersIndexName));
        List<Lecturer> lecturers = GetLecturers();

        client.IndexMany(lecturers, lecturersIndexName);
        return lecturers.Count;
    }

When I get lecturers from Database using following method:

private List<Lecturer> GetLecturers() {
            using (Context context = new Context(connectionString))
            {
                return context.Lecturers
                    .ToList<Lecturer>();
            }
        }

Program creates following mapping:

{
  "lecturers" : {
    "mappings" : {
      "properties" : {
        "firstName" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        },
        "fullName" : {
          "type" : "text"
        },
        "id" : {
          "type" : "integer"
        },
        "isActive" : {
          "type" : "boolean"
        },
        "isLecturerHasGraduateStudents" : {
          "type" : "boolean"
        },
        "isNew" : {
          "type" : "boolean"
        },
        "isSecretary" : {
          "type" : "boolean"
        },
        "lastLogin" : {
          "type" : "date"
        },
        "lastName" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        },
        "middleName" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        },
        "skill" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        },
        "user" : {
          "properties" : {
            "skypeContact" : {
              "type" : "text"
            }
          }
        }
      }
    }
  }
}

So I can't understand, why it ignores my mapping and adds all fields instead of ones I choose? Please tell me how to fix it. And may be I have to create mapping in a different way?

Answer 1

What's likely happening is that

1. The explicit mapping you define on index creation is applied
2. Elasticsearch adds new field mappings for properties it sees in JSON documents that do not have a mapping, and infers the field mapping type for them.

Point 2 is the default behaviour of Elasticsearch, but it can be changed by changing the dynamic property when creating the index and mapping.

Based on the contents of the question, it looks like you're using Elasticsearch 6.x, which would be

var client = new ElasticClient(settings);

client.CreateIndex("index_name", c => c
    .Mappings(m => m
        .Map<Lecturer>(m => m
            .Dynamic(false)
            .Properties(prop => prop
                .Text(s => s
                    .Name(n => n.FullName)
                )
                .Boolean(o => o
                    .Name(s => s.IsActive)
                 )
                .Number(s => s
                    .Name(n => n.Id)
                    .Type(NumberType.Integer)
                )

                .Date(d => d
                    .Name(n => n.User.LastLogin)
                )
                .Object<User>(u => u
                    .Name(n => n.User)
                    .Properties(pr => pr
                        .Text(t => t
                            .Name(n => n.SkypeContact)
                            )
                        )
                    )
            )
        )
    )
);

Per the documentation link, dynamic value of false will ignore new fields and not create new field mappings or index the fields, but the fields will still be in the _source document. You may also want to set [JsonIgnore] attribute on properties of Lecturer that should be ignored, so that they are not serialized and sent to Elasticsearch.