I am trying to make a mapping for the following document work:
{
"eventDatabase": "abc",
"usageLibraryEventType": "ABC",
"name": "Prionti",
"namespace": "Prionti's namespace",
"latestBuildTimestamp": 1581348323634,
"flattenedEventProperties": [
"User Id"
],
"eventDefinitions": [
{
"buildInfo": {
"baseVersion": "1",
"branch": "master",
"buildName": "something.com",
"orgName": "Prionti's org",
"repoName": "myrepo",
"buildTimestamp": 1581348323634,
"packageName": "myrepo",
"packagePath": "",
"resolvedVersion": "1.2920",
"rootModuleName": "repo",
"rootPackagePath": ""
},
"eventKey": "myEvent",
"eventDefinition": {
"name": "myName",
"namespace": "myNamespace",
"meta": {
"description": "No description available",
"database": "myDatabase",
"owner": null,
"codeOwners": [
"Prionti Nasir"
],
"imgSrc": null,
"isPublic": null,
"yamlSrc": {
"packageName": "my-package",
"packageVersion": "static-1.2920",
"relativePath": "something.yaml"
}
},
"properties": {
"userId": {
"type": "number",
"options": null,
"isOptional": false,
"description": null
}
},
"class": "interaction"
}
}
]
}
I will exclude the buildInfo and a few other fields, so I created a mapping accordingly:
{
"settings": {
"index": {
"number_of_replicas": "2",
"number_of_shards": "25",
"analysis": {
"analyzer": {
"autocomplete": {
"filter": [
"lowercase",
"autocomplete_filter"
],
"tokenizer": "standard",
"type": "custom"
},
"prefixMatch": {
"filter": [
"lowercase"
],
"tokenizer": "standard",
"type": "custom"
}
},
"filter": {
"autocomplete_filter": {
"min_gram": "3",
"max_gram": "10",
"type": "edge_ngram"
}
}
}
}
},
"mappings": {
"usageLibraryEventType": {
"dynamic": false,
"properties": {
"eventDatabase": {
"properties": {
"name": {
"type": "string"
}
},
"enabled": "false"
},
"eventType": {
"type": "string",
"analyzer": "autocomplete"
},
"name": {
"type": "string",
"analyzer": "autocomplete"
},
"namespace": {
"type": "string",
"analyzer": "autocomplete"
},
"latestBuildTimestamp": {
"type": "long"
},
"flattenedEventProperties": {
"type": "string"
},
"eventDefinitions": {
"properties": {
"eventKey": {
"type": "string",
"index": "not_analyzed"
},
"eventDefinition": {
"properties": {
"name": {
"type": "string",
"index": "no"
},
"namespace": {
"type": "string",
"index": "no"
},
"meta": {
"properties": {
"description": {
"type": "string",
"analyzer": "prefixMatch"
},
"owner": {
"type": "string",
"index": "not_analyzed",
"null_value" : "N/A"
},
"codeOwners": {
"type": "string",
"index": "not_analyzed",
"null_value" : "N/A"
}
}
},
"class": {
"type": "string",
"index": "not_analyzed"
}
}
}
}
}
}
}
}
}
This is giving me a MapperParsingException. eventDefinitions is supposed to be a list of json objects each of which will contain buildInfo, eventKey and eventDefinition. eventDefinition further contains json objects as you can see.
@POST
public UsageLibraryEventType indexEventType(UsageLibraryEventType usageLibraryEventType) {
elasticsearchIndexerClient.addDocumentRequest(
new IndexRequest(config.getUsageLibrarySourceTopic(), TYPE)
.source(
"eventDatabase", usageLibraryEventType.getEventDatabase(),
"eventType", usageLibraryEventType.getUsageLibraryEventType(),
"name", usageLibraryEventType.getName(),
"namespace", usageLibraryEventType.getNamespace(),
// fix these field names
"latestBuildTimestamp", usageLibraryEventType.getLatestBuildTimestamp(),
"flattenedEventProperties", usageLibraryEventType.getFlattenedEventProperties(),
"eventDefinitions", usageLibraryEventType.getEventDefinitions()),
config.getUsageLibrarySourceTopic())
.join();
return usageLibraryEventType;
}
(eventDefinitions is a list of EventDefinitionWithBuildInfo, and each EventDefinitionWithBuildInfo contains buildInfo, eventKey and eventDefinition. EventDefinition further contains a few fields and an object called meta. Although I have mapped all of this out in the mapping, I don't explicitly hand over values for each of the fields to the last branch in the tree. There's no way for me to enter each field in each eventDefinitionWithBuildInfo and then in eventDefinition separately of course, so I have to give it the list, as a result of which it does not get mapped all the way to the last unit. What can I do about this? Should I define new types called EventDefinitionWithBuildInfo and EventDefinition ?
@IanGabes is a savior. I spent my weekend crying and trying to get this working. But my object is so nested, so layered, and I was expecting ES to detect the inner fields on its own. I showed this to so many people and they were clueless. Then, as I was hopelessly watching Rick and Morty with chips falling off of my mouth, @IanGabes came like an angel and asked me to simply use Jackson to convert my object to Json, and that it will not deserialize to the last unit the way I was doing it. Thank you. You rock, I suck.