How can I efficiently search nested JSON objects in Elasticsearch and return only the relevant data without returning the entire document?
I have a JSON object representing a menu structure that will be ingested per user. Each menu item has nested fields such as Text, Url, Area, and Children. I would like to be able to search on any of these fields and return only the relevant data for the search query. For example, if I search for 'Workflow Roles', I would like to only return the relevant menu item and not all the extra data.
I have tried creating a nested mapping for the menu object, but my queries using inner hits return too much data. I have also experimented with path filters but have not found a satisfactory solution.
So my questions are:
Here is an example of the JSON object that needs to be searched:
[
{
"MenuStructure": 0,
"NavigationLinkId": 0,
"Children": [
{
"MenuStructure": 0,
"NavigationLinkId": 111,
"Text": "Bulk Actions",
"Action": "Index",
"Controller": "BulkUpload",
"Area": "Company",
"Url": "/some-url/bulk-upload",
}
]
},
{
"MenuStructure": 1,
"NavigationLinkId": 1,
"Children": [
{
"MenuStructure": 1,
"NavigationLinkId": 2,
"Text": "Company Config",
"Area": "Company",
"Url": "javascript:void(0);",
"Children": [
{
"MenuStructure": 1,
"NavigationLinkId": 0,
"Text": "Basic Settings",
"Url": "javascript:void(0);",
"Children": [
{
"MenuStructure": 1,
"NavigationLinkId": 60668,
"Text": "Company Settings",
"Icon": "fas fa-cogs",
"Url": "javascript:void(0);",
"Children": [
{
"MenuStructure": 1,
"NavigationLinkId": 27,
"Text": "Basic Company Information",
"Icon": "fa fa-building",
"Action": "Index",
"Controller": "CompanyProfile",
"Area": "Company",
"Url": "/some-url/company-profile",
},
{
"MenuStructure": 1,
"NavigationLinkId": 163,
"Text": "Reminder Contact Details",
"Icon": "fas fa-balance-scale-right",
"Url": "/some-url/classic/163",
}
]
},
{
"MenuStructure": 1,
"NavigationLinkId": 0,
"Text": "Dropdown Management",
"Icon": "fas fa-users-class",
"Url": "javascript:void(0);",
"Children": [
{
"MenuStructure": 1,
"NavigationLinkId": 261,
"Text": "Workflow Roles",
"Action": "Index",
"Controller": "CompanyWorkflowRoles",
"Area": "Company",
"Url": "/some-url/workflow-roles"
},
{
"MenuStructure": 1,
"NavigationLinkId": 50323,
"Text": "Other Dropdowns",
"Icon": "fas fa-balance-scale-right",
"Url": "/some-url/classic/50323"
}
]
},
{
"MenuStructure": 1,
"NavigationLinkId": 50327,
"Text": "Legislative Configurations",
"Icon": "fas fa-balance-scale-right",
"Url": "javascript:void(0);",
"Children": [
{
"MenuStructure": 1,
"NavigationLinkId": 244,
"Text": "Statistics SA Config",
"Url": "/some-url/classic/244"
}
]
}
]
},
{
"MenuStructure": 1,
"NavigationLinkId": 167,
"Text": "Security",
"Icon": "fa fa-lock",
"Url": "javascript:void(0);",
"Children": [
{
"MenuStructure": 1,
"NavigationLinkId": 167,
"Text": "Security",
"Icon": "fa fa-lock",
"Url": "javascript:void(0);",
"Children": [
{
"MenuStructure": 1,
"NavigationLinkId": 168,
"Text": "User Profiles",
"Url": "/some-url/classic/168",
},
{
"MenuStructure": 1,
"NavigationLinkId": 169,
"Text": "Security Roles",
"Url": "/some-url/classic/169",
},
{
"MenuStructure": 1,
"NavigationLinkId": 188,
"Text": "User Org. Unit Permissions",
"Url": "/some-url/classic/188",
},
{
"MenuStructure": 1,
"NavigationLinkId": 60610,
"Text": "Activate Users",
"Url": "/some-url/classic/60610",
}
]
}
]
}
]
}
]
}
]
To achieve this, I have tried creating a nested field type mapping for the menu object. Here is an example mapping:
PUT my-index
{
"mappings": {
"properties": {
"user_id": {
"type": "keyword"
},
"menu": {
"type": "nested",
"properties": {
"MenuStructure": {
"type": "long"
},
"NavigationLinkId": {
"type": "long"
},
"Text": {
"type": "text"
},
"Action": {
"type": "keyword"
},
"Controller": {
"type": "keyword"
},
"Area": {
"type": "keyword"
},
"Url": {
"type": "keyword"
},
"Icon": {
"type": "keyword"
},
"Children": {
"type": "nested",
"properties": {
"MenuStructure": {
"type": "long"
},
"NavigationLinkId": {
"type": "long"
},
"Text": {
"type": "text"
},
"Action": {
"type": "keyword"
},
"Controller": {
"type": "keyword"
},
"Area": {
"type": "keyword"
},
"Url": {
"type": "keyword"
},
"Icon": {
"type": "keyword"
},
"Children": {
"type": "nested",
"properties": {
"MenuStructure": {
"type": "long"
},
"NavigationLinkId": {
"type": "long"
},
"Text": {
"type": "text"
},
"Action": {
"type": "keyword"
},
"Controller": {
"type": "keyword"
},
"Area": {
"type": "keyword"
},
"Url": {
"type": "keyword"
},
"Icon": {
"type": "keyword"
},
"Children": {
"type": "nested",
"properties": {
"MenuStructure": {
"type": "long"
},
"NavigationLinkId": {
"type": "long"
},
"Text": {
"type": "text"
},
"Action": {
"type": "keyword"
},
"Controller": {
"type": "keyword"
},
"Area": {
"type": "keyword"
},
"Url": {
"type": "keyword"
},
"Icon": {
"type": "keyword"
},
"Children": {
"type": "nested",
"properties": {
"MenuStructure": {
"type": "long"
},
"NavigationLinkId": {
"type": "long"
},
"Text": {
"type": "text"
},
"Action": {
"type": "keyword"
},
"Controller": {
"type": "keyword"
},
"Area": {
"type": "keyword"
},
"Url": {
"type": "keyword"
},
"Icon": {
"type": "keyword"
}
}
}
}
}
}
}
}
}
}
}
}
}
}
How can I search for any of the children's nested properties, I tried the below and some other similar types of queries:
GET my-index/_search
{
"query": {
"nested": {
"path": "menu",
"query": {
"bool": {
"must": [
{"match":
{
"menu.Children.Area": "Company"
}
}
]
}
},
"inner_hits": {"highlight": {"fields": {"menu.Children.Area": {}}}}
}
}
}
the above query does not really work as its basically returning so much data. I only want to return the inner hits, for instance if you search for 'Workflow Roles' only return
{
"MenuStructure": 1,
"NavigationLinkId": 261,
"Text": "Workflow Roles",
"Action": "Index",
"Controller": "CompanyWorkflowRoles",
"Area": "Company",
"Url": "/some-url/workflow-roles"
}
not all the extra data. I have tried quite a few combination with path_filters etc. But I think I am going down the path.
You need multiple levels of nested fields (menu + all Children sub-fields) and design your query accordingly, then it will work as you expect.
The query will need to be designed in a way that each nested field needs its own nested query, so you'd probably have to do something like this:
{
"query": {
"bool": {
"minimum_should_match": 1,
"should": [
{
"nested": {
"path": "menu",
"query": {
"match": {
"menu.Text": "Company"
}
}
}
},
{
"nested": {
"path": "menu.Children",
"query": {
"match": {
"menu.Children.Text": "Company"
}
}
}
},
{
"nested": {
"path": "menu.Children.Children",
"query": {
"match": {
"menu.Children.Children.Text": "Company"
}
}
}
},
...
]
}
}
}