Join elasticsearch indices while matching fields in nested/inner objects

I am trying to join 2 elasticsearch indices by using terms filter lookup. I referred to http://www.elasticsearch.org/blog/terms-filter-lookup/ and http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-terms-filter.html. These Examples lookup on an array of fields like "followers" : ["1", "3"] and join works fine for similar data.

My requirement is to join with a field inside an array of objects. When I extend the above example to include an array of objects, my query fails. Following is the sample data:

PUT /users/user/2 {
   "followers" : [
  {
    "userId":"1",
    "username":"abc",
    "location":"xyz"
   },
   {
    "userId":"3",
    "username":"def",
    "location":"xyz"
   }
}
]
}

PUT /tweets/tweet/1 {
   "user" : "2"
}

PUT /tweets/tweet/2 {
   "user" : "1"
}

I am now trying to find tweets that are created by followers of user 2

POST /tweets/_search {
  "query" : {
"filtered" : {
  "filter" : {
    "terms" : {
      "user" : {
        "index" : "users",
        "type" : "user",
        "id" : "2",
        "path" : "followers.userId"
      },
      "_cache_key" : "user_2_friends"
    }
  }
}
  }
}

My search results are 0 for above query. I tried 2 other approaches as well 1)declare the followers object as a nested object during mapping and use "nested" in the query, 2)tried to add a match query for followers.userId after giving path as "followers". None yielded results.

Does terms filter lookup support array of objects? Any pointers to solving my problem would be of great help

Solution

Clear the indices if you have any

curl -XDELETE "http://example.com:9200/currencylookup/"

curl -XDELETE "http://example.com:9200/currency/"

Create the lookup table

curl -XPUT http://example.com:9200/currencylookup/type/2 -d '
{ "conv" : [ 
{  "currency":"usd","username":"abc", "location":"USA" }, 
{  "currency":"inr", "username":"def", "location":"India" },
{  "currency":"IDR", "username":"def", "location":"Indonesia" }]
}'

Lets put some dummy docs

curl -XPUT "http://example.com:9200/currency/type/USA" -d '{ "amount":"100", "currency":"usd", "location":"USA" }'

curl -XPUT "http://example.com:9200/currency/type/JPY" -d '{ "amount":"50", "currency":"JPY", "location":"JAPAN" }'

curl -XPUT "http://example.com:9200/currency/type/INR" -d '{ "amount":"50", "currency":"inr", "location":"INDIA" }'

curl -XPUT "http://example.com:9200/currency/type/IDR" -d '{ "amount":"30", "currency" : "IDR", "location": "Indonesia" }'

Time to check the output

curl http://example.com:9200/currency/_search?pretty -d '{
   "query" : {
 "filtered" : {
   "filter" : {
     "terms" : {
       "currency" : {
         "index" : "currencylookup",
         "type" : "type",
         "id" : "2",
         "path" : "conv.currency"
       },
       "_cache_key" : "currencyexchange"
     }
   }
 }
   }
 }'

Results

# curl http://example.com:9200/currency/_search?pretty -d '{
   "query" : {
 "filtered" : {
   "filter" : {
     "terms" : {
       "currency" : {
         "index" : "currencylookup",
         "type" : "type",
         "id" : "2",
         "path" : "conv.currency"
       },
       "_cache_key" : "currencyexchange"
     }
   }
 }
   }
 }'
{
  "took" : 2,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "failed" : 0
  },
  "hits" : {
    "total" : 2,
    "max_score" : 1.0,
    "hits" : [ {
      "_index" : "currency",
      "_type" : "type",
      "_id" : "INR",
      "_score" : 1.0,
      "_source":{ "amount":"50", "currency":"inr", "location":"INDIA" }
    }, {
      "_index" : "currency",
      "_type" : "type",
      "_id" : "USA",
      "_score" : 1.0,
      "_source":{ "amount":"100", "currency":"usd", "location":"USA" }
    } ]
  }
}

Conclusion

Capital letters are culprit here.

You can see 'IDR' is in caps so the match is failed for it and 'JPY' is not in look up even if it was there it would not have got matched because it is in caps.

cross matching values must be in small letters or numbers like

eg:

abc
1abc