Search code examples
elasticsearchelastic-stackelasticsearch-painless

Elastic search painless script issues during re indexing


I want to re-index older data of geoip to geopoints.

previous data contains location in this format.

"geoip": {
    "location": {
        "lon": 67.0703,
        "lat": 24.9206
    }
} 

I want to re-index location in geo point in array like this

"geoip": {
    "location": [lon, lat]
} 

this is mapping

PUT logs-audit-geopoint/_mapping/doc
{
  "properties": {
    "json":{
      "properties": {
        "geoip":{
          "properties":{
            "location": {
              "type": "geo_point"
            }
          }
        }
      }
    }
  }
}

this is my request to perform re indexing.

POST _reindex
{
  "source": {
    "index": "logs-audit"
  },
  "dest": {
    "index": "logs-audit-geopoint"
  },
  "script": {
     "source": "def geoip = ctx._source.json.geoip; if(geoip != null) { geoip.location = [geoip.longitude, geoip.latitude]; }",
      "lang": "painless"
    }
}

Question it does not overriding location: {} to location: []


Solution

  • Used temporary index to tranform the data:

    logs-audit(source)
    test(temporary)
    logs-audit-geopoint(destination)

    Process of migration step by step:

    1- Transfer from source index [logs-audit] to [test] index(empty with no mapping) with new variable location_new.

        POST _reindex
        {
          "source": {
            "index": "logs-audit"
          },
          "dest": {
            "index": "test"
          },
          "script": {
             "source": "def geoip = ctx._source.json.geoip; if(geoip != null && geoip != '' ) { geoip.location = null; geoip.location_new = [geoip.longitude, geoip.latitude] }",
              "lang": "painless"
            }
        }
    

    2- Create new index with following mapping(geoip.location = geo_point)

        PUT logs-audit-geopoint/_mapping/doc
        {
          "properties": {
            "json":{
              "properties": {
                "geoip":{
                  "properties":{
                    "location": {
                      "type": "geo_point"
                    }
                  }
                }
              }
            }
          }
        }
    

    3- Then transfer from test index to new index.

        POST _reindex
        {
          "source": {
            "index": "test"
          },
          "dest": {
            "index": "logs-audit-geopoint"
          },
          "script": {
             "source": "def geoip = ctx._source.json.geoip; if(geoip != null && geoip != '' ) { geoip.location = geoip.location_new }",
              "lang": "painless"
            }
        }