Why after set mapping, index return nothing?

Question

I am using Elasticsearch 7.12.0 , Logstash 7.12.0, Kibana 7.12.0 on Windows 10 x64. Logstash config file logistics.conf

input {
  jdbc {
    jdbc_driver_library => "D:\\tools\\postgresql-42.2.16.jar"
    jdbc_driver_class => "org.postgresql.Driver"
    jdbc_connection_string => "jdbc:postgresql://localhost:5433/ld"
    jdbc_user => "xxxx"
    jdbc_password => "sEcrET"
    schedule => "*/5 * * * *"
    statement => "select * from inventory_item_report();"
    }
}

filter {
  uuid {
    target => "uuid"
  }
}

output {
  elasticsearch {
    hosts => "http://localhost:9200"
    index => "localdist"
    document_id => "%{uuid}"
    doc_as_upsert => "true"
    }
}

Run logstash

logstash -f logistics.conf

If I don't set mapping explicit, the query

GET /localdist/_search
{
  "query": {
    "match_all": {}
  }
}

return many result.

My mappings

POST localdist/_mapping
{
  
}

DELETE /localdist

PUT /localdist
{
  
}

POST /localdist
{
  
}

PUT localdist/_mapping
{
  "properties": {
    "unt_cost": {
      "type": "double"
    },
    "ii_typ": {
      "type": "keyword"
    },
    "qty_uom_id": {
      "type": "keyword"
    },
    "prod_id": {
      "type": "keyword"
    },
    "root_cat_id": {
      "type": "keyword"
    },
    "uom": {
      "type": "keyword"
    },
    "product_name": {
      "type": "text"
    },
    "ii_id": {
      "type": "keyword"
    },
    "wght_uom_id": {
      "type": "keyword"
    },
    "iid_seq_id": {
      "type": "long"
    },
    "avai_diff": {
      "type": "double"
    },
    "invt_change_typ": {
      "type": "keyword"
    },
    "ccy": {
      "type": "keyword"
    },
    "exp_date": {
      "type": "date"
    },
    "req_amt": {
      "type": "text"
    },
    "pur_cost": {
      "type": "double"
    },
    "tot_pri": {
      "type": "long"
    },
    "own_pid": {
      "type": "keyword"
    },
    "doc_type": {
      "type": "keyword"
    },
    "ii_date": {
      "type": "date"
    },
    "fac_id": {
      "type": "keyword"
    },
    "shipment_type_id": {
      "type": "keyword"
    },
    "lot_id": {
      "type": "keyword"
    },
    "phy_invt_id": {
      "type": "keyword"
    },
    "facility_name": {
      "type": "text"
    },
    "amt_ohand_diff": {
      "type": "double"
    },
    "reason_id": {
      "type": "keyword"
    },
    "cat_id": {
      "type": "keyword"
    },
    "qty_ohand_diff": {
      "type": "double"
    },
    "@timestamp": {
      "type": "date"
    }
  }
}

run query

GET /localdist/_search
{
  "query": {
    "match_all": {}
  }
}

return nothing.

How to fix it, how to make explicit mappings works correctly?

Answer 1

If I got you right, you are indexing via logstash. Elastic then create the index if missing, indexes the documents, and tries to guess the mapping for your documents based on the very first documents.

TL;DR: You are DELETING your index containing the data by yourself.

With

DELETE /localdist

you are deleting the whole index including all data. After that, by issuing

PUT /localdist
{
  
}

you are re-creating the previously deleted index which is empty again. And at the end, you are setting the index mapping with

PUT localdist/_mapping
{
  "properties": {
    "unt_cost": {
      "type": "double"
    },
    "ii_typ": {
      "type": "keyword"
    },
    ...

Now, as you have an empty elastic index with a mapping set, start the logstash pipeline again. If your documents are matching the index mapping, the docs should start to appear very quickly.