Search code examples
elasticsearchlogstashkibanakibana-4

multiple words visualization in a field


I have am trying to visualize kibana in a way that I can plot graphs where which customers added something to cart VS which customers removed stuff from the cart. I am grabbing this data from error logs.

here is a json example of the data for

{
  "_index": "filebeat-2016.03.08",
  "_type": "php-error",
  "_id": "AVNUOptErt39_iTb3Riw",
  "_score": null,
  "_source": {
    "message": "[Mon Mar 07 20:15:35.571673 2016] [:error] [pid 13829] [client 24.220.167.100:56888] {\"email\":null,\"name\":\" \",\"message\":\"remove from cart\",\"remaining_cart_items\":\"{\\\\\"total\\\\\":\\\\\"1 item(s) - $16.26\\\\\"}\"}, referer: https://exsite.com/",
    "@version": "1",
    "@timestamp": "2016-03-08T03:15:36.786Z",
    "beat": {
      "hostname": "ip-110-0-0-154",
      "name": "ip-110-0-0-154"
    },
    "count": 1,
    "fields": null,
    "input_type": "log",
    "source": "/var/log/apache2/error.log",
    "type": "log_format_error",
    "host": "ip-10-0-0-154"
  },
  "fields": {
    "@timestamp": [
      1457406936786
    ]
  },
  "sort": [
    1457406936786
  ]
}

you can see there is a "remove from cart" that I can use to search in dicover and then create a visual out of it. Here is add to cart data

{
  "_index": "filebeat-2016.03.08",
  "_type": "php-error",
  "_id": "AVNUOhKOrt39_iTb3Riv",
  "_score": null,
  "_source": {
    "message": "[Mon Mar 07 20:14:56.377612 2016] [:error] [pid 13839] [client 24.220.167.100:56882] {\"email\":null,\"name\":\" \",\"message\":\"Added to cart\",\"add_to_cart\":\"{\\\\\"success\\\\\":\\\\\"Success: You have added <a href=\\\\\\\\\\\\\"https:\\\\\\\\\\\\/\\\\\\\\\\\\/site.com\\\\\\\\\\\\/whitesting\\\\\\\\\\\\\">My Test<\\\\\\\\\\\\/a> to your <a href=\\\\\\\\\\\\\"https:\\\\\\\\\\\\/\\\\\\\\\\\\/site.com\\\\\\\\\\\\/index.php?route=checkout\\\\\\\\\\\\/cart\\\\\\\\\\\\\">shopping cart<\\\\\\\\\\\\/a>!\\\\\",\\\\\"total\\\\\":\\\\\"2 item(s) - $32.51\\\\\"}\"}, referer: https://exsite.com/dtesting",
    "@version": "1",
    "@timestamp": "2016-03-08T03:15:01.767Z",
    "beat": {
      "hostname": "ip-110-0-0-154",
      "name": "ip-110-0-0-154"
    },
    "count": 1,
    "fields": null,
    "input_type": "log",
    "source": "/var/log/apache2/error.log",
    "type": "log_format_error",
    "host": "ip-10-0-0-154"
  },
  "fields": {
    "@timestamp": [
      1457406901767
    ]
  } 
}

So now how to I plot this data where I can see how many counts of remove and how many Added to cart are there.

The query that I have in discovery is

"remove from cart" OR "Added to cart"

thanks


Solution

  • ideally you would parse the log lines using something like logstash, and therefor not have to resort to querying like you are, but if you must you could always use a filters aggregation.

    When you're building a visualization (let's say you're using a bar chart) you would create the following aggregates to visualize this data:

    1. A metric that counts the documents in each bucket (this is there by default)
    2. A bucket for the x-axis that is a date histogram on your time field
    3. A bucket that splits the bars using a filters aggregation. Then give this agg two filters, one with "remove from cart" and another with "added to cart"