Search code examples
logstashlogstash-configuration

Block unauthorized and distinguish users

Logstash has an open port where everyone can send in data.

  1. Anonymous data messes everything up
  2. All data from all customers in one pool is messed up too.

So I read and tried https://www.elastic.co/guide/en/x-pack/5.6/logstash.html (also for Version 6 and 7) but this does not seem to be it. It rather authenticates against elasticsearch than in front of logstash. What I like to have is some sort of this:

input {
    # One port to rule them all - possible?
    tcp {
        port => 5000
    }
}

output {
    elasticsearch {
        hosts => "elasticsearch:9200"
        user => elastic
        password => nope
        document_id => "%{[@metadata][fingerprint]}"
        # Here comes the user prefix again.
        index => "%{[user]}-%{[host]}-%{+YYYY.MM.dd}"
    }
    if [user] == "foo" {
        # Also put things in IRC
    }
    if [user] == "qux" {
        # Forward somewhere else
    }
}

In the end we like to have the data separated available when using Elastic or Kibana which may be no big deal when I read the documentation. But I also think that there should be some auth in front of logstash. Correct me if I am wrong.

If not:

  1. How to prevent anonymous data?
  2. How to distinguish them well?
Solution

  • Since you can't change the original message, you will need to filter based on the host that sent that message, you can use the host field for that, as it seems you are already doing on your output.

    To prevent anonymous data you can use SSL, so each machine sending to your logstash will need to have the certificate files, or you can configure a firewall on the logstash machine and configure it to allow connections from your costumers only.

    The x-pack security feature that you mentioned is a paid feature, but its focus is on putting a security layer on elasticsearch/kibana, there is no auth in front of logstash, to do that you need firewall rules and/or ssl certificates.