I am currently investigating the ELK (Elasticsearch, Logstash, Kibana) stack for centralized log file analysis.
The plan is to store logs of multiple applications in the same Elasticsearch cluster using logstash and day-based indexes.
All documents contain a field called application, e.g."application": "superapp".
Now we are looking for a way to implement access control like this:
A) Superuser: is able to see log entries of all applications.
B) Developer: can only see log entries of the applications he is allowed to. For example the dev team for application "superapp" should only be able to see the entries for this application.
To wrap it up: we need access control based on the value in the field application.
While reading the documentation for Elastisearch and Shield I could not find an obvious way to do it.
Any ideas how we could realize this in a way that would also work with Kibana 3 and 4?
My first idea was to use aliases which are being automatically assigned to documents using index templates. I am wondering if this is the right direction.
I asked this question here on the elasticsearch Google Group and got this reply:
"You can separate out the different types of logs into their own indices which would make things much easier, you could also setup an alias with a filter and then provide access to that alias to certain users. Currently KB isn't multi-tenanted but it is a feature that is going to be added, you'd have to setup multiple instances with each going to their own alias."
To sum it up: multi-tenancy needs to addressed at the frontend (Kibana) and the backend (Elasticsearch).
Frontend: Use Proxies for Kibana
https://github.com/salyh/elastic-defender
https://github.com/fangli/kibana-authentication-proxy
Backend: Several approaches using filtered alias and alias templates
Limiting Indexes and Operations
Faking Index per User with Aliases -
http://engineering.aweber.com/using-elasticsearchs-aliases/
http://opennomad.com/content/controlling-access-elasticsearch-filtered-aliases-nginx-and-tokens