Search code examples
elasticsearchkuberneteslogstashfilebeat

Does ECK required logstash

I'm quite confused between ECK and ELK. AFIK ELK requires logstash for reading the logs and whether the logstash required for ECK also or only filebeat or metric beat is enough to get logs. Filebeat replaced logstash in case of ECK?

Solution

  • I can understand your confusion between ECK and ELK. Let me clarify the differences between them and how they handle log ingestion.

    1. ELK Stack:
    • ELK stands for Elasticsearch, Logstash, and Kibana. It's a popular open-source stack used for log analysis and visualization.
    • Elasticsearch is a distributed search and analytics engine that stores and indexes the log data.
    • Logstash is a data processing pipeline that ingests logs from various sources, processes them, and sends them to Elasticsearch for storage.
    • Kibana is a web-based visualization tool that allows users to explore and analyze the data stored in Elasticsearch.

    In the traditional ELK setup, Logstash plays a crucial role in parsing and transforming logs before they are indexed into Elasticsearch. Logstash is responsible for handling log data collection, filtering, and processing.

    1. ECK (Elastic Cloud on Kubernetes):
    • ECK is an abbreviation for Elastic Cloud on Kubernetes. It is a tool designed to deploy, manage, and operate the Elastic Stack (Elasticsearch, Kibana, Beats) on Kubernetes.
    • ECK brings the capabilities of the ELK Stack to Kubernetes, allowing you to deploy Elasticsearch and Kibana as native Kubernetes applications.
    1. Beats:
    • Beats are lightweight data shippers that can send various types of data to Elasticsearch directly, bypassing the need for Logstash in some cases.
    • Filebeat is a Beat designed to collect, ship, and centralize log data. It can replace Logstash in certain scenarios, as it can read log files, parse them, and send the data directly to Elasticsearch.
    • Metricbeat is another Beat that can collect and ship system and service-level metrics to Elasticsearch.

    So, to answer your questions:

    • In the context of ECK, Logstash is not required to get logs into Elasticsearch.
    • Instead, you can use Filebeat or Metricbeat to read logs and send them directly to Elasticsearch.

    Filebeat is typically used for log collection, while Metricbeat is used for collecting system metrics. Both can be used independently or together, depending on your specific use case.

    To summarize, ECK allows you to deploy Elasticsearch and Kibana on Kubernetes and use Beats like Filebeat or Metricbeat to send data to Elasticsearch, eliminating the need for Logstash in certain scenarios.