Search code examples
csvelasticsearchlogstashkibanaelk

Logstash how to get inputs from .CSV file?


I have just recently started learning Logstash (and the ELK stack) but I have been struggling to get inputs from a csv file in Logstash (the csv file is in the same directory as the .conf file).

I just have a very simple script. My script is as follows:

input {
    file {
        path => "C:\elastic\logstash-8.3.2\config\in.csv"
        start_position => beginning
    }
}

filter {
    csv {
        separator => ";"
        columns => ["Name","Deposit","Month"]
    }
    mutate {
        convert => {
            "Deposit" => "integer"
        }
    }
}

output {
    stdout {
        codec => rubydebug
    }

    elasticsearch {
        hosts => ["http://localhost:9200"]
        index => "randomname"
    }
}

The csv file is ";" seperated and is like this line by line (random values):

Name;Number;Month
Name;Number;Month
Name;Number;Month
Name;Number;Month
Name;Number;Month
Name;Number;Month
Name;Number;Month

When I run Logstash, there seems to be no errors but I don't see any inputs going to the console and/or ElasticSearch. Why would this be the case? Is there a simple way to fix this?


Solution

  • I changed my .conf file to the following and it seems to work.

    input {
        file {
            path => "C:/elastic/logstash-8.3.2/config/in.csv"
            start_position => "beginning"
            sincedb_path => "NULL"
        }
    }
    
    filter {
        csv {
            separator => ";"
            columns => ["name","deposit","month"]
        }
    }
    
    output {
        elasticsearch {
            hosts => "http://localhost:9200"
            index => "randomname"
        }
        stdout {}
    }