find certain files using glob pattern

Question

I'm using logstash to constantly look for a new files and stream the data to a different systems.

Right now I'm using this config:

input {

  file {
    type => "source"
    path => "/ebs/**/*-*csv"  # recursivly check subfolders
    sincedb_path => "/usr/local/logstash/sincedb"
    sincedb_write_interval => 1
    discover_interval => 4
    start_position => "beginning"
    stat_interval => 1
  }
}

It works fine but what I want to do is something like this:

path => "/ebs/**/*-[0-9]{10}.csv" 

Is it possible to include regex like this in the path?

Answer 1

Logstash File Input api only supports glob(Wildcard), as Unix shells support. For example, if you list files in Unix system by using glob pattern,

$ ls [0-9].csv

It can show the file "1.csv"

But when you add regular expression,

$ ls [0-9]{1}.csv

The system shows

ls: [0-9]{1}.csv: No such file or directory

So, If you want to add regular expression, maybe you need to match globbing pattern. Such as

path => "/ebs/*/*-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9].csv"

Hope this can help you.