Using regex in logstash to filter files by name

Question

I am pulling in a series of files and using logstash to filter out only the ones i need based on a regex before outputting the matched files to a named directory, but the regex is not working and nothing is being filtered out.

output {
    if [filename] =~ /.*[abc|def].*/
 {
        file {
        path => "/my/directory/%{filename}-%{+YYYY-MM-dd}.log"
        codec => line { format => "%{message}"}
        }
    }
}

Incoming files are of the format:

P01_abc_stdout.log-2020-11-10-11.log
P01_rtf_stdout.log-2020-11-10-11.log
P01_ccc_stdout.log-2020-11-10-11.log
P01_def_stdout.log-2020-11-10-11.log
P01_ces_stdout.log-2020-11-10-11.log

but all of these files have come through, even though I'd expect it only to match

P01_abc_stdout.log-2020-11-10-11.log
P01_def_stdout.log-2020-11-10-11.log

Answer 1

if [filename] =~ /.*[abc|def].*/

Square brackets define a character group, so that will match any file that contains one of the characters a through f, or pipe. All of your filenames contain stdout, which includes a d, so they all match. The leading and trailing .* are not needed, since the pattern is not anchored. Try

if [filename] =~ /(abc|def)/