Downloading files using wildcards with scp in Ruby

Question

I am successfully downloading the log files in "log_list" from the server to destination using the code below. I am using net/ssh / net/scp : (https://github.com/net-ssh/net-scp)

log_list = ["ini.log", "syslog.log", "network.log"]
fetch(log_list)


    def fetch(fetchlogs)
        root_cli do |ssh|
    
          fetchlogs.each do |testlog|
              source= "/var/log/logfiles/#{testlog}"
              destination = "/home/Documents/testdir/"
              
              ssh.scp.download!(source,destination)
    
            rescue Exception => e
              info "#{fetch} is not a valid file ==++==++ "
            end
        end
      end

Now the log files in "log_list" are rotating logs and can have multiple files like "ini.log ,ini.log.001, ini.log.002, ..." and i want to download all what i tried is with *:

log_list = ["ini.log*", "syslog.log*", "network.log*"]

which normally i thought should work but this is resulting in no file with exception.

ini.log* is not a valid file ==++==++
syslog.log* is not a valid file ==++==++
network.log* is not a valid file ==++==++

I also tried with adding * directly to source instead in log_list

source = "/var/log/logfiles/#{testlog}*"
pp " source #{source}"

ssh.scp.download!(source,"/home/Documents/testdir/")

output:

source ini.log*
source syslog.log*
source network.log*

but files are not being downloaded with same error

ini.log* is not a valid file ==++==++
syslog.log* is not a valid file ==++==++
network.log* is not a valid file ==++==++

Although files exits, Does somebody have any idea, is * character is being escaped ? !!

Answer 1

You can use scp.session.exec! with ls to gather a list of files based on your wildcard choice, and then download each source file one by one.

Something along these lines:

log_list = ["ini.log*", "syslog.log*", "network.log*"]

...
fetchlogs.each do |testlog|
  source = "/var/log/logfiles/#{testlog}"
  destination = "/home/Documents/testdir/"
  
  # Fetch list of wildcard files using ls
  source_files = ssh.scp.session.exec!("ls #{source}").split
  
  source_files.each { |file|         
    ssh.scp.download!(file, destination)
  }

  ...
end