Search code examples
arraysbashsshlsssh-tunnel

Get an array of all files in a remote folder using SSH in BASH


I'm trying to get an array of all files in a remote folder using bash and SSH:

declare -a existing_files=$(ssh -q -i $SSH_KEY_PATH -t $PROXY_SERVER \
ssh -q -i ubuntu_vm $REMOTE_SERVER "ls $RAW_EEG_FOLDER")

but I only get the last filename. If I just run the command: ssh -q -i $SSH_KEY_PATH -t $PROXY_SERVER ssh -q -i ubuntu_vm $REMOTE_SERVER "ls $RAW_EEG_FOLDER" in a terminal window, it return all filenames but when I try to assign it to a variable, I only get. the last one. What am I missing?

UPDATE:

I updated my code with Fravadona's answer to this:

#!/bin/bash

echo $(bash --version)

source 'secrets/secrets.env'
declare -a existing_files;
readarray -t -d '' existing_files < <(
    ssh -q -i $SSH_KEY_PATH -t $PROXY_SERVER ssh -q -i ubuntu_vm $REMOTE_SERVER  "$(
        printf '%q ' find "$RAW_EEG_FOLDER" -mindpeth 1 -maxdepth 1 \
            '(' -type d -printf '%f/\0' ')' -o -printf '%f\0'
    )"
)
echo ${existing_files}

But I'm getting this error:

GNU bash, version 5.2.15(1)-release (aarch64-apple-darwin22.1.0)
Copyright (C) 2022 Free Software Foundation, Inc. License GPLv3+:
GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This
is free software; you are free to change and redistribute it. There
is NO WARRANTY, to the extent permitted by law.

# The important bit here:
bash: -c: line 0: `find /home/.../ -mindpeth 1 -maxdepth 1 
( -type d -printf %f/\0 ) -o -printf %f\0'

Solution

  • UPDATE

    For bash < 4.3 (also added @CharlesDuffy suggestion)

    #!/bin/bash
    
    printf -v remote_cmd '%q ' find "$raw_egg_folder" -mindepth 1 -maxdepth 1 \( -type d -printf '%f/\0' \) -o -printf '%f\0'
    
    existing_files=()
    
    while IFS='' read -r -d '' filename
    do
        existing_files+=("$filename")
    done < <(
        ssh ... "$remote_ubuntu_server"  "$remote_cmd"
    )
    

    OLD ANSWER

    Maybe with:

    #!/bin/bash
    
    readarray -d '' existing_files < <(
        ssh ... "$remote_ubuntu_server"  "$(
            printf '%q ' find "$raw_egg_folder" -mindepth 1 -maxdepth 1 \
                \( -type d -printf '%f/\0' \) -o -printf '%f\0'
        )"
    )
    

    notes:

    • Requires bash >= 4.3 for readarray -d
    • Doesn't work if the remote shell isn't bash and $raw_egg_folder contains control characters
    • Bonus: the directories in $raw_egg_folder, if any, will have a / appended at the end.
    • BTW, you should downcase your shell variables and double-quote any expansion