Snakemake shell strange execution order

Question

I'm trying to download file via FTP and process it after. For example rule like this

rule all:
    output:
        test_input = "ref/" + test_file
    shell:
        """        
            ftp -ni {test_domain} <<-EOF 1> log.txt 2> log.txt
                user anonymous " "
                lcd ref
                cd {test_path}
                get {test_file}
                # quit
            EOF
            ls
        """

Here is two problems. First: In current code ls will be executed on ftp server side. Second: If I uncomment quit - none of commands I add after EOF will ever be executed. Even in regular shell script it works fine.

What am I missing?

Answer 1

It looks like the limit string is missed at the end of the here-doc, as mentioned by @tripleee. This causes the ls to be executed on the server through ftp and the quit command to stop further execution.

Snakemake will not strip leading whitespace in the string literal under the shell directive. To get your script working as expected, you could either remove leading whitespace in your shell directive or be sure that any leading whitespace in the here-doc are only literal tabs.

Here's an updated script that produces the expected output for me:

rule all:
    output:
        test_input = "ref/" + test_file
    shell:
        """
ftp -ni {test_domain} <<EOF 1> log.txt 2> log.txt
user anonymous " "
lcd ref
cd {test_path}
get {test_file}
quit
EOF
ls
"""