Search code examples
linuxbashpipestdoutlftp

How to unbuffer and capture output of LFTP EOF command?


I have the following command to mirror some files with LFTP:

unbuffer lftp $PROTOCOL://$URL -u ${USER},${PASS} << EOF > ${LOGSTDOUT}
set dns:fatal-timeout never
set sftp:auto-confirm yes
set mirror:use-pget-n 50
set mirror:parallel-transfer-count 2
set mirror:parallel-directories yes
set mirror:include-regex $REGEX
set log:enabled/xfer yes
set log:file/xfer $LOG
set xfer:use-temp-file yes
set xfer:temp-file-name *.lftp
mirror -c -v --loop --Remove-source-dirs "$REMOTEDIR" "$LOCALDIR"
quit
EOF

I am trying to capture the output of this command. I already output to a log file using xfer, but this sadly doesn't show the handy progress % that I desire.

Any thoughts? My above attempt with unbuffer is unsuccessful.


Solution

  • Thanks to the suggestion of @pjh I have come to the following solution using script:

    env TERM=dumb script -a $LOGSTDOUT -c "$(cat <<- EOF
        lftp $PROTOCOL://$URL -u ${USER},${PASS} << EOFF
        set dns:fatal-timeout never
        set sftp:auto-confirm yes
        set mirror:use-pget-n 50
        set mirror:parallel-transfer-count 2
        set mirror:parallel-directories yes
        set mirror:include-regex $REGEX
        set log:enabled/xfer yes
        set log:file/xfer $LOG
        set xfer:use-temp-file yes
        set xfer:temp-file-name *.lftp
        mirror -c -v --loop --Remove-source-dirs "$REMOTEDIR" "$LOCALDIR"
        quit
        EOFF
    EOF
    )"
    

    The env TERM=dumb is a random line I found that removes ANSI escape codes from the output to file.