Search code examples
bashunixpipechecksum

Checksum and pipe an input simultaneously


I want to securely move a big file from one computer to another, where it have to be processed before being stored.

I thought to pipe file into ssh running the processing script.

local.example.com$ cat file | ssh remote.example.com process.sh

(If you have any idea better than mine, please suggest)

In the process script i want to both checksum and encrypt the file before saving it. And here comes the problem.

Solutions may be two:

  • pipe input into two commands (cksum and openssl); but all ways I found looked complicated and sub-optional.
  • hack cksum to also do a cat-like work and print result on stderr, so that I can do

    cksum --pipe | openssl enc > myfile
    

    and get the checksum back via stderr. Unfortunately, I looked into the code and it seems hard for me to do that without doing some performance/buffering damage ;)

There may be a cksumming-transfering tool that does this all, but it didn't come me to mind. Anyway I want to avoid non-standard and complex stuff.

Thanks a lot.

Edit: useful link about answer http://www.linuxjournal.com/content/shell-process-redirection


Solution

  • If you start off your receiving script with cat > inputfile, that will eat all STDIN until EOF, then your script can run any actions needed on inputfile.

    You can also use tee:

    echo foo | tee >(sha1sum) >(md5sum)
    foo
    d3b07384d113edec49eaa6238ad5ff00  -
    f1d2d2f924e986ac86fdf7b36c94bcdf32beec15  -