How to do large file parallel encryption using GnuPG and GNU parallel?

Question

I'm trying to write a parallel compress / encrypt backup script for archiving using GNU parallel, xz and GnuPG. The core part's of script is:

tar --create --format=posix --preserve-permissions --same-owner --directory $BASE/$name --to-stdout . \
    | parallel --pipe --recend '' --keep-order --block-size 128M "xz -9 --check=sha256 | gpg --encrypt --recipient $RECIPIENT" \
    | pv > $TARGET/$FILENAME

Without GnuPG encryption, it works great (uncompress and untar works), but after adding parallel encryption, it's fail to decrypt with below error:

[don't know]: invalid packet (ctb=0a)
gpg: WARNING: encrypted message has been manipulated!
gpg: decrypt_message failed: Unexpected error
: Truncated tar archive
tar: Error exit delayed from previous errors.

Because uncompressed size is as same as gnu parallel's block size(around 125M), I assume that it's related GnuPG's support of partial block encryption. How can I solve this problem?

---

FYI

Another parallel gpg encrption issue about random number generation

https://unix.stackexchange.com/questions/105059/parallel-pausing-and-resuming

Answer 1

Pack

tar --create --format=posix --preserve-permissions --same-owner --directory $BASE/$name --to-stdout . |
    parallel --pipe --recend '' --keep-order --block-size 128M "xz -9 --check=sha256 | gpg --encrypt --recipient $RECIPIENT;echo bLoCk EnD" |
    pv > $TARGET/$FILENAME

Unpack

cat $TARGET/$FILENAME |
  parallel --pipe --recend 'bLoCk EnD\n' -N1 --keep-order --rrs 'gpg --decrypt | xz -d' |
  tar tv

-N1 is needed to make sure we pass a single record at a time. GnuPG does not support decrypting multiple merged records.