I am trying to encrypt a text file in Python 3.6 using python-gnupg, and a public key provided by a client, for which they have a private key to decrypt it with. I don't have access to that key. Despite python-gnupg appearing to successfully encrypt the file (though with some confusing errors appearing in the log), the client is unable to decrypt it. We're told the error they're getting is gpg: decryption failed: No secret key
When we tested encrypting a file using Cryptophane (different computer, running Windows instead of Ubuntu) and the same public key, they were able to decrypt it. This is how the encryption was successfully done manually for months. When testing the same code with our company public key, we were able to decrypt it using our private key and Cryptophane.
I've googled extensively for the error messages and general problem, and haven't found anything that seemed to be the same problem getting solved.
Here's the relevant code. filepath is the relative path to the file to be encrypted. pgp_key_name is the name of the .asc file containing the public key. pgp_key_dir is the directory it's in.
def pgp_encrypt_file(filepath, pgp_key_name, pgp_key_dir):
gpg = gnupg.GPG()
output_full_filepath = filepath + '.pgp'
try:
with open(pgp_key_dir + pgp_key_name) as file:
key_data = file.read()
import_result = gpg.import_keys(key_data)
logger.info(msg='Public key imported: {}'.format(pgp_key_name))
public_keys = gpg.list_keys()
fingerprint = public_keys[0]['fingerprint']
logger.info(msg='Attempting to encrypt file: ' +
output_full_filepath)
with open(filepath, 'r') as f:
newfile = f.read()
status = gpg.encrypt(newfile, fingerprint,
output=output_full_filepath)
logger.info(msg='status.ok : ' + str(status.ok))
logger.info(msg='status.status : ' + str(status.status))
except FileNotFoundError as e:
logger.error(msg='File not found: ' + str(e))
except TypeError as e:
logger.error(msg='GNUPG TypeError: ' + str(e))
return output_full_filepath
And the relevant section of the logs:
03-01 15:18:58 gnupg INFO Setting homedir to
'/home/[user]/.config/python-gnupg'
03-01 15:18:58 gnupg ERROR Could neither invoke nor terminate a
gpg process... Are you sure you specified the corrent (and full) path to the
gpg binary?
(That error did NOT appear later, and I was unable to find anything relevant on Google or Stack Overflow for it.)
03-04 09:04:39 gnupg WARNING Ignoring '/usr/bin/gpg' (path is a symlink)
03-04 09:04:39 gnupg ERROR Could not find binary for 'gpg'.
03-04 09:04:39 gnupg INFO Setting homedir to
'/home/[user]/.config/python-gnupg'
03-04 09:04:39 gnupg INFO
Initialised settings:
binary: /usr/bin/gpg2
binary version: `2.0.14\ncfg:pubkey:1;16;17\ncfg:cipher:2;3;4;7;8;9;10;11;12;13\ncfg:ciphername:3DES;CAST5;BLOWFISH;AES;AES192;AES256;TWOFISH;CAMELLIA128;CAMELLIA192;CAMELLIA256\ncfg:digest:1;2;3;8;9;10;11\ncfg:digestname:MD5;SHA1;RIPEMD160;SHA256;SHA384;SHA512;SHA224\ncfg:compress:0;1;2;3\n'
homedir: /home/[user]/.config/python-gnupg
ignore_homedir_permissions: False
keyring: /home/[user]/.config/python-gnupg/pubring.gpg
secring: /home/[user]/.config/python-gnupg/secring.gpg
default_preference_list: SHA512 SHA384 SHA256 AES256 CAMELLIA256 TWOFISH
AES192 ZLIB ZIP Uncompressed
keyserver: hkp://wwwkeys.pgp.net
options: None
verbose: False
use_agent: False
03-04 09:04:39 gnupg INFO Importing: [first few lines of public key]
03-04 09:04:39 root INFO Public key imported: [name of key]
03-04 09:04:39 root INFO Attempting to encrypt file: [file]
03-04 09:04:39 gnupg INFO Writing encrypted output to file:
[file.pgp]
03-04 09:04:39 gnupg INFO Encrypted output written successfully.
Some thoughts and things we've tried:
Though there is a gpg binary in /usr/bin/gpg, we're using a conda virtual environment for the project itself, which I think may be messing this up. However, when I ran this code from the command line, with the environment deactivated, I ended up with the same result. I see that the log file says that it couldn't find the gpg binary, and that it's ignoring a symlink pointing to it, but all of its status messages thereafter seemed to indicate that the encrytion was fine, and again, it worked just fine multiple times with a different public/private key pair.
Examining the pgp object in the IDE once instantiated leads me to think that it found the gpg binary just fine, even without passing any parameters to gnupg.GPG(). Passing in gnupghome='/usr/bin/gpg' leads me to the same place, and passing in gnupghome='not/real/path throws an error.
Setting armor=False on the call to encrypt did not change anything.
I really appreciate any and all thoughts on the matter.
If the answer is that it's just not looking in the right directories for the gpg binary or homedir, due to our virtual environment settings, recommendations on how to work around that would also be appreciated.
Resolved.
In this case, it was the client's error. We later attempted to encrypt the file using a variety of slightly different options, including many done from the command line, and from Python. They were able to decrypt every single one.
For the sake of helping some others down the line, here are a few things that I've learned since starting on this journey:
There are two distinct packages both named python-gnupg.
Since these packages share a name, it is very confusing when googling errors in one or the other. Doing pip install python-gnupg seems to always download the second one. My experience is almost entirely with this second one, so keep that in mind when reading everything else in this post.
On CentOS 6, /usr/bin/gpg is a symlink that points to /usr/bin/gpg2. Python-GNUPG logs errors noting this, but then it seems to find /usr/bin/gpg2 just fine.
Regarding the error Could neither invoke nor terminate a gpg process...: While this concerned me, this also appears to have had no effect at all on any functionality. Your mileage may vary.
Compatibility issues are possible between the Python-GNUPG version and gpg binary version. This can lead to Unknown status message: [SOME-GPG-MESSAGE] errors; for example: Unknown status message: PINENTRY_LAUNCHED which I believe arises when gpg tries to bring up the passphrase prompt (which it does not do in older versions!). If you are NOT trying to make a module with different uses on different OSes (we were), you can try your luck with manually editing the python-gnupg source code once you pip install the package. Specifically, in pretty_bad_protocol._parsers.py in the _handle_status method, there is a tuple of known status messages; just add in any of the "unknown" status messages there, and that error won't trip in the future. I mean, you're on your own after that, but it was something that we tried and it doesn't appear to have harmed anything.
Best of luck to anyone trying to do pgp encryption in the future.