python 3 smtplib: binary attachment encodes incorrectly in flask when gnupg is active

I am trying to send a binary attachment using the python 3.5.2 smtplib over TLS. My platform is OSX and I am using python installed from homebrew.

When I receive the attachment, the encoding appears to be munged. Instead of the original file that starts with this hex:

ffd8 ffe0 0010 4a46 4946 0001 0100 0001

i.e., <FF><D8><FF><E0>^@^PJFIF^@^A^A

the starting hex of my attachment as received has some weird base64 leftovers:

5c75 6463 6666 5c75 6463 6438 5c75 6463 6666 5c75 6463 6530 0010 4a46 4946 0001

i.e., \udcff\udcd8\udcff\udce0^@^PJFIF

This is a minimal case that fails, which is pretty much exactly what is in the official documentation with the exception of the addition of the TLS logic and gnupg:

import gnupg
gpg = gnupg.GPG('/path/to/gpg')

import smtplib
from email.mime.multipart import MIMEMultipart
from email.mime.image import MIMEImage

def send_my_email():
    msg = MIMEMultipart()
    msg['Subject'] = 'subject'
    msg['From'] = 'XXXX@gmail.com'
    msg['To'] = 'YYYY@gmail.com'
    with open('/tmp/image.jpg', mode='rb') as image_file:
        image = MIMEImage(image_file.read())
    msg.attach(image)
    s = smtplib.SMTP('smtp.gmail.com', 587)
    s.starttls()
    s.login('XXXX@gmail.com', 'password')
    s.send_message(msg)
    s.quit()

Based on another question on here, I tried this instead of send_message() and it also failed:

    s.sendmail('XXXX@gmail.com', ['YYYY@gmail.com'], msg.as_string())

I have also tried explicitly adding _subtype='jpg' when I init MIMEImage, adding a Content-Transfer-Encoding header, and adding a Content-Disposition header and none of those seemed to make a difference.

I have verified that I do not have a problem with my email client when it receives base64 encoded attachments from other clients.

I looked at the smtplib source and noticed that the way smtplib handles line separators looks a little odd and wonder if this is possibly related. (also see ref: https://bugs.python.org/issue14645)

Do I need to encode something differently, set something special for my platform, or is this a glitch? Thanks!

Update: This problem only exists when I am running Flask and does not occur outside of Flask. I am in the process of attempting to isolate the problem inside my Flask environment. I thought it might be flask_mail but removing that did not solve the issue. The below code fails when run from Flask on my system, but not if I run it from a shell script from within the same virtual environment and same python binary. I don't expect any answers at this point due to complexity but will leave this open for posterity.

Update 2: I have narrowed this problem down to an interaction with the gnupg library from https://github.com/isislovecruft/python-gnupg/. I have updated my minimal example to reflect this. This monkey-patch of python codecs in python-gnupug is responsible for the problem: codecs.register_error('strict', codecs.replace_errors)

Solution

This was caused due to a bad interaction between gnupg and smtplib + MIME. gnupg calls codecs.register_error('strict', codecs.replace_errors), which interferes with encoding carried out by other packages.

ref: https://github.com/isislovecruft/python-gnupg/issues/49