Get python getaddresses() to decode encoded-word encoding

Question

msg = \
"""To: =?ISO-8859-1?Q?Caren_K=F8lter?= <ck@example.dk>, bob@example.com
Cc: "James =?ISO-8859-1?Q?K=F8lter?=" <jk@example.dk>
Subject: hello

message body blah blah blah

"""

import email.parser, email.utils
import itertools


parser = email.parser.Parser()
parsed_message = parser.parsestr(msg)

address_fields = ('to', 'cc')
addresses = itertools.chain(*(parsed_message.get_all(field) for field in address_fields if parsed_message.has_key(field)))
address_list = set(email.utils.getaddresses(addresses))


print address_list

It seems like email.utils.getaddresses() doesn't seem to automatically handle MIME RFC 2047 in address fields.

How can I get the expected result below?

actual result:

set([('', 'bob@example.com'), ('=?ISO-8859-1?Q?Caren_K=F8lter?=', 'ck@example.dk'), ('James =?ISO-8859-1?Q?K=F8lter?=', 'jk@example.dk')])

desired result:

set([('', 'bob@example.com'), (u'Caren_K\xf8lter', 'ck@example.dk'), (u'James \xf8lter', 'jk@example.dk')])

Answer 1

The function you want is email.header.decode_header, which returns a list of (decoded_string, charset) pairs. It's up to you to further decode them according to charset and join them back together again before passing them to email.utils.getaddresses or wherever.

You might think that this would be straightforward:

def decode_rfc2047_header(h):
    return ' '.join(s.decode(charset or 'ascii')
                   for s, charset in email.header.decode_header(h))

But since message headers typically come from untrusted sources, you have to handle (1) badly encoded data; and (2) bogus character set names. So you might do something like this:

def decode_safely(s, charset='ascii'):
    """Return s decoded according to charset, but do so safely."""
    try:
        return s.decode(charset or 'ascii', 'replace')
    except LookupError: # bogus charset
        return s.decode('ascii', 'replace')

def decode_rfc2047_header(h):
    return ' '.join(decode_safely(s, charset)
                   for s, charset in email.header.decode_header(h))