Search code examples
djangogmailpostfix-mtadkim

Django + Postfix + Gmail + openDKIM -avamis > dkim = neutral (body hash did not verify)


I have followed this tutorial to configure DKIM and Postfix on Debian 7 wheezy. These instructions are pretty much a standard on the interwebz.

I am using Gmail to send and receive emails using my own domain. I followed this instructions to achieve that.

My problem

I can send and receive emails but I can't manage to pass the DKIM test (at least with Gmail). After searching and struggling for a while I have come to the conclusion that the reason of my woes is that my message is getting multiple DKIM signatures (see mail.log below). And this, according to the DKIM directives is enough for the DKIM to fail.

But after reading about on how to solve the multiple signatures problem I found that absolutely all of these solutions refer to having 'amavis' installed. Thing is...I don't have it installed!

In any case these solutions mention changing postfix configurations related to the milters in master.cnf and/or main.cfn. For example, adding this to the 'receive_override_options' (again, I don't have that variable since I don't have amavis installed) should solve the issue:

receive_override_options=no_unknown_recipient_checks,no_header_body_checks,no_milters

Or, another solution is commenting global settings in main.cf...

#smtpd_milters = inet:localhost:12301
#non_smtpd_milters = inet:localhost:12301

...And then adding the milter directive to the "smtpd" and post-amavis services for inbound authentication and outbound signing respectively:

# inbound messages from internet
# will be authenticated by OpenDKIM milter on port 12301
smtp      inet  n       -       -       -       -       smtpd
 .......
  -o smtpd_milters=inet:localhost:12301

# outbound messages have been through amavis
# will be signed by OpenDKIM milter on port 12301
127.0.0.1:10025 inet    n       -       -       -       -       smtpd
 .......
  -o smtpd_milters=inet:localhost:12301

Alas, none of this works for me because I don't have the amavis installed. So what I think is happening is that the Django layer messes with Postfix somehow so the email message gets twice DKIM signed by opendkim (see mail.log below).

This is Gmail response:

ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816;
        h=message-id:date:to:from:subject:content-transfer-encoding
         :mime-version:dkim-signature:arc-authentication-results;
        bh=rFbauTH/rtd1+kK8TxaFUe3HjRRJjkoamWIx2IdGVtM=;
        b=MKXMH0s3t4rJtnbq1NTX/3Pu7WroJ1/QcMEyAMdQQhF4pFM1imdRTA==
ARC-Authentication-Results: i=1; mx.google.com;
       dkim=neutral (body hash did not verify) header.i=@domain.com header.s=mail header.b=X2M3CvND;
       spf=pass (google.com: domain of error@domain.com designates 45.76.171.123 as permitted sender) smtp.mailfrom=error@domain.com

This is my mail.log. Notice how the 'DKIM signature header add' is executed two times: once after localhost connects and, another one after it reconnects again.

Aug 25 16:12:21 domain postfix/smtpd[29238]: connect from localhost[127.0.0.1]
Aug 25 16:12:21 domain postfix/smtpd[29238]: 745D37D599: client=localhost[127.0.0.1]
Aug 25 16:12:21 domain postfix/cleanup[29243]: 745D37D599: message-id=<20170825161221.28656.25384@localhost>
Aug 25 16:12:21 domain opendkim[27899]: 745D37D599: DKIM-Signature header added (s=mail, d=domain.com)
Aug 25 16:12:21 domain postfix/qmgr[29037]: 745D37D599: from=<error@domain.com>, size=44876, nrcpt=1 (queue active)
Aug 25 16:12:21 domain postfix/smtpd[29238]: disconnect from localhost[127.0.0.1]
Aug 25 16:12:21 domain postfix/smtpd[29238]: connect from localhost[127.0.0.1]
Aug 25 16:12:21 domain postfix/smtpd[29238]: 8E8287D5C0: client=localhost[127.0.0.1]
Aug 25 16:12:21 domain postfix/cleanup[29243]: 8E8287D5C0: message-id=<20170825161221.28656.34673@localhost>
Aug 25 16:12:21 domain opendkim[27899]: 8E8287D5C0: DKIM-Signature header added (s=mail, d=domain.com)
Aug 25 16:12:21 domain postfix/qmgr[29037]: 8E8287D5C0: from=<error@domain.com>, size=44876, nrcpt=1 (queue active)
Aug 25 16:12:21 domain postfix/smtpd[29238]: disconnect from localhost[127.0.0.1]
Aug 25 16:12:22 domain postfix/smtp[29244]: 745D37D599: to=<user@gmail.com>, orig_to=<error@domain.com>, relay=gmail-smtp-in.l.google.com[74.125.28.26]:25, delay=0.61, delays=0.05/0.02/0.12/0.41, dsn=2.0.0, status=sent (250 2.0.0 OK 1503677542 r29si5009980pfd.56 - gsmtp)
Aug 25 16:12:22 domain postfix/qmgr[29037]: 745D37D599: removed
Aug 25 16:12:22 domain postfix/smtp[29245]: 8E8287D5C0: to=<user@gmail.com>, orig_to=<error@domain.com>, relay=gmail-smtp-in.l.google.com[74.125.28.26]:25, delay=0.51, delays=0.05/0.01/0.09/0.36, dsn=2.0.0, status=sent (250 2.0.0 OK 1503677542 t196si4944733pgc.158 - gsmtp)
Aug 25 16:12:22 domain postfix/qmgr[29037]: 8E8287D5C0: removed

Probably the localhost user seen in the mail.log is the one set up in my Django's settings.py

EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'
EMAIL_HOST = 'localhost'
EMAIL_PORT = 25
EMAIL_HOST_USER = ''
EMAIL_HOST_PASSWORD = ''
EMAIL_USE_TLS = False

My /etc/postfix/master.cf

smtp      inet  n       -       -       -       -       smtpd
pickup    fifo  n       -       -       60      1       pickup
cleanup   unix  n       -       -       -       0       cleanup
qmgr      fifo  n       -       n       300     1       qmgr
tlsmgr    unix  -       -       -       1000?   1       tlsmgr
rewrite   unix  -       -       -       -       -       trivial-rewrite
bounce    unix  -       -       -       -       0       bounce
defer     unix  -       -       -       -       0       bounce
trace     unix  -       -       -       -       0       bounce
verify    unix  -       -       -       -       1       verify
flush     unix  n       -       -       1000?   0       flush
proxymap  unix  -       -       n       -       -       proxymap
proxywrite unix -       -       n       -       1       proxymap
smtp      unix  -       -       -       -       -       smtp
relay     unix  -       -       -       -       -       smtp
showq     unix  n       -       -       -       -       showq
error     unix  -       -       -       -       -       error
retry     unix  -       -       -       -       -       error
discard   unix  -       -       -       -       -       discard
local     unix  -       n       n       -       -       local
virtual   unix  -       n       n       -       -       virtual
lmtp      unix  -       -       -       -       -       lmtp
anvil     unix  -       -       -       -       1       anvil
scache    unix  -       -       -       -       1       scache
maildrop  unix  -       n       n       -       -       pipe
  flags=DRhu user=vmail argv=/usr/bin/maildrop -d ${recipient}
uucp      unix  -       n       n       -       -       pipe
  flags=Fqhu user=uucp argv=uux -r -n -z -a$sender - $nexthop!rmail ($recipient)
ifmail    unix  -       n       n       -       -       pipe
  flags=F user=ftn argv=/usr/lib/ifmail/ifmail -r $nexthop ($recipient)
bsmtp     unix  -       n       n       -       -       pipe
  flags=Fq. user=bsmtp argv=/usr/lib/bsmtp/bsmtp -t$nexthop -f$sender $recipient
scalemail-backend unix  -   n   n   -   2   pipe
  flags=R user=scalemail argv=/usr/lib/scalemail/bin/scalemail-store ${nexthop} ${user} ${extension}
mailman   unix  -       n       n       -       -       pipe
  flags=FR user=list argv=/usr/lib/mailman/bin/postfix-to-mailman.py
  ${nexthop} ${user}

My /etc/postfix/main.cf

smtpd_banner = $myhostname ESMTP $mail_name (Debian/GNU)
biff = no
append_dot_mydomain = no
readme_directory = no
smtpd_tls_cert_file = /etc/postfix/server.pem
smtpd_tls_key_file = $smtpd_tls_cert_file
smtpd_use_tls=yes
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
myhostname = automatones.com
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
myorigin = /etc/mailname
mydestination = localhost.localdomain, localhost, domain.com
relayhost = 
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = all
inet_protocols = ipv4
virtual_alias_maps = hash:/etc/postfix/virtual
smtpd_sasl_type = dovecot
smtpd_sasl_auth_enable = yes
queue_directory = /var/spool/postfix
smtpd_sasl_path = private/auth
smtpd_sasl_auth_enable = yes
smtpd_sasl_type = dovecot
milter_protocol = 2
milter_default_action = accept
smtpd_milters = inet:localhost:12301
non_smtpd_milters = inet:localhost:12301
virtual_alias_domains = domain.com

My http://dkimvalidator.com/results:

SpamAssassin Score: 0.472
Message is NOT marked as spam
Points breakdown: 
-0.0 SPF_HELO_PASS          SPF: HELO matches SPF record
 0.1 DKIM_SIGNED            Message has a DKIM or DK signature, not necessarily valid
 0.4 RDNS_DYNAMIC           Delivered to internal network by host with
                            dynamic-looking rDNS
 0.0 T_DKIM_INVALID         DKIM-Signature header exists but is not valid

So how can I tweak the postfix configuration for not to trigger the multiple DKMI header signature additions I see in the mail.log? Or how do I configure Django settings for not to trigger this connect and reconnect behaviour?

Any pointers, ideas or suggestions are welcome!

Edit: I found this note in the README file of OPENDMARC Could I get a solution out of this? If so, how can I start implementing it?

(c) If you have a content filter in master.cf that feeds it back into a different smtpd process, you should alter the second smtpd process in     master.cf to contain '-o receive_override_options=no_milters' to    prevent messages being signed or verified twice.  For tips on avoiding DKIM signature breakage, see:    http://www.postfix.org/MILTER_README.html#workarounds

Solution

  • I managed to fix this some time ago after several trails and errors. Problem is I don't remember what exactly I did to solve it. I was playing with several parameters -sometimes simultaneously- without keeping any track (my bad). However, at some point everything worked!

    Here are my configuration files. Hope they can guide/help someone.

    /etc/postfix/master.cf

    #
    # Postfix master process configuration file.  For details on the format
    # of the file, see the master(5) manual page (command: "man 5 master").
    #
    # Do not forget to execute "postfix reload" after editing this file.
    #
    # ==========================================================================
    # service type  private unpriv  chroot  wakeup  maxproc command + args
    #               (yes)   (yes)   (yes)   (never) (100)
    # ==========================================================================
    smtp      inet  n       -       -       -       -       smtpd
      -o content_filter=spamassassin
    #smtp      inet  n       -       -       -       1       postscreen
    #smtpd     pass  -       -       -       -       -       smtpd
    #dnsblog   unix  -       -       -       -       0       dnsblog
    #tlsproxy  unix  -       -       -       -       0       tlsproxy
    submission inet n       -       -       -       -       smtpd
      -o syslog_name=postfix/submission
      -o smtpd_tls_security_level=encrypt
      -o smtpd_sasl_auth_enable=yes
      -o smtpd_recipient_restrictions=permit_mynetworks,permit_sasl_authenticated,reject
      -o milter_macro_daemon_name=ORIGINATING
      -o content_filter=spamassassin
      -o smtpd_sasl_type=dovecot
      -o smtpd_sasl_path=private/auth
    smtps     inet  n       -       -       -       -       smtpd
      -o syslog_name=postfix/smtps
      -o smtpd_tls_wrappermode=yes
      -o smtpd_sasl_auth_enable=yes
      -o smtpd_client_restrictions=permit_sasl_authenticated,reject
      -o milter_macro_daemon_name=ORIGINATING
      -o content_filter=spamassassin
    #628       inet  n       -       -       -       -       qmqpd
    pickup    fifo  n       -       -       60      1       pickup
    cleanup   unix  n       -       -       -       0       cleanup
    qmgr      fifo  n       -       n       300     1       qmgr
    #qmgr     fifo  n       -       n       300     1       oqmgr
    tlsmgr    unix  -       -       -       1000?   1       tlsmgr
    rewrite   unix  -       -       -       -       -       trivial-rewrite
    bounce    unix  -       -       -       -       0       bounce
    defer     unix  -       -       -       -       0       bounce
    trace     unix  -       -       -       -       0       bounce
    verify    unix  -       -       -       -       1       verify
    flush     unix  n       -       -       1000?   0       flush
    proxymap  unix  -       -       n       -       -       proxymap
    proxywrite unix -       -       n       -       1       proxymap
    smtp      unix  -       -       -       -       -       smtp
    relay     unix  -       -       -       -       -       smtp
    #       -o smtp_helo_timeout=5 -o smtp_connect_timeout=5
    showq     unix  n       -       -       -       -       showq
    error     unix  -       -       -       -       -       error
    retry     unix  -       -       -       -       -       error
    discard   unix  -       -       -       -       -       discard
    local     unix  -       n       n       -       -       local
    virtual   unix  -       n       n       -       -       virtual
    lmtp      unix  -       -       -       -       -       lmtp
    anvil     unix  -       -       -       -       1       anvil
    scache    unix  -       -       -       -       1       scache
    #
    # ====================================================================
    # Interfaces to non-Postfix software. Be sure to examine the manual
    # pages of the non-Postfix software to find out what options it wants.
    #
    # Many of the following services use the Postfix pipe(8) delivery
    # agent.  See the pipe(8) man page for information about ${recipient}
    # and other message envelope options.
    # ====================================================================
    #
    # maildrop. See the Postfix MAILDROP_README file for details.
    # Also specify in main.cf: maildrop_destination_recipient_limit=1
    #
    maildrop  unix  -       n       n       -       -       pipe
      flags=DRhu user=vmail argv=/usr/bin/maildrop -d ${recipient}
    #
    # ====================================================================
    #
    # Recent Cyrus versions can use the existing "lmtp" master.cf entry.
    #
    # Specify in cyrus.conf:
    #   lmtp    cmd="lmtpd -a" listen="localhost:lmtp" proto=tcp4
    #
    # Specify in main.cf one or more of the following:
    #  mailbox_transport = lmtp:inet:localhost
    #  virtual_transport = lmtp:inet:localhost
    #
    # ====================================================================
    #
    # Cyrus 2.1.5 (Amos Gouaux)
    # Also specify in main.cf: cyrus_destination_recipient_limit=1
    #
    #cyrus     unix  -       n       n       -       -       pipe
    #  user=cyrus argv=/cyrus/bin/deliver -e -r ${sender} -m ${extension} ${user}
    #
    # ====================================================================
    # Old example of delivery via Cyrus.
    #
    #old-cyrus unix  -       n       n       -       -       pipe
    #  flags=R user=cyrus argv=/cyrus/bin/deliver -e -m ${extension} ${user}
    #
    # ====================================================================
    #
    # See the Postfix UUCP_README file for configuration details.
    #
    uucp      unix  -       n       n       -       -       pipe
      flags=Fqhu user=uucp argv=uux -r -n -z -a$sender - $nexthop!rmail ($recipient)
    #
    # Other external delivery methods.
    #
    ifmail    unix  -       n       n       -       -       pipe
      flags=F user=ftn argv=/usr/lib/ifmail/ifmail -r $nexthop ($recipient)
    bsmtp     unix  -       n       n       -       -       pipe
      flags=Fq. user=bsmtp argv=/usr/lib/bsmtp/bsmtp -t$nexthop -f$sender $recipient
    scalemail-backend unix  -   n   n   -   2   pipe
      flags=R user=scalemail argv=/usr/lib/scalemail/bin/scalemail-store ${nexthop} ${user} ${extension}
    mailman   unix  -       n       n       -       -       pipe
      flags=FR user=list argv=/usr/lib/mailman/bin/postfix-to-mailman.py
      ${nexthop} ${user}
    
    dovecot   unix  -       n       n       -       -       pipe
      flags=DRhu user=email:email argv=/usr/lib/dovecot/deliver -f ${sender} -d ${recipient}
    
    policy-spf  unix  -     n       n       -       -       spawn
         user=nobody argv=/usr/bin/policyd-spf
    
    spamassassin unix -     n       n       -       -       pipe
      user=debian-spamd argv=/usr/bin/spamc -f -e /usr/sbin/sendmail -oi -f ${sender} ${recipient}
    

    /etc/postfix/main.cf

    # See /usr/share/postfix/main.cf.dist for a commented, more complete version
    
    
    # Debian specific:  Specifying a file name will cause the first
    # line of that file to be used as the name.  The Debian default
    # is /etc/mailname.
    #myorigin = /etc/mailname
    
    smtpd_banner = $myhostname ESMTP $mail_name (Debian/GNU)
    biff = no
    
    # appending .domain is the MUA's job.
    append_dot_mydomain = no
    
    # Uncomment the next line to generate "delayed mail" warnings
    #delay_warning_time = 4h
    
    readme_directory = no
    
    # Network information
    myhostname = mail.mysite.com
    mydomain = mysite.com
    myorigin = /etc/mailname
    mydestination = $myhostname, $mydomain, localhost.localdomain, localhost
    relayhost = 
    mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
    mailbox_size_limit = 0
    recipient_delimiter = +
    inet_interfaces = all
    
    # Local alias map
    alias_maps = hash:/etc/aliases
    alias_database = hash:/etc/aliases
    
    queue_directory=/var/spool/postfix
    
    # SSL
    smtpd_tls_cert_file = /etc/letsencrypt/live/mail.mysite.com/fullchain.pem
    smtpd_tls_key_file = /etc/letsencrypt/live/mail.mysite.com/privkey.pem
    smtpd_use_tls=yes
    smtpd_tls_auth_only=yes
    smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
    smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
    smtpd_tls_security_level = may
    smtp_tls_security_level = may
    smtp_tls_loglevel = 1
    smtpd_tls_loglevel = 1
    smtpd_tls_protocols = !SSLv2, !SSLv3
    
    # SPF
    policy-spf_time_limit = 3600s
    
    # https://www.digitalocean.com/community/tutorials/how-to-set-up-a-postfix-e-mail-server-with-dovecot
    #local_recipient_maps = proxy:unix:passwd.byname $alias_maps
    
    # Virtual alias mapping
    virtual_alias_domains = $mydomain
    virtual_alias_maps = hash:/etc/postfix/virtual
    
    # Mail will be stored in users ~/Maildir directories
    home_mailbox = Maildir/
    mailbox_command =
    
    # From http://wiki2.dovecot.org/HowTo/PostfixAndDovecotSASL
    smtpd_sasl_type = dovecot
    smtpd_sasl_path = private/auth
    smtpd_sasl_auth_enable = yes
    
    # DKIM & DMARC
    milter_default_action = accept
    milter_protocol = 6
    smtpd_milters = inet:127.0.0.1:12345, inet:127.0.0.1:8893
    non_smtpd_milters = inet:127.0.0.1:12345, inet:127.0.0.1:8893
    
    # Require a valid HELO or EHLO command with a fully qualified domain name to stop common spambots 
    smtpd_helo_required = yes
    smtpd_helo_restrictions = reject_non_fqdn_helo_hostname,reject_invalid_helo_hostname,reject_unknown_helo_hostname
    
    # Disable the VRFY command 
    disable_vrfy_command = yes
    
    # Reject message to allow Postfix to log recipient address information when the connected client breaks any of the reject rules
    smtpd_delay_reject = yes
    
    # Reject connections from made up addresses that do not use a FQDN or don't exist. Add external spam filters like Spamhaus or CBL blacklists
    # Also add SPF policy (after reject_unauth_destination and after permit_sasl_authenticated)
    smtpd_recipient_restrictions = permit_sasl_authenticated,permit_mynetworks,reject_unauth_destination,check_policy_service unix:private/policy-spf,reject_invalid_hostname,reject_non_fqdn_hostname,reject_non_fqdn_sender,reject_non_fqdn_recipient,reject_unknown_sender_domain,reject_rbl_client sbl.spamhaus.org,reject_rbl_client cbl.abuseat.org
    
    # https://serverfault.com/questions/559088/postfix-not-accepting-relay-from-localhost
    default_transport = smtp
    relay_transport = relay