Search code examples
javajakarta-mailimap

Random "MessagingException: Failed to load IMAP envelope" looping throught Messages (server mail IMAP protocol bug)


I'm getting this random exception while looping through the Messages of my mailServer (is a hMailServer on local network):

javax.mail.MessagingException: Failed to load IMAP envelope
    at com.sun.mail.imap.IMAPMessage.loadEnvelope(IMAPMessage.java:1532)
    at com.sun.mail.imap.IMAPMessage.getSubject(IMAPMessage.java:441)

it happen while I for-loop the Message[] array from the server

The scope of (this part) of the Java program is pretty simple:

  • Read all Messages from the Inbox
  • Read Subject
  • Move to correct Folder

I've alredy tried this know fix: https://javaee.github.io/javamail/FAQ#imapserverbug (both the methods described)

the moveFromFolderTo() is where I apply the fix:

public static Message moveFromFolderTo(Message mail, String sourceFolder, String DestinationFolder) {

        try {
    //this is the javaMailFAQ fix
            MimeMessage tmp = new MimeMessage((MimeMessage) mail);
            ByteArrayOutputStream bos = new ByteArrayOutputStream();
            tmp.writeTo(bos);
            bos.close();
            SharedByteArrayInputStream bis = new SharedByteArrayInputStream(bos.toByteArray());
            MimeMessage cmsg = new MimeMessage(session, bis);
    //this is the javaMailFAQ fix

            List<Message> tempList = new ArrayList<>();
            tempList.add(cmsg);
            Message[] tempMessageArray = tempList.toArray(new Message[tempList.size()]);

            Folder srcFolder = store.getFolder(sourceFolder);
            srcFolder.open(Folder.READ_WRITE);
            Folder destFolder = store.getFolder(DestinationFolder);
            destFolder.appendMessages(tempMessageArray);
            mail.setFlag(Flags.Flag.DELETED, true);
            srcFolder.close(true);

            return cmsg;

the Message retrieved is the simplest for-loop:

Message messages[] = inboxFolder.getMessages();

but doesn't work, at random Message it crashes; the mailServer log doesn't help, at some point:

//for-loop start, Messages[] alredy fetched 
A8 FETCH 2 (ENVELOPE INTERNALDATE RFC822.SIZE)
A8 OK FETCH completed
javax.mail.MessagingException: Failed to load IMAP envelope
    at com.sun.mail.imap.IMAPMessage.loadEnvelope(IMAPMessage.java:1532)

any help?

Made another test without the JavaMail-FAQ fix, looks like mailServer and clients are "out of synch":

Mails in folder: 6  //mails from getMessages.length
cycle-step : 1
mailSubject: test4

cycle-step : 2
mailSubject: test
moveToWorking ex:java.lang.IndexOutOfBoundsException: 2 > 1
moveToWorking ex:java.lang.NullPointerException

cycle-step : 3
mailSubject: test4
moveToWorking ex:java.lang.IndexOutOfBoundsException: 3 > 2
moveToWorking ex:java.lang.NullPointerException

cycle-step :4
//after some mail the usual Exception
javax.mail.MessagingException: Failed to load IMAP envelope
    at com.sun.mail.imap.IMAPMessage.loadEnvelope(IMAPMessage.java:1532)
    at com.sun.mail.imap.IMAPMessage.getSubject(IMAPMessage.java:441)
    at com.horsa.antidatacap.main.Main.main(Main.java:31)

but consider that from Thunderbird i can see only 4 email in my inbox

full debug log (session.setDebug(true);) for this case:

DEBUG: setDebug: JavaMail version 1.6.2
DEBUG: getProvider() returning javax.mail.Provider[STORE,imaps,com.sun.mail.imap.IMAPSSLStore,Oracle]
DEBUG IMAPS: mail.imap.fetchsize: 16384
DEBUG IMAPS: mail.imap.ignorebodystructuresize: false
DEBUG IMAPS: mail.imap.statuscachetimeout: 1000
DEBUG IMAPS: mail.imap.appendbuffersize: -1
DEBUG IMAPS: mail.imap.minidletime: 10
DEBUG IMAPS: enable STARTTLS
DEBUG IMAPS: closeFoldersOnStoreFailure
DEBUG IMAPS: trying to connect to host "192.168.75.131", port 993, isSSL true
* OK IMAPrev1
A0 CAPABILITY
* CAPABILITY IMAP4 IMAP4rev1 CHILDREN IDLE QUOTA SORT ACL NAMESPACE RIGHTS=texk
A0 OK CAPABILITY completed
DEBUG IMAPS: protocolConnect login, host=192.168.75.131, [email protected], password=<non-null>
DEBUG IMAPS: mechanism PLAIN not supported by server
DEBUG IMAPS: mechanism LOGIN not supported by server
DEBUG IMAPS: mechanism NTLM not supported by server
DEBUG IMAPS: mechanism XOAUTH2 disabled by property: mail.imaps.auth.xoauth2.disable
DEBUG IMAPS: LOGIN command trace suppressed
DEBUG IMAPS: LOGIN command result: A1 OK LOGIN completed
A2 CAPABILITY
* CAPABILITY IMAP4 IMAP4rev1 CHILDREN IDLE QUOTA SORT ACL NAMESPACE RIGHTS=texk
A2 OK CAPABILITY completed
DEBUG IMAPS: connection available -- size: 1
A3 SELECT SourceFolder
* 4 EXISTS
* 2 RECENT
* FLAGS (\Deleted \Seen \Draft \Answered \Flagged)
* OK [UIDVALIDITY 1561546646] current uidvalidity
* OK [UIDNEXT 83] next uid
* OK [PERMANENTFLAGS (\Deleted \Seen \Draft \Answered \Flagged)] limited
A3 OK [READ-WRITE] SELECT completed
Mails in folder: 4  //mails from getMessages.length
cycle-step : 1
A4 FETCH 1 (FLAGS)
* 1 FETCH (FLAGS (\Seen))
A4 OK FETCH completed
A5 FETCH 1 (ENVELOPE INTERNALDATE RFC822.SIZE)
* 1 FETCH (RFC822.SIZE 625 INTERNALDATE "28-Jun-2019 12:57:18 +0200" ENVELOPE ("Fri, 28 Jun 2019 12:57:18 +0200" "email1" (("Surname, name" NIL "sender" "host.net")) (("Surname, name" NIL "sender" "host.net")) (("Surname, name" NIL "sender" "host.net")) (("[email protected]" NIL "destiMail" "host.net")) NIL NIL NIL "<[email protected]>"))
A5 OK FETCH completed
A6 FETCH 1 (BODY[]<0.16384>)
* 1 FETCH (BODY[]<0> {625}
Return-Path: [email protected]
Received: from [192.168.75.96] (******.*********.net [192.168.75.96])
    by INFINITYHR with ESMTPA
    ; Fri, 28 Jun 2019 12:56:08 +0200
To: [email protected]
From: "Surname, name" <[email protected]>
Subject: email1
Message-ID: <[email protected]>
Date: Fri, 28 Jun 2019 12:57:18 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101
 Thunderbird/60.7.2
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Content-Language: en-US




)
A6 OK FETCH completed
DEBUG IMAPS: no connections in the pool, creating a new one
* OK IMAPrev1
B0 CAPABILITY
* CAPABILITY IMAP4 IMAP4rev1 CHILDREN IDLE QUOTA SORT ACL NAMESPACE RIGHTS=texk
B0 OK CAPABILITY completed
DEBUG IMAPS: mechanism PLAIN not supported by server
DEBUG IMAPS: mechanism LOGIN not supported by server
DEBUG IMAPS: mechanism NTLM not supported by server
DEBUG IMAPS: mechanism XOAUTH2 disabled by property: mail.imaps.auth.xoauth2.disable
DEBUG IMAPS: LOGIN command trace suppressed
DEBUG IMAPS: LOGIN command result: B1 OK LOGIN completed
B2 CAPABILITY
* CAPABILITY IMAP4 IMAP4rev1 CHILDREN IDLE QUOTA SORT ACL NAMESPACE RIGHTS=texk
B2 OK CAPABILITY completed
B3 SELECT SourceFolder
* 4 EXISTS
* 2 RECENT
* FLAGS (\Deleted \Seen \Draft \Answered \Flagged)
* OK [UIDVALIDITY 1561546646] current uidvalidity
* OK [UIDNEXT 83] next uid
* OK [PERMANENTFLAGS (\Deleted \Seen \Draft \Answered \Flagged)] limited
B3 OK [READ-WRITE] SELECT completed
DEBUG IMAPS: no connections in the pool, creating a new one
* OK IMAPrev1
C0 CAPABILITY
* CAPABILITY IMAP4 IMAP4rev1 CHILDREN IDLE QUOTA SORT ACL NAMESPACE RIGHTS=texk
C0 OK CAPABILITY completed
DEBUG IMAPS: mechanism PLAIN not supported by server
DEBUG IMAPS: mechanism LOGIN not supported by server
DEBUG IMAPS: mechanism NTLM not supported by server
DEBUG IMAPS: mechanism XOAUTH2 disabled by property: mail.imaps.auth.xoauth2.disable
DEBUG IMAPS: LOGIN command trace suppressed
DEBUG IMAPS: LOGIN command result: C1 OK LOGIN completed
C2 CAPABILITY
* CAPABILITY IMAP4 IMAP4rev1 CHILDREN IDLE QUOTA SORT ACL NAMESPACE RIGHTS=texk
C2 OK CAPABILITY completed
C3 SELECT tempFolder
* 2 EXISTS
* 0 RECENT
* FLAGS (\Deleted \Seen \Draft \Answered \Flagged)
* OK [UIDVALIDITY 1561724024] current uidvalidity
* OK [UIDNEXT 28] next uid
* OK [PERMANENTFLAGS (\Deleted \Seen \Draft \Answered \Flagged)] limited
C3 OK [READ-WRITE] SELECT completed
B4 COPY 1 tempFolder
B4 OK COPY completed
A7 STORE 1 +FLAGS (\Deleted)
* 1 FETCH (FLAGS (\Deleted \Seen) UID 78)
A7 OK STORE completed
B5 CLOSE
* 1 FETCH (FLAGS (\Deleted \Seen) UID 78)
B5 OK CLOSE completed
DEBUG IMAPS: added an Authenticated connection -- size: 1
DEBUG IMAP: pool is full, not adding an Authenticated connection
C4 CLOSE
* 3 EXISTS
* 3 RECENT
C4 OK CLOSE completed
C5 LOGOUT
* BYE Have a nice day
C5 OK Logout completed
cycle-step : 2
A8 FETCH 2 (FLAGS)
* 2 FETCH (FLAGS (\Seen))
A8 OK FETCH completed
A9 FETCH 2 (ENVELOPE INTERNALDATE RFC822.SIZE)
* 2 FETCH (RFC822.SIZE 627 INTERNALDATE "28-Jun-2019 12:57:30 +0200" ENVELOPE ("Fri, 28 Jun 2019 12:57:30 +0200" "email3" (("Surname, name" NIL "sender" "host.net")) (("Surname, name" NIL "sender" "host.net")) (("Surname, name" NIL "sender" "host.net")) (("[email protected]" NIL "destiMail" "host.net")) NIL NIL NIL "<[email protected]>"))
A9 OK FETCH completed
A10 FETCH 2 (BODY[]<0.16384>)
* 2 FETCH (BODY[]<0> {627}
Return-Path: [email protected]
Received: from [192.168.75.96] (******.*********.net [192.168.75.96])
    by INFINITYHR with ESMTPA
    ; Fri, 28 Jun 2019 12:56:21 +0200
To: [email protected]
From: "Surname, name" <[email protected]>
Subject: email3
Message-ID: <[email protected]>
Date: Fri, 28 Jun 2019 12:57:30 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101
 Thunderbird/60.7.2
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Content-Language: en-US





)
A10 OK FETCH completed
DEBUG IMAPS: connection available -- size: 1
B6 SELECT SourceFolder
* 3 EXISTS
* 0 RECENT
* FLAGS (\Deleted \Seen \Draft \Answered \Flagged)
* OK [UIDVALIDITY 1561546646] current uidvalidity
* OK [UIDNEXT 83] next uid
* OK [PERMANENTFLAGS (\Deleted \Seen \Draft \Answered \Flagged)] limited
B6 OK [READ-WRITE] SELECT completed
DEBUG IMAPS: no connections in the pool, creating a new one
* OK IMAPrev1
D0 CAPABILITY
* CAPABILITY IMAP4 IMAP4rev1 CHILDREN IDLE QUOTA SORT ACL NAMESPACE RIGHTS=texk
D0 OK CAPABILITY completed
DEBUG IMAPS: mechanism PLAIN not supported by server
DEBUG IMAPS: mechanism LOGIN not supported by server
DEBUG IMAPS: mechanism NTLM not supported by server
DEBUG IMAPS: mechanism XOAUTH2 disabled by property: mail.imaps.auth.xoauth2.disable
DEBUG IMAPS: LOGIN command trace suppressed
DEBUG IMAPS: LOGIN command result: D1 OK LOGIN completed
D2 CAPABILITY
* CAPABILITY IMAP4 IMAP4rev1 CHILDREN IDLE QUOTA SORT ACL NAMESPACE RIGHTS=texk
D2 OK CAPABILITY completed
D3 SELECT tempFolder
* 2 EXISTS
* 0 RECENT
* FLAGS (\Deleted \Seen \Draft \Answered \Flagged)
* OK [UIDVALIDITY 1561724024] current uidvalidity
* OK [UIDNEXT 29] next uid
* OK [PERMANENTFLAGS (\Deleted \Seen \Draft \Answered \Flagged)] limited
D3 OK [READ-WRITE] SELECT completed
B7 COPY 2 tempFolder
B7 OK COPY completed
A11 STORE 2 +FLAGS (\Deleted)
* 2 FETCH (FLAGS (\Deleted \Seen) UID 81)
A11 OK STORE completed
B8 CLOSE
* 2 FETCH (FLAGS (\Deleted \Seen) UID 81)
B8 OK CLOSE completed
DEBUG IMAPS: added an Authenticated connection -- size: 1
DEBUG IMAP: pool is full, not adding an Authenticated connection
D4 CLOSE
* 3 EXISTS
* 3 RECENT
D4 OK CLOSE completed
D5 LOGOUT
* BYE Have a nice day
D5 OK Logout completed
cycle-step : 3
A12 FETCH 3 (FLAGS)
A12 OK FETCH completed
A13 FETCH 3 (ENVELOPE INTERNALDATE RFC822.SIZE)
A13 OK FETCH completed
moveToWorking ex:javax.mail.MessagingException: Failed to load IMAP envelope
javax.mail.MessagingException: Failed to load IMAP envelope
    at com.sun.mail.imap.IMAPMessage.loadEnvelope(IMAPMessage.java:1532)
cycle-step : 4
A14 FETCH 4 (FLAGS)
    at com.sun.mail.imap.IMAPMessage.getSize(IMAPMessage.java:511)
    at javax.mail.internet.MimeMessage.<init>(MimeMessage.java:245)
    at com.horsa.antidatacap.utilities.MailHandler.moveFromFolderTo(MailHandler.java:100)
    at com.horsa.antidatacap.main.Main.main(Main.java:31)
A14 OK FETCH completed
A15 FETCH 4 (ENVELOPE INTERNALDATE RFC822.SIZE)
A15 OK FETCH completed
moveToWorking ex:javax.mail.MessagingException: Failed to load IMAP envelope
javax.mail.MessagingException: Failed to load IMAP envelope
    at com.sun.mail.imap.IMAPMessage.loadEnvelope(IMAPMessage.java:1532)
    at com.sun.mail.imap.IMAPMessage.getSize(IMAPMessage.java:511)
    at javax.mail.internet.MimeMessage.<init>(MimeMessage.java:245)
    at com.horsa.antidatacap.utilities.MailHandler.moveFromFolderTo(MailHandler.java:100)
    at com.horsa.antidatacap.main.Main.main(Main.java:31)
fine
DEBUG IMAPS: IMAPStore cleanup, force false
DEBUG IMAPS: close folder
DEBUG IMAP: pool is full, not adding an Authenticated connection
A16 LOGOUT
* BYE Have a nice day
A16 OK Logout completed
B9 LOGOUT
* BYE Have a nice day
B9 OK Logout completed
DEBUG IMAPS: IMAPStore cleanup done

Solution

  • At least had to change mail server, i finally get it to work with Apache James; maybe a little more complicated than hMailServer but surly more stable.

    https://james.apache.org/server/3/quick-start.html