I am using Spring Integration to read email from Outlook 365 (cloud) using IMAP inbound-channel-adapter.
Scenario: Target mailbox in Outlook 365 is doing virus scanning for new emails once arrived, during this scan outlook is detaching the attachment and attaching it again once virus scan is completed.
Problem:
Attachment is missing in very few cases (1 mail out of 50 approx), this is because of those emails are read by inbound-channel-adapter when the attachment is not available in outlook ( detached by virus scanner).
Question:
How can ensure the attachment was read every time? If I make the thread waiting for 2 mins inside handleMessage method, then will it block the reading of next email just arrived?
OR please let me know any other solution to handle this situation.
Spring-integration.xml:
<util:properties id="javaMailProperties">
<prop key="mail.imap.socketFactory.class">javax.net.ssl.SSLSocketFactory</prop>
<prop key="mail.imap.socketFactory.fallback">false</prop>
<prop key="mail.store.protocol">imaps</prop>
<prop key="mail.debug">${imap.debug}</prop>
<prop key="mail.imaps.partialfetch">false</prop>
<prop key="mail.imaps.fetchsize">102400</prop> <!-- 100KB, default is 16KB -->
</util:properties>
<mail:inbound-channel-adapter id="imapAdapter"
store-uri="${imap.uri}"
channel="recieveEmailChannel"
should-delete-messages="false"
should-mark-messages-as-read="true"
auto-startup="true"
simple-content="true"
auto-close-folder="true"
java-mail-properties="javaMailProperties">
<int:poller fixed-delay="${imap.polling.interval}" time-unit="SECONDS"/>
</mail:inbound-channel-adapter>
<int:channel id="recieveEmailChannel">
<int:interceptors>
<int:wire-tap channel="logger"/>
</int:interceptors>
</int:channel>
<int:logging-channel-adapter id="logger" level="DEBUG"/>
<int:service-activator input-channel="recieveEmailChannel" ref="emailReceiver" method="handleMessage"/>
This solution is based on @Artem Bilan's answer & comment. Thanks for the direction Arten :-).
Problem: Failed to read attachment intermittently because of Outlook's virus scanning.
Idea: Delay email reading by some seconds, hoping that virus scan will be completed by that time.
Solution: Using custom SearchTermStrategy and OlderTerm.
SideEffect of this solution: As discussed in comments of @Artem's answer, while polling for new email JavaMail & IMAP has to search ENTIRE INBOX for each poll and find match based on custom searchTermStatregy. In my case, this mailbox receives ~1000 mails per day, so after some months/year INBOX will grow in size, email poller has to read million+ mails per poll. This will be a huge performance hit. So we need to tackle this issue as well.
=================================================================
Solution-Step1: Use custom searchTermStatregy to delay email reading:
Reading emails which are at least 10 mins old and not read already.
/**
*
*/
package com.abc.xyz.receiver;
import javax.mail.Flags;
import javax.mail.Folder;
import javax.mail.search.AndTerm;
import javax.mail.search.FlagTerm;
import javax.mail.search.SearchTerm;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.integration.mail.SearchTermStrategy;
import org.springframework.stereotype.Service;
import com.sun.mail.imap.OlderTerm;
/**
* @author Sundararaj Govindasamy
*
*/
@Service
public class MySearchTermStrategy implements SearchTermStrategy {
Logger logger = LoggerFactory.getLogger(MySearchTermStrategy.class);
@Override
public SearchTerm generateSearchTerm(Flags supportedFlags, Folder folder) {
logger.info("entering MySearchTermStrategy");
SearchTerm older10MinsTerm = new OlderTerm(600);// in seconds
SearchTerm flagTerm = new FlagTerm(new Flags(Flags.Flag.SEEN), false);
SearchTerm[] searchTermArray = new SearchTerm[2];
searchTermArray[0] = older10MinsTerm;
searchTermArray[1] = flagTerm;
SearchTerm andTerm = new AndTerm(searchTermArray);
return andTerm;
}
}
In spring-integration.xml, configure custom searchTermStatregy as below.
<mail:inbound-channel-adapter id="imapAdapter"
store-uri="${imap.uri}"
channel="recieveEmailChannel"
should-delete-messages="false"
should-mark-messages-as-read="true"
auto-startup="true"
simple-content="true"
auto-close-folder="false"
search-term-strategy="mySearchTermStrategy"
java-mail-properties="javaMailProperties">
<int:poller fixed-delay="${imap.polling.interval}" time-unit="SECONDS"/>
</mail:inbound-channel-adapter>
======================================================================
Solution-Step2: Move emails which were read from INBOX to another folder daily at 10pm, so that email poller's load will be reduced.
Please note, Spring Integration framework don't have support for mail moving, When I tried with Spring Integration, it thrown below ClassCastException as discussed in this thread: https://stackoverflow.com/a/58033889/1401019
So I decided to use plain JavaMail APIs to move mails between folders as below. This Spring scheduler will run once per day and move all read mails from INBOX folder to SunTest folder as below.
/**
*
*/
package com.abc.xyz.receiver;
import java.util.Properties;
import javax.mail.Flags;
import javax.mail.Folder;
import javax.mail.Message;
import javax.mail.Session;
import javax.mail.Store;
import javax.mail.search.FlagTerm;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Component;
import com.sun.mail.imap.IMAPFolder;
/**
* @author Sundararaj Govindasamy
* Move read email from INBOX to another folder
* to improve INBOX search performance.
*/
@Component
public class EmailMover {
Logger logger = LoggerFactory.getLogger(EmailMover.class);
/**
* move all read mails everyday 10PM, after business hours.
*/
@Scheduled(cron = "0 0 22 * * *")
public void moveReadMails() {
logger.info("entering: moveReadMails Scheduler");
// create session object
Session session = this.getImapSession();
try {
// connect to message store
Store store = session.getStore("imaps");
store.connect("outlook.office365.com", 993, "documents.local@mycompany.onmicrosoft.com",
"thisispassword");
// open the inbox folder
IMAPFolder inbox = (IMAPFolder) store.getFolder("INBOX");
inbox.open(Folder.READ_WRITE);
// fetch messages
Message[] messageArr = inbox.search(new FlagTerm(new Flags(Flags.Flag.SEEN), true));
logger.info("No. of read emails in INBOX:{}", messageArr.length);
// read messages
for (int i = 0; i < messageArr.length; i++) {
Message msg = messageArr[i];
String subject = msg.getSubject();
logger.info("Subject:{}", subject);
}
Folder targetFolder = inbox.getStore().getFolder("SunTest");
inbox.moveMessages(messageArr, targetFolder);
logger.info("All read mails were moved successfully from INBOX to {} folder", targetFolder);
} catch (Exception e) {
logger.error("Exception while moving emails, exception:{}", e.getMessage());
}
logger.info("exiting: moveReadMails Scheduler");
}
/**
*
* @return
*/
private Session getImapSession() {
Properties props = new Properties();
props.setProperty("mail.store.protocol", "imaps");
props.setProperty("mail.debug", "true");
props.setProperty("mail.imap.host", "outlook.office365.com");
props.setProperty("mail.imap.port", "993");
props.setProperty("mail.imap.ssl.enable", "true");
Session session = Session.getDefaultInstance(props, null);
session.setDebug(true);
return session;
}
}