I'm running a sample Outlook add-in dialog application. It just displays a dialog with 2 buttons, does not have any logic nor it makes any calls to REST service.
If more than 5 minutes elapse before closing the dialog, Outlook's compose frame becomes unresponsive and the user is forced to refresh the entire page
From your screenshot and the code you linked to, it looks like you're using an On Send add-in.
No, the behavior you're seeing is a bug. Specially, OWA shouldn't stop responding after 5 minutes.
Thank you for reporting this issue. We apologize for any frustration or inconvenience we have caused with this issue. Please be patient as we work to fix this issue as quickly as we can.
For Execute Function add-in commands, the add-in is automatically closed after 5 minutes, if event.completed() hasn't been called before then. Any open dialog is closed at that time.
For On Send add-ins, my understanding is the behavior should be the same as Execute Function, except that the email wouldn't be sent in the case of a timeout.
It's not configurable. The 5-minute timeout applies to all Execute Function and On Send add-in commands.
For recommendations, assuming that OWA doesn't stop responding, I don't think there is a single best option, and it depends on your On Send scenario.