The recommended way to reference officejs is via CDN script at the <head>.
If we do this in the taskpane html and call Office.context.ui.displayDialogAsync, the target url is opened in a nonmodal non-iframe "dialog".
If I place the same reference officejs in this target url, it fails, resulting in the dialog closing immediately, i.e.
<!--- Placing this script causes errors in the target dialog url, causing the dialog to close immediately -->
<head>
<script src="https://appsforoffice.microsoft.com/lib/1/hosted/office.js"></script>
</head>
I've since added an issue on officejs and created a minimal reproduction of the problem via scriptlab.
But in the event someone more knowledgeable can answer it here, it'd be much appreciated.
I've tried excluding office reference from the dialog url and just navigating to a subsequent url that contains the office reference but this fails as well.
In case this is able to help someone later. I do not think it is documented or maybe it's implied by the use of officejs in an iframe context.
Apparently, my localhost was serving the dialog url with the response header Cross-Origin-Opener-Policy: same-origin-allow-popups, once i switched this off, the dialog url was able to properly load the OfficeJS reference in the head.