There is some info regarding a Iframe, https://robocorp.com/docs/development-guide/browser/how-to-work-with-iframes
But there is no information regarding how to reach a modal.
What also works is
use a
Execute Javascript .click
How to execute JavaScript in Robot Framework
it sort of focuses the modal and then after that the keywords can be used again.