Modal JDialog gets hidden behind main frame on macOS Big Sur

Question

The modal dialog gets hidden behind the main frame when clicking somewhere on the frame.

The main frame is blocked (correctly), and the dialog can only be brought back in front by clicking on the main frame title bar.

Tested on a Mac mini (2018, Intel Core i5, macOS Big Sur 11.1)

Tested with AdoptOpenJDK 8 (both HotSpot and OpenJ9), AdoptOpenJDK 11, AdoptOpenJDK 15, Amazon Corretto 15 and Oracle JDK 16.

Can anyone else reproduce this problem?

import javax.swing.JButton;
import javax.swing.JDialog;
import javax.swing.JFrame;

public class Test {
  public static void main(String[] args) {
    JFrame frame = new JFrame();
    frame.getContentPane().setLayout(new FlowLayout());
    JButton button = new JButton("Open modal dialog");
    button.addActionListener(e -> actionOpenDialog(frame));
    frame.getContentPane().add(button);
    frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
    frame.setSize(600, 450);
    frame.setLocationRelativeTo(null);
    frame.setVisible(true);
  }

  private static void actionOpenDialog(JFrame frame) {
    JDialog dialog = new JDialog(/* parent */ frame, /* modal */ true);
    dialog.setDefaultCloseOperation(JDialog.DISPOSE_ON_CLOSE);
    dialog.setSize(300, 200);
    dialog.setLocationRelativeTo(dialog.getParent());
    dialog.setVisible(true);
  }
}

Bug report: JDK-8263801.

Answer 1

We have been encountered this problem for several years. It started around Java 1.8.151 and we still see it with Java 11. But we can reproduce it in some environments only. Version of macOs seems not relevant. Unfortunatly I'm not aware of any workaround. There is a already a ticket https://bugs.openjdk.org/browse/JDK-8236162 but nobody seems to care.

Edit 1: I can reproduce this issue when working remotely on the machine via TeamViewer.

Edit 2: Eventually we found a very hackish workaround to restore the window order periodically: (Note: The reflective access to getModalBlocker can be avoided by searching for the blocking dialog via getAllWindows())

Thread t = new Thread(() -> {
    while (true) {
        SwingUtilities.invokeLater(()->{
            try {
                final Method m = Window.class.getDeclaredMethod("getModalBlocker");
                 m.setAccessible(true);
                 Dialog dialog = (Dialog)m.invoke(myMainFrame);
                 if (dialog != null && dialog.isVisible()){
                     dialog.toFront();
                 }
             } catch (Throwable e) {
                 e.printStackTrace();
             }    
         });
         try {
             Thread.sleep(100);
         } catch (InterruptedException e) {
             e.printStackTrace();
         }
    }
}, "Restore-Window-Order-Worker");
t.setDaemon(true);
t.start();

Comments and better solutions are welcome.