No idea since when this was introduced but recently updated to Monterey and it started to appear.
How can I disable it? I prefer one editor per workspace. The new way conflicts with normal tab switching.
Either disable it globally in macOS system settings, or disable it for IntelliJ IDEA only by adding ide.mac.bigsur.window.with.tabs.enabled=false
in Help | Edit Custom Properties (IDE restart is required).