Prevent entering fullscreen while a dialog is open (fixes input lockup)

[trasbd]

Fixes an issue where the window becomes unresponsive after toggling fullscreen while a dialog is open (mostly seen via controller hotkey).

Reproduce:

• Open a dialog (Config → Input and hotkeys works)
• Enter fullscreen (controller hotkey)
• Close the dialog while still fullscreen
• Exit fullscreen

The window then can't be clicked and behaves like a popup still has focus.

Fix:

Block entering fullscreen if a modal dialog or popup is active.

Exiting fullscreen is still allowed so you don't get stuck.

Fixes #1504