When enabling a second screen (screen 2) from System Settings > Display and Monitor, all open windows are instantly moved to screen 2, despite their Window-Specific settings being set to "Force screen 1", and despite screen 1 being designated as "Primary Screen".
This should be fixed now as of Plasma 5.26.