When you switch on a FOV symbol and switch the view with space bar, the orientation of the frame remain the same
Edit your FOV symbol, is "Lock to Celestial Pole" checked? It should be.
You're absolutely right. I didn't saw this option. Sorry. Works fine.