Under certain circumstances, changing the height of the document viewport can cause the scroll position to change by a significant amount, e.g. about half a page or so, for no intuitive reason.

Example steps to reproduce:
1. Load a large PDF document with a TOC.
2. Use the TOC sidebar to jump to a specific page.
3. Press Ctrl+M to toggle the menu bar. The scroll position will jump by about half a page, a much larger amount than the height of the menu bar.

As an alternative to toggling the menu bar, toggling the sidebar pane works too (probably because it usually causes a horizontal scrollbar to be toggled as well, causing another viewport height change). This is actually the more common and annoying scenario for me: I frequently have Okular and a Konsole window open side-by-side on a small screen because I'm reading a programming-related text and what to try out in Konsole what I read about. Because my laptop screen is rather small, I usually collapse the sidebar to maximize the available space for the document. But occassionally I have to uncollapse it to check something in the TOC or so - and then the scroll position jumps.

Interestingly step 2 seems to be a necessary condition to reproduce the problem, although there might be other ways to set this up (it happens frequently enough for me that there are propably more triggers, but this is the reliable one I know of). If the document was manually dragged to a position using the Browse tool the jump doesn't occur (the scroll position will still change, but only to account for the (dis)appearing menu bar or horizontal scroll bar).