Not super sure who should do this, might be KIO.

When one browses through directories there's a good chance one may go up and down and up and down. The trouble is every time the dir changes we PreviewJob the new entries even when we know that they haven't changed. At the best of times that will simply result in the job poking the thumbnailer and the thumbnailer reading the cache file from disk, over and over again. At the worst of times the previewable content has caching disabled entirely (Text thumbs, directory thumbs) and we spend energy on needlessly updating stuff all the time. The current way of doing things also means there can be a noticeable amount of time between going up to the previous directory and the (unchanged) previews appearing again.

To improve things I'd propose having a global static FIFO in-memory cache of previews with some N entries that attaches a QFileSystemWatcher or something to kick the cache entry out in case the file changes.
When loading previews for $URI first try to pop them out of the in-memory cache and only if that misses start a PreviewJob.

For example:
I'm in dolphin's src dir and descend into the views dir only to find that the file I was looking for is in fact not here so I go back up and into the kitemviews dir instead.