Bug 321299 - Folder size and folder/file type not initally filled in
Summary: Folder size and folder/file type not initally filled in
Alias: None
Product: nepomuk
Classification: Miscellaneous
Component: general (show other bugs)
Version: 4.10.80
Platform: openSUSE Linux
: NOR normal (vote)
Target Milestone: ---
Assignee: Nepomuk Bugs Coordination
Depends on:
Reported: 2013-06-17 22:37 UTC by bill p. (aka google01103)
Modified: 2013-06-21 23:07 UTC (History)
4 users (show)

See Also:
Latest Commit:
Version Fixed In:

my view properties foilder that appears to have caused the problem (3.42 KB, application/x-rar-compressed)
2013-06-19 00:42 UTC, bill p. (aka google01103)
A Patch which fixes the issue on the Dolphin side (3.74 KB, patch)
2013-06-20 16:47 UTC, Vishesh Handa

Note You need to log in before you can comment on or make changes to this bug.
Description bill p. (aka google01103) 2013-06-17 22:37:04 UTC
In Dolphin when switching folders (or initial start) I see the  size column for folders being filled (file sizes already filled) and the type column also being filled in for both folders and files.

When paging down the cascading effect of those 2 columns being filled continues

ext3 local filesystem
size of folder doesn't seem to matter

Reproducible: Always

Steps to Reproduce:
1. open Dolphin
Actual Results:  
watch size and type folders being populated

Expected Results:  
bam all columns filled as in earlier versions
Comment 1 Frank Reininghaus 2013-06-18 09:31:09 UTC
Thanks for the report. Some questions:

(a) Do you use "Sort by Name/Size/Type", or something else?

(b) Do you really see a difference between KDE 4.10 and 4.11 beta 1 in small folders (500 items or less)?

There are some changes regarding the loading of data which are very expensive to retrieve (like icons, mime types, folder contents count, etc.): https://git.reviewboard.kde.org/r/110839/

However, even in KDE 4.10, the folder size and mime types were in general not loaded immediately. I just tried it again, and I can definitely see that the columns are filled asynchronously after the initial loading. How fast that happens depends on many different factors though. If it happens fast enough (like when everything is cached by the kernel already), it could appear that it happens instantaneously.

The main difference between the old and the new implementation is that we do not retrieve the data for all items any more if there are more than 500 items in the folder. The reason is that mime type determination and things like that can easily keep the CPU and the hard drive busy for several minutes (but without blocking the GUI) when entering a folder with many items. There were user complaints at the forum and other places about this, and that's why I changed it.

We now only load the data for the items in the visible area, those a few pages before and after, and those on the first and last page of the view, i.e., those which can be reached quickly.

But as I said, with less than 500 items, the behaviour should actually not have changed much (even though there are a few glitches in some corner cases, which will be adressed before beta 2).
Comment 2 bill p. (aka google01103) 2013-06-18 10:14:45 UTC
a) Detail columns sort by name
b) I don't recall this behaver in 4.10 and we are talking folders with 7 folders and 14 files and even smaller (only 5 files and I'm seeing type being filled)

The strange thing here is if I "kdesu -u" to open my test user's Dolphin this behavior is not being seen.  Renaming ~/.kde4/share/config/dolphinrc doesn't fix it but the behavior  is sort of erratic sometimes it does and then it doesn't

Toggling natural sorting doesn't fix it either
Comment 3 Frank Reininghaus 2013-06-18 10:54:38 UTC
Thanks for the quick reply. I'm afraid I can't say what the problem is then (or if there is a new problem compared to KDE 4.10 at all).

Instructions to reproduce the problem would be appreciated. As I said, there will be some more changes until beta 2, so it would be great if you could re-test it then.
Comment 4 bill p. (aka google01103) 2013-06-18 11:49:24 UTC
To reproduce - just open Dolphin.

Logging in as my test user and no problem with the same folders - there is something in my config causing it but no idea what it could be. And pretty darn sure I did not experience this in 4.10.x

Running  kbuildsycoca4 --noincremental  did not help, will later erase kdecache

Obviously will test when beta2 is available

Comment 5 Frank Reininghaus 2013-06-18 13:32:09 UTC
> To reproduce - just open Dolphin.

OK, your comment 2 sounded like the behaviour is erratic/non-reproducible. But anyway, I could not observe a difference to KDE 4.10 yet.
Comment 6 bill p. (aka google01103) 2013-06-18 17:11:30 UTC
my first screen capture and my first youtube upload

Comment 7 bill p. (aka google01103) 2013-06-19 00:41:25 UTC
played around with Dolphin config and apps file/folders and found that deleting "~/.kde/share/apps/Doplhin/view properties" seems to have solved the problem - does this make any sense?
Comment 8 bill p. (aka google01103) 2013-06-19 00:42:21 UTC
Created attachment 80622 [details]
my view properties foilder that appears to have caused the problem
Comment 9 Frank Reininghaus 2013-06-19 05:41:13 UTC
Thanks for the update - good to know that it works now!

(In reply to comment #7)
> played around with Dolphin config and apps file/folders and found that
> deleting "~/.kde/share/apps/Doplhin/view properties" seems to have solved
> the problem - does this make any sense?

That obviously depends on how you customized the view after you removed the view properties folder ;-)

I see that you had an "Image Size" column in your video. If removing that one fixes the problem, then it might have something to do with our interactions with Nepomuk. I don't think that anything important has changed on the Dolphin side, however.
Comment 10 bill p. (aka google01103) 2013-06-19 09:20:02 UTC
Nepomuk was not enabled, though at one time it was and when it was "image size" was checked

Enabling Nepomuk (and not file indexing) and having image size checked then disabling Nepomuk and the issue returns

I think there's an issue with the interaction with Nepomuk, especially strange how it was disabled at the time
Comment 11 Frank Reininghaus 2013-06-19 14:55:34 UTC
Thanks for the new information. I'm not really familiar with the Nepomuk internals, but if you enable a column that can only be filled with Nepomuk's help, and indexing is disabled (such that no fast retrieval of the information from the database is possible), then it's probably not surprising that there is a delay.
Comment 12 bill p. (aka google01103) 2013-06-19 15:05:17 UTC
I would then think that this would be a bug, the attempted retrieval of data no longer available because a process (Nepomuk) is no longer enabled. 

If Nepomuk is not available then those columns provided by it should not be attempted to be filled. Is this done by Dolphin or by a KDE lib?.

As this wasn't an issue with 4.10 then I would think it should be considered a regression.
Comment 13 Emmanuel Pescosta 2013-06-19 15:30:28 UTC
I had similar problems after upgrading to KDE 4.11 Beta 1.

I have a folder with 23 folder and 81 files + Show in Groups + Show Preview + Additional Infomation: Size, Data, Rating + Sort by Type

It took > 45 seconds to open this folder (UI was frozen) and a lot of nepomuk and soprano errors appeared. 

Solution for this problem:
I have completely removed the nepomuk database and re-indexed my home directory -> Everything works fine and really fast now.
Comment 14 Frank Reininghaus 2013-06-20 15:15:33 UTC
Thanks for the update. But then it sounds like it's due to changes in Nepomuk, and we can't do much about the issue in Dolphin. Even though quite a few things have been changed in KFileItemModelRolesUpdater, the way we talk to Nepomuk should be unaffected by these.

I'll reassign to Nepomuk. @Vishesh: Feel free to close this if re-indexing the home directory is the recommended way to make everything work smoothly in KDE 4.11 (but then this should maybe be announced somehow).
Comment 15 Vishesh Handa 2013-06-20 16:44:32 UTC
This doesn't seem like a Nepomuk problem. It seems more like an issue of Dolphin trying to use Nepmuk when it isn't running, and because of that there is a delay. The main reason this delay is more obvious is that with 4.11 connecting to Nepomuk is a lot more expensive. In 4.10 days we just connected to a local socket. With 4.11, a full connection is made to virtuoso.

Therefore, dolphin should not unnecessarily try to use any of the Nepomuk2::Resource classes when it knows Nepomuk is not running. This is because each use of them attempts a connection to Nepomuk.
Comment 16 Vishesh Handa 2013-06-20 16:47:08 UTC
Created attachment 80673 [details]
A Patch which fixes the issue on the Dolphin side

This almost fixes the issue, there is still a problem of the Nepomuk2::FileMetadataWidget attempting to connect to Nepomuk multiple times. I'll fix that in the nepomuk-widgets repository.

A slight problem with this patch is that the roles are not updated if Nepomuk is switched off, Dolphin is started and then Nepomuk is started. This problem was always there, but a refresh used to fix it. Now one needs to restart Dolphin.
Comment 17 Frank Reininghaus 2013-06-20 17:59:13 UTC
Thanks for looking into the problem so quickly, Vishesh!

(In reply to comment #16)
> Created attachment 80673 [details]
> A Patch which fixes the issue on the Dolphin side

This looks good from my point of view, feel free to commit that to master!
Comment 18 Vishesh Handa 2013-06-21 23:07:25 UTC
Git commit 44791e7f155024d0c8961f907544676eebd5ea17 by Vishesh Handa.
Committed on 20/06/2013 at 18:39.
Pushed by vhanda into branch 'master'.

Dolphin: Do not try to connect to Nepomuk if it is not running

Each time one uses any of the Nepomuk classes, an attempt is made to
connect to the database. This slows down the application since
connecting to Nepomuk is not so cheap.

M  +29   -22   dolphin/src/kitemviews/kfileitemmodelrolesupdater.cpp