Bug 151122

Summary: Opening Albums Has Become Very Slow
Product: [Applications] digikam Reporter: jgoerzen
Component: Albums-MainViewAssignee: Digikam Developers <digikam-bugs-null>
Status: RESOLVED FIXED    
Severity: normal CC: marcel.wiesweg
Priority: NOR    
Version: 0.9.2   
Target Milestone: ---   
Platform: Debian testing   
OS: Linux   
Latest Commit: Version Fixed In: 0.9.4
Sentry Crash Report:
Attachments: part of the strace

Description jgoerzen 2007-10-21 04:37:26 UTC
Version:           0.9.2 (using KDE KDE 3.5.7)
Installed from:    Debian testing/unstable Packages
OS:                Linux

This problem did not exist in 0.9.1.  It began with 0.9.2 and is still present in 0.9.3 beta.

Sometime after upgrading to 0.9.2, digikam got REALLY slow on big albums
-- or tags that selected lots of photos.  top showed kio_digikamalbum
using resources, and stracing it revealed that it was reading every byte
of every file in the album.  It did that before displaying anything
about the album's contents.  That is, it wasn't just rebuilding
thumbnails.  It also is not faster on subsequent access attempts.

My photos are on a local XFS filesystem.  They used to be on a local ext3 filesystem, which exhibited the same problem.
Comment 1 Arnd Baecker 2007-10-21 08:56:05 UTC
Created attachment 21883 [details]
part of the strace


Hi,

I don't experience the problem, so what do you mean by
"big albums", i.e. how many entries are there?

Also, I had a look at the output of strace, i.e. using
  strace -o/tmp/strace.txt digikam
which is pretty overwhelming
(Even if nothing is done inside of digikam one gets a loop of:
gettimeofday({1192949034, 900696}, NULL) = 0
select(21, [3 5 7 9 12 13 14 15 16 17 18 19 20], [], [], {0, 95}) = 0
(Timeout)
gettimeofday({1192949034, 904522}, NULL) = 0
stat64("/home/abaecker/Pictures", {st_mode=S_IFDIR|0755, st_size=4096,
...}) = 0
gettimeofday({1192949034, 904786}, NULL) = 0
gettimeofday({1192949034, 904832}, NULL) = 0
gettimeofday({1192949034, 904856}, NULL) = 0
gettimeofday({1192949034, 904914}, NULL) = 0
)
Anyway, concerning reading every file: I have an album with
174 images, of which the first 4 are visible in the
album view, and only for these (apart from the initial scan during start-up)
I find an access to the file (see the attachment).
So this suggests to me (but I might be completely wrong)
that only the first few are accessed
(whether they are completely read or not is
beyond my capabilities to interpret the output of strace ...)

Arnd
Comment 2 jgoerzen 2007-10-21 15:27:36 UTC
"big" albums mean generally 100 or more photos.  I have a number of albums with around 150 photos in them that cause this problem.  Though, for some reason, there are a few that size that do not.

The process that I straced was not digikam, but rather kio_digikamalbums.  This was the process (not digikam) that showed up using all the CPU in top, and this is the process that read through every file (not just the visible ones) according to strace.

Also, terminating digikam does not necessarily make this process exit immediately or stop doing its scan immediately.
Comment 3 jgoerzen 2007-10-21 15:34:28 UTC
One other thing I should point out: this is not limited to kio_digikamalbums.  If I use the tags view instead of the albums view, and select a tag that has many matching photos, the same phenomenon occurs, but with kio_digikamtags chewing up CPU.
Comment 4 Arnd Baecker 2007-10-21 16:01:40 UTC
Sorry, I did not read your original message carefully enough ...
No idea about what is going on here ... 
Comment 5 Arnd Baecker 2007-11-07 21:15:44 UTC
Just to archive a bit of a discussion on the mailing list:

Marcel wrote:
The user reports that kio_digikamalbums is taking all the CPU, or digikamtags
etc. If this was a problem with regenerating thumbnails, it would be the
digikamthumbnail ioslave. If it was scanning, it would not be in digikamtags
(only albums is doing the scanning, in 0.9). It must be listing. But I have
no idea what could be reading full images there.

To which I wrote:
Marcel, is there any way to debug this (e.g. is strace the way to go?)
I only fear that on our machines the problem might not
be reproducable (at least I don't see any symptoms ...)

Comment 6 Marcel Wiesweg 2007-11-07 21:41:38 UTC
There is a way to get a gdb backtrace:
Find out the process id of the process eating CPU, and attach gdb with "gdb attach <PID>". GDB will load the symbols and suspend the process. You can get a backtrace typing "bt". If there is an endless loop, this might show us where (but it might just as well show nothing)
Comment 7 jgoerzen 2007-11-15 00:30:43 UTC
I will do this today or tomorrow.
Comment 8 David Fraser 2008-01-01 18:04:59 UTC
0.9.3 seems to fix this problem for me, it's nice and fast now on my thousands of images. Perhaps this should be marked fixed?
Comment 9 Arnd Baecker 2008-01-01 18:51:19 UTC
Hi David, well, I am not sure whether this bug is really fixed.
For example, I never had any problems.
Therefore I would suggest to wait for feedback from the orginal reporter.

jgoerzen, do you have any update on this issue with the released 0.9.3?

Thanks a lot in advance, Arnd
Comment 10 bmarsh 2008-01-01 19:19:44 UTC
I just ran a test on 0.9.2.    From clicking on the digikam icon to main window up too 20 secs.  Not too bad because DK was doing some scans during that time.

But once the main window was up, it took  1:20 minutes for the thumbnails to show.  This is on a 5,500 picture album.  

And you say "that's a lot of pictures, what do you expect?"  and I could agree with you except that it went much more quickly with previous versions of DK.  Something did change quite drastically.

Just my $.02.   I haven't yet determined what X11 libs I need to compile 0.9.3 but I'm looking into it.  
Comment 11 Marcel Wiesweg 2008-01-02 17:53:02 UTC
Have you enabled "Show image dimensions" in the settings? This greatly slows down listing in 0.9.x.
Comment 12 bmarsh 2008-01-02 18:13:21 UTC
No....   "show image dimensions" is not selected.
Comment 13 Arnd Baecker 2008-02-16 09:00:28 UTC
jgoerzen, do you have any update on this issue?

Best, Arnd
Comment 14 jgoerzen 2008-02-16 15:10:27 UTC
I have tried this on 0.9.3 and indeed it appears to be resolved for me in the released 0.9.3 version.

I cannot comment on the experiences others have been reporting.
Comment 15 Arnd Baecker 2008-02-16 16:47:34 UTC
Many thanks for the feedback!

bmarsh: did you test this issue with 0.9.3?
If everything works fine now, we could close this bug ...

Many thanks in advance, Arnd 
Comment 16 bmarsh 2008-02-16 18:54:35 UTC
I'm running  kubuntu 7.10  and don't have 0.9.3  yet.  Been trying to compile it but haven't yet found all the pieces.   Is there a deb file anywhere avail?

Thanks
Comment 17 Arnd Baecker 2008-03-14 12:49:34 UTC
bmarsh, now there are kubuntu gutsy packages, see 
http://www.mpe.mpg.de/~ach/kubuntu/gutsy/Pkgs.php

Could you try them and see if the problem persists?

Many thanks in advance, Arnd 
Comment 18 bmarsh 2008-03-14 14:11:54 UTC
Yes, the problem is fixed in 0.9.3.  Thank you much.
Comment 19 caulier.gilles 2008-03-14 14:20:14 UTC
Thanks Bmarsh. I close this file now.

Gilles Caulier
Comment 20 caulier.gilles 2008-03-14 14:20:29 UTC
Closed