Bug 216145 - Infinite Loop when scanning large folders
Summary: Infinite Loop when scanning large folders
Status: RESOLVED FIXED
Alias: None
Product: digikam
Classification: Applications
Component: Database-Scan (show other bugs)
Version: 0.10.0
Platform: Gentoo Packages Linux
: NOR crash
Target Milestone: ---
Assignee: Digikam Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2009-11-25 20:09 UTC by David H
Modified: 2018-01-07 16:16 UTC (History)
2 users (show)

See Also:
Latest Commit:
Version Fixed In: 5.8.0


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description David H 2009-11-25 20:09:28 UTC
Version:           digikam-0.10.0-r1 (using KDE 4.3.3)
OS:                Linux
Installed from:    Gentoo Packages

A user was having problems (unrelated to digikam) with her KDE4.3.3 plasma workspace.  We had recently upgraded the systems from kde3.5.  I moved her .kde4 directory to a backup location and logged her in again.  This fixed the desktop problems we were having but of course all of her previous [application] settings had to be redone.

We started digikam and pointed it at her "Pictures" directory.  
The users "Pictures" folder had a link to a network drive with 19G of images in it: eg

./Pictures (digikam images folder)
    digikamdb.4 (200M+)
    SomeLocalFolders
    LinkToNetworkFolder (19G)


The digikam dialog said it was scanning images in the folders.  This went on for hours.  I noticed the digikam4.db file was reaching several hundred megabytes in size.  Eventually the information at the bootom of this message started coming out slowly.  I let it run for a while until I noticed the message sets repeating (after about 257 messages) at which point I killed the process.

As mentioned earlier the users "Pictures" folder had a link to a network drive with 19G of images in it: eg

./Pictures (digikam images folder)
    digikamdb.4 (200M+)
    SomeLocalFolders
    LinkToNetworkFolder (19G)

When I removed the link, the program was able to start correctly, and the digikamdb.4 file returned to a sane size (279k).

There are other users on the network with access to the same network folder in a setup similar to the structure listed above (and same KDE and digikam versions).  Their .db files are on the order of 5M or so.

This particular users digikam had been working fine previously.  I tried replacing her ./share/apps/digikam and ./share/config/digikamrc with files from her original .kde4 directory to see if it helped, but the same errors occurred.

Below is the repeating section of output created by digikam while "Scanning the folders".  I have removed some [200] of the repeating lines and added some comments inside brackets < like this >:

Note
------------------ output ------------------------
$digikam

<pause of several minutes while dialog indicates folders are being scanned >

Warning: Directory OlympusCs, entry 0x0101: Strip 0 is outside of the data area; ignored. 
Warning: No image data to encode Exif.OlympusCs.PreviewImageStart.  
Warning: Directory OlympusCs, entry 0x0101: Strip 0 is outside of the data area; ignored. 
Warning: No image data to encode Exif.OlympusCs.PreviewImageStart. 

< there is no OlympusCs directory in any of the images folders >
< this message repeats some 200 times >

Warning: Directory OlympusCs, entry 0x0101: Strip 0 is outside of the data area; ignored. 
Warning: No image data to encode Exif.OlympusCs.PreviewImageStart.  
Warning: Directory OlympusCs, entry 0x0101: Strip 0 is outside of the data area; ignored. 
Warning: No image data to encode Exif.OlympusCs.PreviewImageStart.  
libpng warning: zTXt: CRC error 
libpng warning: zTXt: CRC error 
libpng warning: zTXt: CRC error 
libpng warning: zTXt: CRC error 
Warning: XMP toolkit support not compiled in. 
Warning: Failed to decode XMP metadata. 
Warning: XMP toolkit support not compiled in. 
Warning: Failed to decode XMP metadata. 
Warning: XMP toolkit support not compiled in. 
Warning: Failed to decode XMP metadata. 
Warning: XMP toolkit support not compiled in. 
Warning: Failed to decode XMP metadata. 
Warning: XMP toolkit support not compiled in. 
Warning: Failed to decode XMP metadata. 
Warning: Directory NikonPreview, entry 0x0201: Data area exceeds data buffer, ignoring it.
Error: Directory NikonPreview with 2020 entries considered invalid; not read. 
Error: Directory Sony with 1c00 entries considered invalid; not read. 
Error: Directory Sony with 1c00 entries considered invalid; not read. 
Error: Directory Sony with 1c00 entries considered invalid; not read. 
Error: Directory Sony with 1c00 entries considered invalid; not read. 
Error: Directory Sony with 1c00 entries considered invalid; not read. 
Error: Directory Sony with 1c00 entries considered invalid; not read. 
Error: Directory Sony with 1c00 entries considered invalid; not read. 
Error: Directory Sony with 1c00 entries considered invalid; not read. 
Error: Directory Sony with 1c00 entries considered invalid; not read. 
Error: Directory Sony with 1c00 entries considered invalid; not read. 
Error: Directory Sony with 1c00 entries considered invalid; not read. 
Error: Directory Sony with 1c00 entries considered invalid; not read. 
Error: Directory Sony with 1c00 entries considered invalid; not read. 
Error: Directory Sony with 1c00 entries considered invalid; not read. 
Error: Directory Sony with 1c00 entries considered invalid; not read. 
Error: Directory Sony with 1c00 entries considered invalid; not read. 
Error: Directory NikonPreview with 2020 entries considered invalid; not read. 
Warning: Directory NikonPreview, entry 0x0201: Data area exceeds data buffer, ignoring it.
Error: Directory NikonPreview with 2020 entries considered invalid; not read. 
Error: Directory NikonPreview with 2020 entries considered invalid; not read. 
Warning: Directory NikonPreview, entry 0x0201: Data area exceeds data buffer, ignoring it.
Warning: Directory NikonPreview, entry 0x0201: Data area exceeds data buffer, ignoring it.
Warning: XMP toolkit support not compiled in. 
Warning: Failed to decode XMP metadata. 
Warning: XMP toolkit support not compiled in. 
Warning: Failed to decode XMP metadata. 

< then the process starts over again >

Warning: Directory OlympusCs, entry 0x0101: Strip 0 is outside of the data area; ignored. 
Warning: No image data to encode Exif.OlympusCs.PreviewImageStart.  

< ... >


------------------end output ---------------------
Comment 1 caulier.gilles 2009-11-25 21:05:40 UTC
Sound like a problem with your Exiv2 which hasn't compiled with XMP support...

Gilles Caulier
Comment 2 Marcel Wiesweg 2009-11-25 22:42:12 UTC
The error messages do not explain any underlying problem leading to an endless loop. Nonetheless, I would recommend to update digikam to recent 1.0 beta and libexiv2 to version 0.18.2. Switch on debug message for digikam (kdebugdialog, code 50003) - usually you see on the console which files are currently scanned, and if there is a loop.
Comment 3 caulier.gilles 2009-12-25 20:13:54 UTC
digiKam 1.0.0 is out since few days...

http://www.digikam.org/drupal/node/491

Please try with this version coming with more than 400 bug-fixes.

Thanks in advance

Gilles Caulier
Comment 4 caulier.gilles 2010-01-25 21:51:06 UTC
digiKam 1.1. release will be done in few days. Please check if this entry still valid.

Thanks in advance

Gilles Caulier
Comment 5 caulier.gilles 2010-04-04 10:45:03 UTC
digiKam and Kipi-plugins 1.2.0 are out. Please check if crash is still valid there.

Thanks in advance

Gilles Caulier
Comment 6 caulier.gilles 2010-06-07 10:17:23 UTC
No report there from user since a long time. Please try again with digiKam and kipi-plugins 1.2.0, else i will close this file...

Gilles Caulier
Comment 7 caulier.gilles 2010-06-07 14:40:56 UTC
We need user feedback there to continue...

Gilles Caulier
Comment 8 caulier.gilles 2017-11-30 09:30:56 UTC
Please update this entry from bugzilla with current 5.8.0 pre-release bundle to see if problem remain.

https://files.kde.org/digikam/

Thanks in advance

Gilles Caulier
Comment 9 caulier.gilles 2018-01-07 16:16:49 UTC
No feedback since a very long time in this file with a lots of commits done in
source code. I close this file now. Reopen if
necessary.