Bug 156146

Richard, 

This entry still valid using last stable 0.9.3 release ?

Gilles Caulier

Richard,

Without a fresh report from you, nothing can be done and this file will be closed...

Gilles Caulier

i have the same probleme on a mandriva 2008 with the RPM 9.2 & 9.3

 

Sylvain,

Sound like a package problem... I use/develop digiKam under Mandriva 2008.0 without any problem of course...

My root album path is under a dedicated SATA disk (250Mb)

Gilles Caulier

use it under mandriva 2008.0 without problem but i buy à news hard drive and when i add my old photo digikam doesnt start et bug on "lecture de la base de donnée".

the problems start when i have over 30 000 photos.

when i move photo to have less than 30 000 photos Digikam start normally.

i have found a solution :
start digikam when it freeze i start a second digikam when he freeze to i kill the first digikam and then the second digikam start normally.

how can i give you more informations for troubleshoot this bug.

Sylvain,

Report here all messages from the console when digiKam start.

Marcel, any tips about database hack here ?

Gilles

when i start digikam from konsole i have only  :
croucrou@localhost:~$ digikam
Found dcraw version: 8.60

and then digikam freeze on the message "Lecture de la base de donnée"

is there a solution to have debug information when i start digikam
 

Hi,

I have 27500 pictures in my folder, the bd file is about 2.7 MO.
I've no error message when i start digikam with konsole.

About the tip's of Sykvain (comment #5) : "start digikam when it freeze i start la second digikam when he freeze to i kill the first digikam and then the second digikam start normally". It run good only one time :-(

Richard

To enable full debug info, you need to recompile digiKam using "./configure --enable-debug=full"

All debug messages will appears on the console

Also, in HACKING file, there is a method given to profiling program. Look "Profiling with cachegrind" section for details.
Gilles Caulier

I have always the same problem on the mandriva 2008.1 and digikam 9.3

i have compile digikam 9.4 beta 3 on a mandriva 2008.1

and i have always the same problem. freeze on "reading database"

messages write in console after start digikam :

Found dcraw version: 8.83
digikam: ScanLib: Recherche d'albums non existants: 409 ms
digikam: ScanLib: Finding items not in database: 3262 ms
digikam: ScanLib: Mise à jour des éléments sans date: 41 ms
digikam: KDirWatch method = FAM


 

Just to be sure: the images are not on an NFS (or samba etc.) mounted
directory?

Gilles,Marcel: could the number of directories put into KDirWatch 
cause such a problem? Or could it be some image, with which exiv2
has problems? How could one debug this problem further?

images and database are on a Fat32 partion

i need FAT 32 or NTFS to développe RAW under Nikon Capture NX.

i notice that the problème arrive when i have more than 39XXX photos.

when i have less than 39000 photos digikam start normaly.

I also had that problem with digikam 0.9.2 under Mandriva 2008.0. Not every time but often. I just tried again to get the digikam version and it only got stuck on the third consecutive start.

With the compiled svn version, I don't have the problem but it may only be the way it is compiled : if I remember well, it was also the case (working) with the compiled svn version at the time 0.9.2 got out.

i tried once to do some debugging. Again if I remember well, the program was stuck in a libfam call inside KDirWatch. I didn't get further.

Cheers,
  Loïc

i try with the SVN version 0.9.4 beta 4 and the problème is alwas the same.

Look sqlite version. perhaps it's relevant of this entry :

http://bugs.kde.org/show_bug.cgi?id=160966

Gilles Caulier

my sqlite version is 3.5.6

Hi,

I have the same problem under Mandriva 2008.1 64-bit Power Pack, the suggested solution by CROUZILLAT Sylvain (Comment #5) worked. Earlier, it used to not start the first time, but the second or latest third time it started up. I don't know how many fotos I have in there.
On the console, it stops doing anything when the followong message appears:
Found dcraw version: 8.83

One more comment: I just found a way to count the files, there are some 22700 images.

hi,

where is your photos ?
on a ext3, NTSF, Fat32, ... partition ?

I download & install "inotify-tools" and "dnotify"
I download the last svn vertion.
I comment the tree line in albummanger.cpp containing "d->dirWatch->addDir"
I compile digikam and now digikam start as speed as road Runner (bip-bip).
every thing work.

The Marcel Wiesweg solution explaine for Thomas Eschenbacher
http://bugs.kde.org/show_bug.cgi?id=162535

solve my two bug i have with digikam
CPU usage over 40% : http://bugs.kde.org/show_bug.cgi?id=161093
and digikam don't start : https://bugs.kde.org/show_bug.cgi?id=156146

Marcel,

Sound like KDirWatch internal selection method is a big problem for digiKam. We have no fine control on KDirWatch to solve it.

We must fork this report to KDirWatch maintener from KDELibs core team ?

Gilles Caulier

Yes, we have absolutely no control on that. I dont know if the maintainers are aware of possible deficiencies and shortcomings with the one or other method, it cant hurt to fork this report.

Marcel, 

Look here : http://bugs.kde.org/show_bug.cgi?id=85989

Gilles

David, 

I have assigned this file to you, accordingly with all files relevant of KDirWatch already assigned to you...

Best

Gilles Caulier

Hmm, Dirk and Flavio are better KDirWatch experts than me... I'm just assigned all kio bugs, but that doesn't mean I can fix them all ;)
This bug report is also very unprecise about what the problem is; someone with the problem should start by running the kdirwatch tests in kdelibs.

Hi,

I haven't got the problem with Digikam 0.10.0 betas & RC on mandriva 2009

Maybe these two things help a little:

1: around 21000 files, 250 dirs seems to be a boundary after which it gets more unlikely to get digikam started.

2: I've run 0.9.[45] with with 44584 files, 545 dirs on two debian/lenny installations:
  o Works on my old 32bit installation
  o Fails on my fresh 64 bis installation

I'm also getting this bug reliably in my debian install of digikam 0.9.5.

If I attach gdb to the frozen digikam process, the output is:

   **** Backtrace of frozen digikam 0.9.5 ****
(gdb) bt
#0  0x00007fe3cd927f5b in write () from /lib/libc.so.6
#1  0x00007fe3c654313b in Client::writeToServer () from /usr/lib/libfam.so.0
#2  0x00007fe3c6545f7f in ?? () from /usr/lib/libfam.so.0
#3  0x00007fe3cb7a0aa9 in KDirWatchPrivate::useFAM (this=0x26e6460, 
    e=0x29a6ca0)
    at /build/buildd/kdelibs-3.5.10.dfsg.1/./kio/kio/kdirwatch.cpp:592
#4  0x00007fe3cb7a12d4 in KDirWatchPrivate::addEntry (this=0x26e6460, 
    instance=0x244f830, _path=<value optimized out>, sub_entry=0x0, isDir=true)
    at /build/buildd/kdelibs-3.5.10.dfsg.1/./kio/kio/kdirwatch.cpp:885
#5  0x00007fe3cfad2d10 in Digikam::AlbumManager::scanPAlbums (this=0x234ced0)
    at /build/buildd/digikam-0.9.5~beta3/./digikam/digikam/albummanager.cpp:515

I've found that I can reliably unfreeze digikam by killing famd. The
debugging output of famd just before the freeze is:

  famd[30114]: New FileWatch for fe05 3213e

  famd[30114]: told dnotify to monitor "aa-0987.jpg" = dev 254/5, ino 205118
  famd[30114]: sent event to client 69: request 7 "aa-0987.jpg" Exists
  famd[30114]: express() name: aa-0803.jpg
  
  famd[30114]: New FileWatch for fe05 32114
  
  famd[30114]: told dnotify to monitor "aa-0803.jpg" = dev 254/5, ino 205076
  famd[30114]: sent event to client 69: request 7 "aa-0803.jpg" Exists
  famd[30114]: -chdir to "/"

The specific versions I am running are debian packages:
  fam: 2.7.0-13.3
  digikam: 2:0.9.5~beta3-1

I'm unable to strace digikam from the beginning because strace
crashes.  However, if I strace digikam after the freeze, it appears to
be blocked on:

  write(17, "\0\0\0R"..., 4

fd17 is:
   0 lrwx------ 1 xxx xxx 64 2009-03-17 01:50 /proc/30272/fd/17 -> socket:[671664]

If I then kill famd, the strace continues with:

write(17, "\0\0\0R"..., 4)              = -1 ECONNRESET (Connection reset by peer)
select(18, [17], NULL, NULL, {0, 0})    = 1 (in [17], left {0, 0})
read(17, "\0\0\0\20e6 aa-0467.jpg\n\0\0\0\0\20e6 aa-076"..., 3000) = 3000
select(18, [17], NULL, NULL, {0, 0})    = 1 (in [17], left {0, 0})
read(17, ".jpg\n\0\0\0\0\20e7 aa-0362.jpg\n\0\0\0\0\20e7 "..., 2986) = 2986
select(18, [17], NULL, NULL, {0, 0})    = 1 (in [17], left {0, 0})
read(17, "\0\0\0\20e7 aa-0283.jpg\n\0\0\0\0\20e7 aa-074"..., 3000) = 2660
select(18, [17], NULL, NULL, {0, 0})    = 1 (in [17], left {0, 0})
read(17, ""..., 3000)                   = 0
read(17, ""..., 3000)                   = 0
stat("/home/user/.kde/share/config/", {st_mode=S_IFDIR|0700, st_size=4096, ...}) = 0
access("/home/user/.kde/share/config/kdebugrc", W_OK) = -1 ENOENT (No such file or directory)
access("/home/user/.kde/share/config/kdebugrc", F_OK) = -1 ENOENT (No such file or directory)
access("/home/user/.kde/share/config", W_OK) = 0

Looking at the strace, it appears the problem may be some sort of
lack-of-buffering deadlock; the directory that contains aa-0467.jpg
and aa-0283.jpg has 170-350 files and 25k of text in the
filenames. Curiously, aa-0362.jpg is the 16396'th (2**14+12) file in a
'find .' in the album directory tree. I have total of 21k photos, 466
dirs and just over 1mb of text composing the path names. The filesystem 
is ext3.

For completeness famd is blocked on:
  select(70, [3 5 10], [69], NULL, NULL

0 lrwx------ 1 root root 64 2009-03-17 01:49 /proc/30200/fd/10 -> socket:[671592]
0 lrwx------ 1 root root 64 2009-03-17 01:49 /proc/30200/fd/3 -> socket:[671215]
0 lr-x------ 1 root root 64 2009-03-17 01:49 /proc/30200/fd/5 -> pipe:[671325]

If I run digikam under ltrace the last function calls are:


SYS_write(17, "M331 1000 4 /tmp/Pictures/2007/2007-02/2007-02-YYYYYYYYY\n", 87)                  = 87
SYS_select(18, 0x7fff09851c80, 0, 0, 0x7fff09851d00)                                             = 0
QPtrCollection::newItem(void*)(0x1504690, 0x1cc68b0, 0x1cc68b0, 0x1cc6960, 0x1cc68a0)            = 0x1cc68b0
QPtrCollection::newItem(void*)(0x1504690, 0x1cc6900, 0x1cc6900, 0x1cc6960, 0x1504658)            = 0x1cc6900
QPtrCollection::newItem(void*)(0x161b6c0, 0x1cc68b0, 0x1cc61c0, 0, 0x1cc6160)                    = 0x1cc68b0
QPtrCollection::newItem(void*)(0x1cc6160, 0x1cc6870, 0x1cc6870, 150, 24)                         = 0x1cc6870
QPtrCollection::newItem(void*)(0x1cc6870, 0x1cc6970, 1878, 0x7f10fcee3c42, 0x7f10fcee3c43)       = 0x1cc6970
QPtrCollection::newItem(void*)(0x1a051c0, 0x1cc6900, 408, 0x1cc69b0, 0x1cc6990)                  = 0x1cc6900
QPtrCollection::newItem(void*)(0x15dd500, 0x1cc69e0, 0x15dd8d0, 0, 666)                          = 0x1cc69e0
QPtrCollection::newItem(void*)(0x161a690, 0x1cc6250, 6, 0xd9e0852, 31)                           = 0x1cc6250
QPtrCollection::newItem(void*)(0x161a6c0, 0x1cc6250, 11, 1, 31)                                  = 0x1cc6250
SYS_stat("/tmp/Pictures/2007/2007-02/2007-02-XXXXXXXXXXXXXXXXXXXXXX", 0x7fff09852010)            = 0
QPtrCollection::newItem(void*)(0x1cc7188, 0x1cc6aa0, 32, 0, 0x1cc70c0)                           = 0x1cc6aa0
SYS_select(18, 0x7fff09851c80, 0, 0, 0x7fff09851d00)                                             = 1
SYS_read(17, "", 3000)                                                                           = 430
SYS_select(18, 0x7fff09851c80, 0, 0, 0x7fff09851d00)                                             = 0
QPtrCollection::newItem(void*)(0x1504690, 0x1cc7230, 0x1cc7230, 0x1cc72e0, 0x1cc7220)            = 0x1cc7230
QPtrCollection::newItem(void*)(0x1504690, 0x1cc7280, 0x1cc7280, 0x1cc72e0, 0x1504658)            = 0x1cc7280
QPtrCollection::newItem(void*)(0x161b6c0, 0x1cc7230, 0x1cc70a0, 0, 0x1cc7080)                    = 0x1cc7230
QPtrCollection::newItem(void*)(0x1cc6f90, 0x1cc7350, 0x1cc7350, 0x1cc7380, 0x1cc7340)            = 0x1cc7350
QPtrCollection::newItem(void*)(0x1cc7350, 0x1cc6da0, 1878, 0x7f10fcee3c42, 0x7f10fcee3c43)       = 0x1cc6da0
QPtrCollection::newItem(void*)(0x1a051c0, 0x1cc7280, 408, 0x1cc73a0, 0x1cc7380)                  = 0x1cc7280
QPtrCollection::newItem(void*)(0x15dd500, 0x1cc73d0, 0x15dd8d0, 0, 667)                          = 0x1cc73d0
SYS_open("/proc/sys/kernel/ngroups_max", 0, 031)                                                 = 18
SYS_read(18, "65536\n", 31)                                                                      = 6
SYS_close(18)                                                                                    = 0
SYS_getgroups(65536, 0x1cc7460, 0, 0x7f10ff1e5a70, 0x1cc7450)                                    = 9
SYS_geteuid()                                                                                    = 1000
SYS_write(17, "", 4)                                                                             = -512
--- SIGTERM (Terminated) ---
+++ killed by SIGTERM +++

I can do some additional tests if you can propose any.

As a temporary workaround for other people experiencing this bug and
see this report in bugzilla, the freeze can be avoided by killing or not
starting famd.

With 0.10 we add one dirwatch only per collection with the WatchSubDirs flag set. This might cure these problems (unless it works around FAM limitations internally by resorting to per-subdir watches).
This API flag is not available for KDE3 afaik.

IMHO, if the situation that could lead to a crash is now fixed for the KDE4 version and there is not going to be any KDE3 new release, the report could be marked as fixed. (But we need to ensure that the crash is properly fixed;) ) Thanks.

closing, please reopen if still valid

Not reproducible since digiKam use a multithreaded interface instead KIO to handle database connections.