Bug 162038 - digikam hangs and eats CPU when changing files
Summary: digikam hangs and eats CPU when changing files
Status: RESOLVED FIXED
Alias: None
Product: digikam
Classification: Applications
Component: Database-Sqlite (show other bugs)
Version: 0.9.4
Platform: unspecified Linux
: NOR crash
Target Milestone: ---
Assignee: Digikam Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2008-05-13 14:14 UTC by Marcus Hardt
Modified: 2020-08-26 03:57 UTC (History)
4 users (show)

See Also:
Latest Commit:
Version Fixed In: 5.0.0


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Marcus Hardt 2008-05-13 14:14:25 UTC
Version:           0.9.4-beta5 (using 3.5.9, Debian Package 4:3.5.9.dfsg.1-2+b1 (lenny/sid))
Compiler:          Target: i486-linux-gnu
OS:                Linux (i686) release 2.6.23.14-1

When changing a file (rating, moving it to another subdirectory) or creating a new album, the process "digikam" utilises 100% CPU and the GUI hangs.

After about 60s normal operation resumes.

Affected Versions:
x 0.9.3 that ships with debian/unstable
x 0.9.4 SVN as of beginning of May

Things trid so far:
o remove digikam3.db (all metadata stored in my images)
o remove $HOME/.kde/share/config/digikamrc
o use a completely different user

=> No effect shown


Removing most of the pictures greatly increased the speed. 

I will investigate further. Any hint?
Comment 1 caulier.gilles 2008-05-13 14:26:48 UTC
Hi Marcus,

Yes, please check libsqlite version. Somebody has reported in this room than last version break digiKam everywhere. We have not find yet the right way to fix it.

Gilles Caulier
Comment 2 Marcus Hardt 2008-05-13 16:03:03 UTC
Quoting Gilles Caulier
[bugs.kde.org quoted mail]

Ok; found these packages matching sqlite on my system:

libmono-sqlite1.0-cil:	1.9.1+dfsg-1 
libsqlite0:		2.8.17-4 
libsqlite0-dev:		2.8.17-4 
libsqlite3-0:		3.5.8-1 
libsqlite3-dev:		3.5.8-1 

I can upgrade the lisqlite3-* packages to 3.5.8-4, if you think it helps.
Comment 3 caulier.gilles 2008-05-13 16:08:13 UTC
Marcus,

It will be better to test with libsqlite 3.5.6...


Gilles Caulier
Comment 4 Marcus Hardt 2008-05-13 18:07:39 UTC
I downloaded (from CVS) libsqlite as of 2008/02/06 (as this was the date when 
the file VERSION, containing 3.5.5 was comitted). 

I can confirm that the described behaviour does not occur in this version. I 
removed digikam3.db and re-initialised the db  (using 100GB of pictures).

Will trace upwards the version tree of libsqlite.

One problem though is that in my svn version of 
digikam, /usr/lib/libsqlite3.so.0 seems to be hardlinked, because setting 
LD_LIBRARY_PATH to my own sqlite installation didn't help. I had to redirect 
the link, which is not so nice...
Comment 5 caulier.gilles 2008-05-13 18:45:31 UTC
Marcus,

thanks for the feedback.

Arnd,

there is a duplicate entry in B.K.O ?

Gilles
Comment 6 Marcus Hardt 2008-05-13 18:49:02 UTC
The problem occurs between these two CVS versions (obtained via cvs log 
VERSION in the sqlite cvs tree)

revision 1.170
date: 2008/04/15 14:37:51;  author: drh;  state: Exp;  lines: +1 -1
Increment the version number.
----------------------------
revision 1.169
date: 2008/03/11 15:41:14;  author: drh;  state: Exp;  lines: +1 -1
Increase the version number to 3.5.7 in anticipation of the next release.



The strange part is that the fiele VERSION of 2008/03/11 15:41:14;  contains 
3.5.6 (and works) while the one of 2008/04/15 14:37:51;  contains 3.5.7 and 
does not work.

According to the cvs comments either the versioning is wrong or I 
misunderstand things; Just writing this to make that clear.

Hope it helps.

If you want me to try a specific different point in time: Here I am.
Comment 7 Arnd Baecker 2008-05-13 19:56:15 UTC
Gilles, this sounds like 
https://bugs.kde.org/show_bug.cgi?id=160966
for which I filed a bug for sqlite, see the discussion in 
http://www.sqlite.org/cvstrac/tktview?tn=3087

Marcus, great detective work! In the above entry they claim
that this has been fixed with 
http://www.sqlite.org/cvstrac/chngview?cn=5026
Can you maybe confirm this?

Thanks a lot, Arnd
Comment 8 Marcus Hardt 2008-05-14 14:04:29 UTC
Hi Arnd,

Will check the fix you suggest. I've ran a couple of compilations and found 
that the CVS version as of
	2008-05-26-15:00:00 
works, while
	2008-05-26-16:00:00 
does not.

I have no deep understanding of sqlite, so I'd rather post the excerpt of the 
diff that IMO causes the trouble observed.

The Diff was created having the 15:00 version, diffing against 16:00. It can 
be reproduced using;

cvs up -D 2008/03/26\ 15:00:00
cvs diff -D 2008/03/26\ 16:00:00 -u

Index: src/where.c
===================================================================
RCS file: /sqlite/sqlite/src/where.c,v
retrieving revision 1.292
retrieving revision 1.291
diff -u -r1.292 -r1.291
--- src/where.c 26 Mar 2008 14:56:35 -0000      1.292
+++ src/where.c 25 Mar 2008 09:47:35 -0000      1.291
@@ -16,7 +16,7 @@
 ** so is applicable.  Because this module is responsible for selecting
 ** indices, you might also think of this module as the "query optimizer".
 **
-** $Id: where.c,v 1.292 2008/03/26 14:56:35 drh Exp $
+** $Id: where.c,v 1.291 2008/03/25 09:47:35 danielk1977 Exp $
 */
 #include "sqliteInt.h"

@@ -743,10 +743,7 @@
   }
   prereqAll = exprTableUsage(pMaskSet, pExpr);
   if( ExprHasProperty(pExpr, EP_FromJoin) ){
-    Bitmask x = getMask(pMaskSet, pExpr->iRightJoinTable);
-    prereqAll |= x;
-    pTerm->prereqRight |= x-1; /* ON clause terms may not be used with an 
index
-                               ** on left table of a LEFT JOIN.  Ticket #3015 
*/
+    prereqAll |= getMask(pMaskSet, pExpr->iRightJoinTable);
   }
   pTerm->prereqAll = prereqAll;
   pTerm->leftCursor = -1;
@@ -2046,36 +2043,14 @@
     pWhere = 0;
   }

-  /* Assign a bit from the bitmask to every term in the FROM clause.
-  **
-  ** When assigning bitmask values to FROM clause cursors, it must be
-  ** the case that if X is the bitmask for the N-th FROM clause term then
-  ** the bitmask for all FROM clause terms to the left of the N-th term
-  ** is (X-1).   An expression from the ON clause of a LEFT JOIN can use
-  ** its Expr.iRightJoinTable value to find the bitmask of the right table
-  ** of the join.  Subtracting one from the right table bitmask gives a
-  ** bitmask for all tables to the left of the join.  Knowing the bitmask
-  ** for all tables to the left of a left join is important.  Ticket #3015.
-  */
-  for(i=0; i<pTabList->nSrc; i++){
-    createMask(&maskSet, pTabList->a[i].iCursor);
-  }
-#ifndef NDEBUG
-  {
-    Bitmask toTheLeft = 0;
-    for(i=0; i<pTabList->nSrc; i++){
-      Bitmask m = getMask(&maskSet, pTabList->a[i].iCursor);
-      assert( (m-1)==toTheLeft );
-      toTheLeft |= m;
-    }
-  }
-#endif
-
   /* Analyze all of the subexpressions.  Note that exprAnalyze() might
   ** add new virtual terms onto the end of the WHERE clause.  We do not
   ** want to analyze these virtual terms, so start analyzing at the end
   ** and work forward so that the added virtual terms are never processed.
   */
+  for(i=0; i<pTabList->nSrc; i++){
+    createMask(&maskSet, pTabList->a[i].iCursor);
+  }
   exprAnalyzeAll(pTabList, &wc);
   if( db->mallocFailed ){
     goto whereBeginNoMem;





	

[bugs.kde.org quoted mail]
Comment 9 Marcus Hardt 2008-05-14 14:14:24 UTC
Quoting Arnd Baecker
[bugs.kde.org quoted mail]
Arnd told, that sqlite claims, the bug was fixed with:
http://www.sqlite.org/cvstrac/chngview?cn=5026

I can confirm that this does _NOT_ fix the bug.

But the CVS version as of today (May 24th) does not contain the bug anymore.

> Can you maybe confirm this?
>
> Thanks a lot, Arnd

Comment 10 Marcus Hardt 2008-05-20 18:59:52 UTC
[bugs.kde.org quoted mail]

In Debian/unstable this problem is fixed by 
libsqlite3-0 Version 3.5.9-1
Comment 11 caulier.gilles 2008-05-20 19:13:38 UTC
Thanks Marcus,

Question what is the sqlite version where the problem is appears exactly ?

Arnd, 

This is want mean than we need to :

1/ Patch README file dependencies descriptions.
2/ Patch configure.in.in to check the right sqlite version.
3/ To post a message in mailing lists about this problem.
4/ To post a new blog entry to www.digikam.org

Gilles
Comment 12 Marcus Hardt 2008-05-26 11:20:45 UTC
The bug appeared somewhere in versions 3.5.7 and 3.5.8.
It is is not present in 3.5.9 

I confirmed this for the CVS versions as well as the corresponding debian 
packages of 3.5.8(currently in testing) and 3.5.9.(currently in unstable)




Note: The dates of the CVS I've specified earlier in this mailing were 
contradicting: All dates I wanted to mention were in March, not in May.


M.



[bugs.kde.org quoted mail]
Comment 13 caulier.gilles 2008-05-26 12:10:56 UTC
Achim,

How we can use KDE_PCK_CHECK_MODULES in .configure.in.in to check libsqlite version istalled using these conditions:

sqlite3 >= 3.5.9
OR
(sqlite3 >= 3.0 && sqlite3 <= 3.5.6)

Gilles Caulier
Comment 14 David Fraser 2008-08-22 08:51:48 UTC
I'm apparently getting the same bug fairly frequently when changing ratings on files. Actually I have libsqlite3-0 3.4.2-2 on Ubuntu 8.04, but the same symptoms. I mention it because of the workaround below that may be useful to others:

Actually digikam uses lots of CPU but so does the kio_digikamalbums kdeinit process. I found that running `pkill -f kio_digikamalbums` resolves the problem within a few seconds (digikam launches a new slave process).
Comment 15 Andi Clemens 2010-09-25 17:43:13 UTC
Closing some old bugreports that are related to old digiKam version, and that have not received answers for two years now.

If you think the reports are still valid, feel free to re-open them, but please provide updates and do not just open them without giving feedback.
Comment 16 caulier.gilles 2015-07-04 06:04:44 UTC
New digiKam 4.11.0 is available.

https://www.digikam.org/node/740

Can you reproduce the problem with this release ?
Comment 17 swatilodha27 2016-07-02 16:28:10 UTC
GUI doesn't hang on performing any of the above action in digiKam. Try 5.0.0 version.

If you think the report is still valid, feel free to re-open.