Hi, Using digikam for a long time, I'm experiencing database corruption from time to time (this is still going on with 4.4.0, the last Debian packaged version). I initially reported the problem and my analysis to the Debian Bug Tracker ( https://bugs.debian.org/756512 ) and maintainers just asked me to report it here. So, here is the copy-paste of my initial bug report (but I confirm the bug is still present in 4.4.0). =============================== I'm using digikam for several years to handle all my photos. Regularly, some images are present in repository on disk, they are counted into the album list (for example, I see '8 (221)' that tells me there is 221 images in the album/directory named '8', and it is indeed what there is on filesystem). However, they are not displayed in the central part of digikam. I looked for lots of thing on Internet about this bug. I read and test lots of things. Eventually, I found the java program DigikamDBValidator. I know it is not maintain anymore and it is not necessarly fully working with recent version of digikam, but it allows me to start to look into the digikam database. And I found several problems. And theses problems come back (with other images) after I first corrected them. So the bug was present into the 4.0.0 digikam version. Upgrading to the 4.1.0 does not fix the database problems (but I do not know (yet) if 4.1.0 will corrupt the database again) For information, I'm using a mysql database for a long time (nearly when it has been available in the Debian package) First, I found several image related tables that refer to non existing images. For example, I get several rows with the following request : SELECT ImageComments.id FROM ImageComments LEFT OUTER JOIN Images ON Images.id=ImageComments.imageid WHERE Images.name IS NULL I correct them with : DELETE i FROM ImageComments i LEFT OUTER JOIN Images ON Images.id=i.imageid WHERE Images.name IS NULL; And I had this problem in ImageComments, ImageCopyright, ImageInformation, ImageMetadata, and ImageTags tables (I not sure I cheked all required tables, there can be more). That said, fixing this does not fix my problem with images not displayed. I found these image with this request : SELECT id FROM Images WHERE Images.id NOT IN (SELECT ImageInformation.imageid FROM ImageInformation); This are images into the 'Images' table but not into the 'ImageInformation' table. I get the path to these images with the following request : SELECT Images.id, concat(albumRoot, ":", relativePath, "/" , Images.name) FROM Images JOIN Albums on Images.album=Albums.id WHERE Images.id NOT IN (SELECT ImageInformation.imageid FROM ImageInformation); And indeed, the listed images where the ones counted into the album list but not displayed by digikam. I fix this by creating empty lines in the ImageInformation table (but this image ID): insert into ImageInformation (imageid) SELECT id FROM Images WHERE Images.id NOT IN (SELECT ImageInformation.imageid FROM ImageInformation); Starting digikam just after this 'fix' makes all my missing images displayed! Looking at the database, I needed to execute 'read metadata from images again' in the album menu (I'm not sure of the exact english text: my digikam is in french) in the album where I had missing images in order to got the ImageInformation fully filled. Needless to say that executing this entry menu before creating the lines in the ImageInformation table did not have any effect. It is something I tried lots of times. So, I'm not sure what leads to this database corruption but it is something I observe several times. And it happens with the 4.x series (ie, I'm sure I fixed the database while I was already using digikam 4.x and today I needed to fix it again). =============================== Regards, Vincent
This does look like solid is likely reporting the mounts as bad in this case.
Scott, you seem to better understand the report than me. Could you please explain where you see badly reported mount points?
I just got the problem again with the last digikam version in Debian (4:4.4.0-1.1+b2, ie upstream version 4.4.0) I just imported photo from a smartphone with the digikam importer in the zz_import/6 subalbum. Before the operation, digikam correctly shows the 9 images that were already present in zz_import/6. Then, I did a mass rename (with 'F2' and the rule '[date:yyyyMMdd_hh:mm]_[file]{range:1,}{lower}.[ext]{lower}') of all new photos. Then, I move all new photos to zz_import/4. At this point, in digikam, I only see the 9 old images that I did not touch. Correct. But, in fact, digikam tells me there is more image in the album (in the state bar at the bottom) and, indeed, on disk, there is still lots of files. It was between 100 and 200 images (sorry, I did not remember the exact number). So, I get again my notes about how to fix the databse. I closed digikam and started mysql and type: mysql> use digikamdb Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Database changed mysql> DELETE i FROM ImageComments i LEFT OUTER JOIN Images ON Images.id=i.imageid WHERE Images.name IS NULL; Query OK, 189 rows affected (0.02 sec) mysql> DELETE i FROM ImageCopyright i LEFT OUTER JOIN Images ON Images.id=i.imageid WHERE Images.name IS NULL; Query OK, 0 rows affected (0.12 sec) mysql> DELETE i FROM ImageInformation i LEFT OUTER JOIN Images ON Images.id=i.imageid WHERE Images.name IS NULL; Query OK, 142 rows affected (0.11 sec) mysql> DELETE i FROM ImageMetadata i LEFT OUTER JOIN Images ON Images.id=i.imageid WHERE Images.name IS NULL; Query OK, 152 rows affected (0.10 sec) mysql> DELETE i FROM ImageTags i LEFT OUTER JOIN Images ON Images.id=i.imageid WHERE Images.name IS NULL; Query OK, 464 rows affected (0.30 sec) mysql> select CONCAT(relativePath, '/', name) from Images LEFT OUTER JOIN ImageInformation on Images.id=ImageInformation.imageid LEFT OUTER JOIN Albums on Images.album=Albums.id where ImageInformation.imageid is NULL INTO OUTFILE '/tmp/digikam.log' ; Query OK, 146 rows affected (0.06 sec) I will add the /tmp/digikam.log to this bug. You will see that it contains file from /zz_import/6 and /zz_import/4. Files from /zz_import/4 are probably due to a previous database corruption that I did not notice (because there is too much photos in this album from me to notice the difference between the displayed photos and number of photo stated in the status bar). However, photos listed in /zz_import/6 are all photos I just imported/renamed. You can see that some (most) of them have their old name (IMG*) and a few have the new name (*_img_*) after the rename. So, for me, the corruption occurs during the mass rename operation. Just for the record, for other, I fixed all of this by : 1) touching all these files ($ touch zz_import/6/* ) 2) start digikam 3) touching againt all these files 4) restart digikam (perhaps, it is possible to be more efficient here) At this point, images where visible again in Digikam. I then needed to force the reload of all metadata in the zz_import/6 album I succeed but for the film (no exif metadata...) By chance, my smartphone include the date in the filename, so I've been able to manually fix the date for this file. I you need more specific information when I see this bug, please just tell me. Regards, Vincent
Created attachment 93207 [details] info extracted from the corrupted database See my bug comment to know how this file has been generated
I got no answer to comment #2, reassigning back to digikam developers.
Vincent, If the database is corrupted, did you tried to rebuild it with a new recent digiKam release. Last on is 4.11.0 commit with 90 bugs closed since 4.10.0. Gilles Caulier
Hi Gilles, I'm not sure to understand the version number you tell me about (4.11.0, 4.10.0). It is digikam version itself ? It is an internal version ? More over, do you have a document telling me how to do such a rebuild? If the rebuild comes only from on-disk files, I'm relunctant. Indeed, my current database contains lots of information that is not duplicated into files. The more obvious is the date of all my films that is only stored into the current database, not in the films themselves. That said, I hope you see that, in my last report, my problem comes from files I just imported. Perhaps there is another inconsistency in the database that triggered this behavior but it seems a bit strange to me. If you want that I look into the current database, just tell me. I know how to explore the mysql database. I can verify what you want. For now, my database seems clean wrt the small consistency checks I do : mysql> use digikamdb Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Database changed mysql> DELETE i FROM ImageComments i LEFT OUTER JOIN Images ON Images.id=i.imageid WHERE Images.name IS NULL; Query OK, 0 rows affected (0.02 sec) mysql> DELETE i FROM ImageCopyright i LEFT OUTER JOIN Images ON Images.id=i.imageid WHERE Images.name IS NULL; Query OK, 0 rows affected (0.11 sec) mysql> DELETE i FROM ImageInformation i LEFT OUTER JOIN Images ON Images.id=i.imageid WHERE Images.name IS NULL; Query OK, 0 rows affected (0.10 sec) mysql> DELETE i FROM ImageMetadata i LEFT OUTER JOIN Images ON Images.id=i.imageid WHERE Images.name IS NULL; Query OK, 0 rows affected (0.09 sec) mysql> DELETE i FROM ImageTags i LEFT OUTER JOIN Images ON Images.id=i.imageid WHERE Images.name IS NULL; Query OK, 0 rows affected (0.29 sec) mysql> select CONCAT(relativePath, '/', name) from Images LEFT OUTER JOIN ImageInformation on Images.id=ImageInformation.imageid LEFT OUTER JOIN Albums on Images.album=Albums.id where ImageInformation.imageid is NULL; Empty set (0.05 sec) Regards, Vincent
4.10.0 or 4.11.0 is digiKam version release. Gilles Caulier
Ok. Digikam is a software with lots of dependencies. I will wait for its packaging in Debian (even in experimental) and not try to recompile it myself. I will report back here if I observe the bug again (such as today) in a more recent version. That said, it is a bug I observe since a long time and through several releases. As a side note (it should probably not be in this bug report), I would very welcome any information telling me how to regenerate thumbnails for films. The 'Refresh' entry in the 'Albums' menu seems to do it for image but not for films. Regards, Vincent
Are you still able to reproduce the problem with digiKam5.0.0-Beta7 ?
This file still valid using last digiKam 5.0.0 ? Gilles Caulier
Hi, I install digikam from Debian packages but the 5.0.0 is not packaged yet (even in experimental). So I will test and report when it will be packaged (there are too many dependencies for digikam to recompile it locally without messing my system) Regards Vincent
Vincent, DigiKam 5.0.0 is out. Please see if you could still reproduce the problem and provide necessary updates, if any. Thanks
Vincent, DK 5.0.0 is packaged for Debian in Philips Johnson repository: https://launchpad.net/~philip5/+archive/ubuntu/extra Gilles Caulier
Swati, This problem cannot be reproduced with current implementation. I recommend to close it with 5.1.0 release. What do you think ? Gilles Caulier
Yes. I think this could be closed. I'll do it.