Bug 317210

Created attachment 78300 [details]
debug console when deleting, and reattempting delete.

No problem is relevant of Database which host thumbnails images...

Gilles Caulier

Looking in the db directly, i can still see it. Should it be gone?

mysql> select * from Images where name='20130322_liam_9828.JPG';
+-------+-------+------------------------+--------+----------+---------------------+----------+----------------------------------+
| id    | album | name                   | status | category | modificationDate    | fileSize | uniqueHash                       |
+-------+-------+------------------------+--------+----------+---------------------+----------+----------------------------------+
| 61593 |   876 | 20130322_liam_9828.JPG |      1 |        1 | 2013-03-22 18:04:50 |  4627091 | 93356016d4136a3c7385acb4f7a55c5c |
+-------+-------+------------------------+--------+----------+---------------------+----------+----------------------------------+
1 row in set (0.00 sec)


mysql> select * from UniqueHashes where uniqueHash='93356016d4136a3c7385acb4f7a55c5c';
+----------------------------------+----------+---------+
| uniqueHash                       | fileSize | thumbId |
+----------------------------------+----------+---------+
| 93356016d4136a3c7385acb4f7a55c5c |  4627091 |   76927 |
+----------------------------------+----------+---------+
1 row in set (0.00 sec)

Perhaps its also useful to know this is ubuntu 12.10 quantal with 2.8 previously. I noticed this problem, so upgraded to 3.0 using Philip Johnsson PPA hoping newer build fixed it.

I dont see debug output from digikam, only kio which is irrelevant. Please enable 50003 in kdebugdialog. Does pressing F5 fix the issue (=> force rescan)?

Oh, you are right. Pressing F5 makes the image from the gallery disapear :)

But, changing albums, and even restarting digikam does not!

Is this expected bahaviour? Do you still want the debug output?

Yes, please give a full debug output starting the program, delete one image (not recognized), close.
At least on Linux, file watch should work absolutely flawless, we use inotify directly.

OK, so added image "test.JPG" and closed digikam.

Started debug console, and digikam. Deleted image from digikam. Waited 5 seconds for it to do something. Tested UI, it was responsive and ready. Closed digikam.

Attached is the output from console.

Created attachment 78435 [details]
open digikam, delete image, fail recognise, close digikam

Also, a couple of days ago i went to create the above log, and it actually recognised the deletes. I almost fell of my chair. But, now its not working again. There has not been any changes to packages in the last few days. If i see it recognising the deletes again, will capture the console.

Comment on attachment 78435 [details]
open digikam, delete image, fail recognise, close digikam

Opening inotify looks all right, but the deletion is not recognized. Don't know why at the moment.

Same here. Pressing F5 also removes the thumbnail from the strip, but if I happen to be focused on a thumbnail I previously deleted then after pressing F5 focus jumps back to the first image in the strip, so I can't use the "next image" shortcut, which makes this an annoying bug.

digiKam version 3.1.0
Exiv2 can write to Jp2: Yes
Exiv2 can write to Jpeg: Yes
Exiv2 can write to Pgf: Yes
Exiv2 can write to Png: Yes
Exiv2 can write to Tiff: Yes
Exiv2 supports XMP metadata: Yes
LibCImg: 130
LibClapack: external shared library
LibExiv2: 0.23
LibJPEG: 80
LibJasper: 1.900.1
LibKDE: 4.10.2
LibKExiv2: 2.3.0
LibKGeoMap: 2.0.0
LibKdcraw: 2.2.0
LibLCMS: 119
LibPGF: 6.12.27 - external shared library
LibPNG: 1.5.13
LibQt: 4.8.4
LibRaw: 0.15.0-Beta1
LibTIFF: LIBTIFF, Version 4.0.2 Copyright (c) 1988-1996 Sam Leffler Copyright (c) 1991-1996 Silicon Graphics, Inc.
Marble Widget: 0.15.1 (stable version)
Parallelized PGF codec: No
Parallelized demosaicing: No
RawSpeed codec support: No
Database backend: QSQLITE
Kipi-Plugins: 3.1.0
LibKface: 2.0.0
LibKipi: 2.0.0
LibOpenCV: 2.4.3
Libface: 0.2

*** Bug 269577 has been marked as a duplicate of this bug. ***

*** Bug 297546 has been marked as a duplicate of this bug. ***

No update on this issue? It is pretty bothering.

Haven't noticed it in recent versions.

I also have not observed this on recent builds. As the original requestor, i think this can be resolved. Unless others can still reproduce.

I experienced it in git master actually... but I can't reproduce at the moment.

I appear to get this problem too. It been in existence for a long time.
If you try to delete a second time it complains that the file isn't there but then the thumbnail disappears.
Even worse though, when deleting photos that are apparently still there, digikam then deletes a DIFFERENT (real) file and its impossible to restore it.

Camera - Nikon CoolpixS9100 connected in PTP mode.

Fixed by 4.12.0 as far as I can tell.

Not for me. I still experience the described behaviour.

I use digikam5.0.0-beta6

Steps to reproduce:
1)Create a new folder and add some images.
2)Go to the thumbnail view.
3) Move an image to trash.

Actual results
The image is removed from file system and the thumbnail doesn't exist anymore in digiKam.

Expected results
Same as actual results.

When you said that the thumbnail do not exists, you want mean in thumbnail DB ?

It's true for all DB type ?

I'm surprised because a garbage collector to clean up DB entries about non valid items have never been implemented, as i know...

Perhaps with an item removed from DK interface it will work, but what about to remove file form hard drive by another application while DK session is down. I'm sure that nothing is removed from the DB. But this case is fully relevan of a future garbage collector tool for DB maintenance (i think another bugzilla entry exist about this topic).

Gilles Caulier

(In reply to caulier.gilles from comment #23)

I understand you now. I think I misinterpreted the issue faced by the user. I just checked if thumbnail is removed the DK interface. The file still exists in my Image table. 

I apologize for the mistake.

Using 7fb769c27cf48408cf45cad65ba4330940663c05

I imported over 100 photos. Back in the main digiKam window, once importing finished, I went to the new album, clicked on the last thumb, jumped 50 photos back using the "left arrow" key, then I held shift and clicked on the first photo in the album, to select the first >50, then I shift+deleted them. The confirmation popped up, I agreed, and digiKam deleted the photos... only the thumbnails have not disappeared.
https://i.imgur.com/8bHLn3j.png

Clicking F5 reloaded the album and then the deleted photos disappeared.

Using the SQLite database.

SO it's just a refresh problem. Right ?

Gilles Caulier

Can you run digiKam from a console and post here the trace of operations when you try again a similar deletion of files ?

Gilles Caulier

Yes, looks like just a refresh problem.

I tried to reproduce the problem from a console now by just copying an existing album under a new name, starting digiKam and deleting photos from the new album, but the thumbs were correctly removed from digiKam, so I couldn't reproduce. I will run it from a console the next time I import and delete and report back to you.

Thumbnails remain after file deletion in version 5.0.0.0.Beta5 on Fedora Linux 24:
Version     : 5.0.0
Release     : 0.10.beta5.fc24

For example, I have 127 images in a directory. Thumbnails of all original photos appear. After I process each image, I save a new version, e.g., pix-01.jpg saved as pix-01_v1.jpg. Later I remove the original photos, select pix-01.jpg and move it to Trash. 

The original file is moved to Trash, but the thumbnail remains in Album view.
When I try to delete the file a second time, an error message appears:

  File pix-01.jpg does not exist

When I try View -> Refresh, placeholder images (a bare tree at sunset?) appear for thumbnails. Thumbnails and associated metadata seem to remain in any case.

Also, additional directories appear within my current directory in the Album menu (left sidebar). They show different numbers of images. Now the left sidebar shows four subdirectories ("albums") in the album I'm using, but actually there are none.
  
The bug and another one (missing sidebar menu icons) appeared after I upgraded Fedora 23 to 24 and then digiKam from version 4.x to 5.0.0.Beta5).

Thumbnails remain after file deletion in version 5.0.0.0.Beta5 on Fedora Linux 24:
Version     : 5.0.0
Release     : 0.10.beta5.fc24

For example, I have 127 images in a directory. Thumbnails of all original photos appear. After I process each image, I save a new version, e.g., pix-01.jpg saved as pix-01_v1.jpg. Later I remove the original photos, select pix-01.jpg and move it to Trash. 

The original file is moved to Trash, but the thumbnail remains in Album view.
When I try to delete the file a second time, an error message appears:

  File pix-01.jpg does not exist

When I try View -> Refresh, placeholder images (a bare tree at sunset?) appear for thumbnails. Thumbnails and associated metadata seem to remain in any case.

Also, additional directories appear within my current directory in the Album menu (left sidebar). They show different numbers of images. Now the left sidebar shows four subdirectories ("albums") in the album I'm using, but actually there are none.
  
The bug and another one (missing sidebar menu icons) appeared after I upgraded Fedora 23 to 24 and then digiKam from version 4.x to 5.0.0.Beta5).

Swati,

For this file, outside the fact that a DB garbage collector need to be write to clean up the database (there is one other bug entry about this topic), it must miss a call to a sql query somewhere in thumbnails database interface when file is removed from the collection. 

Gilles Caulier

Using digiKam 5.0.0, I did the following with MySQL DB:

1)Add photos in a new album.
2)In thumbnails view, move multiple images to Trash.

In UI: The images are no more visible in Thumbnails view.

In DB: 
After adding images, they're visible correctly in Images table.
After deletion, images still present in DB but with album column = "NULL". If we restore these images, then NULL entries remains in the table, and new rows are added for those images.

It's clear. The table in DB must be cleaned properly. That all..

Probably the same dysfunction must exist with sqlite.

Gilles Caulier

What's happen if you delete an album as well ? All item entries in DB are properly removed ?

Gilles Caulier

I think that what happens now is partially right. 'Cause when an item is "Moved to trash", it is just removed and not permanently deleted. So it is right to exist in table with album=NULL

The issue lies when that item is "permanently" deleted and there's no change in the Images table.

Hi, I just uploaded a patch in https://bugs.kde.org/show_bug.cgi?id=374591 that introduces garbage collection as maintenance stage. This cleans the core, thumbs and recognition db from stale entries. But be advised: The patch is still in testing phase. I did some tests on my own databases and had no problems. But just to be sure, backup your databases before you test.

New 5.5.0 AppImage is done with garbage database collector patches.
Uploading to GDrive is under progress. It will be online in few minutes at usual
place :

https://drive.google.com/drive/folders/0BzeiVr-byqt5Y0tIRWVWelRJenM

New database Garbage Collector options are there :

https://www.flickr.com/photos/digikam/32549923912/in/dateposted-public/
https://www.flickr.com/photos/digikam/32549923632/in/dateposted-public/

Gilles Caulier

Git commit 834221c8d1300bbdd4ae78e345cdc0fce3e92be8 by Mario Frank.
Committed on 09/02/2017 at 20:51.
Pushed by mfrank into branch 'master'.

Updated the news. The newly introduced garbage collection gets rid of thumbnails in DB that are not referenced by any image any more.
FIXED-IN: 5.5.0

M  +2    -1    NEWS

https://commits.kde.org/digikam/834221c8d1300bbdd4ae78e345cdc0fce3e92be8