Bug 226853

Summary: No images shown after upgrading from previous version image database
Product: [Applications] digikam Reporter: doma <kretebe>
Component: Database-SchemaAssignee: Digikam Developers <digikam-bugs-null>
Status: RESOLVED FIXED    
Severity: normal CC: caulier.gilles, dev+kde, Klaus.Weidenbach, marcel.wiesweg
Priority: NOR    
Version: 1.1.0   
Target Milestone: ---   
Platform: openSUSE   
OS: Linux   
Latest Commit: Version Fixed In: 1.2.0
Sentry Crash Report:

Description doma 2010-02-14 17:55:13 UTC 
Version:           1.1.0 (using KDE 4.4.0)
Compiler:          gcc
OS:                Linux (i686) release 2.6.25.20-0.5-pae
Installed from:    openSUSE RPMs

After updating digikam from version 0.9.3, to 1.1.0 and launching the application the welcome wizard appears asking for album and database file locations.

Both were kept previously at the same location on a mounted NFS4 share with properly set-up file locking (lockd) so the SQLite engine ran flawlessly with version 0.9.3. (took me a few days to get it working trying it with NFS3 in unsuccessfully). I tried all the below on a local drive as well just to make sure and the same thing happens.

After entering the locations the wizard recognized the previous database file and asked to choose between upgrading it or creating a new DB. As I had lots of tags/ratings I decided to go for the upgrade. It started crunching the files, it was 60+ GB so it took a while but finished successfully.

After the upgrade I could not see any images in the browser window.

The followings work properly:
- all albums were shown properly
- Creating/deleting new albums works properly
- all my old tags were imported properly 
- all import/export functions, bulk tools saw all images, the OpenGL image viewer displayed the slideshow properly

The following does not work:
- None of the original images are shown in the browser window
- Selecting "Scan for new images" scans all the images but still nothing is shown
- Making a change on the file system (renaming an album folder) and then "Scan for new images" scans all the images and properly detects the change in the album structure but no images are shown
- Selecting "Rebuild Thumbnails" starts rebuilding thumbnails but after completing still no images/thumbnails are shown

The following scenario I carried over might help to tackle the issue:
- Creating a new album named Test - success
- Importing an image file from a memory card a.jpg works and the newly imported image's thumbnail shows properly. All image related functions work properly (tried: editing, saving, emailing, assigning tags, modifying EXIF data)
- mv a.jpg b.jpg - digikam detects the change and the image is displayed correctly
- cp b.jpg c.jpg - digikam detects the change and both images are displayed correctly
- cp ../Existing_album/existing_file.jpg ./d.jpg - the new image d.jpg is not shown (../Existing_album/existing_file.jpg was in the old DB so subject to the upgrade process)
- mv d.jpg e.jpg - nothing changes only b.jpg and c.jpg are displayed (same image)
- mv e.jpg c.jpg - whoops, digikam detects the change and the second thumbnail is replaced with the new content
- cd ../Existing_album
- touch * - nothing happens, still no images are displayed


This is where I stopped. Somewhere the watching algorithm for directory content changes an the DB upgrade output mess up with each other I suspect, knowing virtually nothing of digikam internals.
Comment 1 Klaus Weidenbach 2010-02-14 18:36:08 UTC 
This sounds exactly like the problem I posted yesterday in digikam-devel. I can confirm this. I use KDE 4.3.4 and upgrading from digiKam 0.9.6.

I can add these observations to the ones already in the report:
Calender view only shows the numbers of video and sound files in parentheses and I can see these videos items there only, no pictures. The same for Timeline view. Tags view shows the correct numbers of all items in parentheses, but I just see videos. If I go to Searches and search for something I see the pictures that fit to my search!
Comment 2 doma 2010-02-14 18:41:59 UTC 
I can confirm Klaus' observations it is the same with me, videos show correctly, with proper thumbnails.
Comment 3 caulier.gilles 2010-02-14 18:42:01 UTC 
Marcel,

KIOSlave or Database relevant ?

Gilles
Comment 4 doma 2010-02-14 18:45:39 UTC 
When choosing "Create new Database" (this is how I use it for now) digikam does a full rebuild from scratch - everything works properly.

This tells to me that nor KIOSlave neither Database is likely to be of any relevance. This must be something to to with the upgrade itself.
Comment 5 Marcel Wiesweg 2010-02-14 22:45:58 UTC 
Gilles, it's not the ioslaves, because new pictures are displayed. Klaus will send me his database. Currently I dont know what's wrong here.
Comment 6 Marcel Wiesweg 2010-02-15 19:42:22 UTC 
Ok, this is an interaction between a change in the ImageScanner (to not overwrite the rating on rescan) and the schema update. Noone has tested schema update recently.

The only problem is that there is no entry in the ImageInformations table. The ImageScanner now uses UPDATE instead of REPLACE.

It is almost fortunate that anyone hit by this bug will notice instantly ;-)
Comment 7 Marcel Wiesweg 2010-02-15 21:05:38 UTC 
SVN commit 1090694 by mwiesweg:

For now just add the ImageInformation entry when schema upgrading.

A better solution (update or insert) in the Image scanner leads to
the SQL "upsert" problem, for which I found no good solution for SQLite.

CCBUG: 226853

 M  +9 -0      schemaupdater.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=1090694
Comment 8 Marcel Wiesweg 2010-02-15 21:06:20 UTC 
Klaus or doma, can you test current SVN? Remove your digikam4.db and re-upgrade. I hope it works then.
Comment 9 doma 2010-02-15 23:16:08 UTC 
Am checking it out from SVN, takes a decent while. 

I may not have a proper build environment but will give it a try.

In the meanwhile are you basically saying that a

"REPLACE INTO ImageInformation (imageId) SELECT id FROM Images;"

query will do the trick after the conversation as a quick workaround?
Comment 10 Klaus Weidenbach 2010-02-15 23:39:55 UTC 
I couldn't test SVN yet. Have to prepare some things first. But I patched 1.1.0 sources and it is not working there: "The schema updating process from version 4 to 5 failed, caused by an error that we did not expect...."
But the SVN file also looks bit different, so I will try with SVN next, but I don't think I can do it before next weekend.
Comment 11 doma 2010-02-16 00:23:07 UTC 
Did the conversion again.

Then executed:

REPLACE INTO ImageInformation (imageId) SELECT id FROM Images;

Now it works flawlessly thanks.

Compiling from source is beyond my possibilities now.

Thanks again.
Comment 12 Marcel Wiesweg 2010-02-16 18:22:21 UTC 
Doma, you will probably not have got your ratings and comments from the old db now (the statement need to be executed at a certain point during the conversion process)
Comment 13 Marcel Wiesweg 2010-02-16 18:30:28 UTC 
SVN commit 1091099 by mwiesweg:

Hope this works now.

CCBUG: 226853

 M  +1 -5      schemaupdater.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=1091099
Comment 14 Marcel Wiesweg 2010-02-16 18:31:19 UTC 
Klaus: patching 1.1.0 is sufficient. Please try with this corrected patch.
Comment 15 Klaus Weidenbach 2010-02-16 22:51:03 UTC 
The new patch against 1.1.0 seems to work. The import was successful and I can see now all pictures in all views. Tags and ratings seem to be there, too.
As far as I can see this bug is fixed. Thank you very much!
Comment 16 doma 2010-02-17 13:56:05 UTC 
I backed up the tables changed after my initial conversion, chechked out from SVN, compiled and did the conversion again. I can confirm it works for all items. Thanks.
Comment 17 Marcel Wiesweg 2010-03-15 20:50:01 UTC 
*** Bug 230606 has been marked as a duplicate of this bug. ***