Bug 175923 - digikam uses wrong album root path/wrong disk uuid
Summary: digikam uses wrong album root path/wrong disk uuid
Status: RESOLVED UPSTREAM
Alias: None
Product: digikam
Classification: Applications
Component: Database-Schema (show other bugs)
Version: 1.0.0
Platform: Fedora RPMs Unspecified
: NOR normal
Target Milestone: ---
Assignee: Digikam Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2008-11-23 19:37 UTC by Roland Wolters
Modified: 2017-07-25 19:37 UTC (History)
5 users (show)

See Also:
Latest Commit:
Version Fixed In: 1.0.0


Attachments
Solid output with external disk attached (41.13 KB, text/plain)
2008-11-24 00:33 UTC, Roland Wolters
Details
Solid output withou the attached disk. (37.63 KB, text/plain)
2008-11-24 00:39 UTC, Roland Wolters
Details
$ solid-hardware list details (34.67 KB, application/octet-stream)
2009-10-11 08:27 UTC, Milan Knížek
Details
HAL does not list md's (50.14 KB, application/octet-stream)
2010-09-09 23:39 UTC, Harald Nikolisin
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Roland Wolters 2008-11-23 19:37:57 UTC
Version:            (using KDE 4.1.3)
Installed from:    Fedora RPMs

Digikam only shows the included albums when I have my backup disk attached - although all paths in digikamrc and even in the digikam4.db-file are set right.
When I start Digikam without the external disk I simply get an empty digikam. If I remove the folder in the settings and re-add it, the problem persists. I have to remove the database file to get rid of this problem.

Background information:
My files are in /home/liquidat/Bilder/Photoalben/* (several folders). I have an additional external usb disk with a backup of these data under /media/BackupDisk/Backup/Bilder/Photoalben/*.

My guess:
I opened the database file and looked around, and finally found this:
sqlite> select * from AlbumRoots;
          id = 1
       label =
      status = 0
        type = 1
  identifier = volumeid:?uuid=313804a2-7421-49cd-a6be-b689886a4dea
specificPath = /home/liquidat/Bilder/Photoalben
The uuid is in fact the one of my external hard disk - my local hard disk is bound in by VolumeManagement:
$ mount
...
/dev/mapper/VolGroup00-LogVol01 on /home type jfs (rw)
...

So, first of all, how could it ever happen that the wrong uuid was ever chosen? And how can I correct it now - a VM ususally does not has a uuid.
Comment 1 caulier.gilles 2008-11-23 20:46:04 UTC
Yes, same problem here with my VirtualBox image

This week end, Marcel has fixed some issue about UUID provide by KDE4 Solid interface. This must solve your problem.

Please, test with current implementation from svn (beta6).

Thanks in advance

Gilles Caulier
Comment 2 Marcel Wiesweg 2008-11-23 22:11:48 UTC
Can you give the output of "solid-hardware list details"? HAL may have problems with RAID or volume management, but in this case there should be a fallback on mount path.

Apart from HAL limitations, I dont see why logical volume dont have uuid, if there is a filesystem you can have a uuid.
Comment 3 Marcel Wiesweg 2008-11-23 22:14:36 UTC
To be more specific: We should check Solid output with your USB disk plugged and unplugged.

What happens if you add any folder in your home dir when your USB disk is unplugged?
Comment 4 Roland Wolters 2008-11-24 00:32:09 UTC
Gilles, unfortunately I can't check this right here with svn6, but maybe the next days.

Marcel, I will add the two outputs - but what do you mean with "what happens"? When I add a folder in my home dir the folder is created in the normal way. The rest of the Linux/KDE environment is totally unaffected, I think. Or do you mean inside Digikam?
Comment 5 Roland Wolters 2008-11-24 00:33:22 UTC
Created attachment 28783 [details]
Solid output with external disk attached

The requested output with the external disk attached. It is an external hard disk mounted at '/media/EternalBackup'.
Comment 6 Roland Wolters 2008-11-24 00:39:30 UTC
Created attachment 28786 [details]
Solid output withou the attached disk.

Here is the file without the external disk.

One thing I forgot:
The real system's hard disk is '/dev/sda'. That one is split into 'sda1' and 'sda2'. 'sda1' is mounted to '/boot'. 'sda2' is encrypted and contains the VM. Inside the VM there are three partitions, '/', '/home' and '/media/local'.
Comment 7 Marcel Wiesweg 2008-11-24 20:35:47 UTC
What gives us mount path and status is StorageAccess.filePath and StorageAccess.accessible.
You can see that your local partitions are not found at all.
I dont quite understand the entry for /dev/sdb1 (mounted with empty file path), but at least your backup disk is recognized.

There is probably a bug in digikam handling empty mount paths and missing information, I will try to fix it.

If you feel like you can file a bug for HAL (or rather append to any existing bug report) if recent versions of HAL still fail to handle volume management.
Comment 8 Marcel Wiesweg 2008-11-24 20:58:19 UTC
SVN commit 888548 by mwiesweg:

Ensure that volumes are mounted and that spurious empty mount paths are taken care for.

CCBUG: 175923

 M  +6 -4      collectionmanager.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=888548
Comment 9 Roland Wolters 2008-11-24 21:20:06 UTC
Marcel, I'm not a HAL guru, so I'm not sure what the output should look like. However, to make taht clear: my external partition is - of course - also encrypted. So the partition is inside an encrpyted container, and that encrypted container lies on '/dev/sdb1'. The partition itself, once encrypted, lies in '/dev/mapper', that's the usual way for luks encrypted devices.

I would file the bug report against HAL - but right now I'm not sure what the actual bug is. :/
Comment 10 Marcel Wiesweg 2008-11-24 21:57:06 UTC
Look here:
https://bugs.freedesktop.org/show_bug.cgi?id=6427

A bug filed in 2006, current status unclear from the report.
Comment 11 Marcel Wiesweg 2008-11-29 16:35:29 UTC
Roland, if you test with beta6 or current svn, does digikam now accept your local directories, just storing the mountpath to identify (volumid:?path), and not playing tricks on you with your backup disk?
Comment 12 caulier.gilles 2008-12-03 16:24:51 UTC
Roland,

digiKam 0.10.0-beta6 is out. Please give us a fresh feedback about your problem. thanks in advance

Gilles Caulier
Comment 13 Roland Wolters 2008-12-03 20:44:32 UTC
Sorry for the late answer, but I had to wait until the beta6 was build for Fedora 10. Next time I will build the packages locally (in worst case with the patches you provide me).

Unfortunately, the bug is still valid:
sqlite> select * from AlbumRoots;
          id = 1
       label =
      status = 0
        type = 1
  identifier = volumeid:?uuid=313804a2-7421-49cd-a6be-b689886a4dea
specificPath = /home/liquidat/Bilder/Photoalben

digikam --version
Qt: 4.4.3
KDE: 4.1.3 (KDE 4.1.3)
digiKam: 0.10.0-beta6

Do I need to change anything manually in the database file? And if so, how?
Comment 14 Marcel Wiesweg 2008-12-04 11:11:06 UTC
You removed your old collection with the setup dialog and added a new one for your path?

If yes, hm. Then I have to rethink again. (In that case, you could try to remove the albumroot from the DB: DELETE FROM AlbumRoots WHERE id=1; and add a new one with the setup dialog)

If no, then please do so and report if your local directory is still recognized as your USB harddisk when the disk is plugged in, and what happens if you remove+add when the disk is not plugged in.
(If you have edited information in your DB file you dont want to lose, make a backup, we can restore that afterwards)

Thank you!
Comment 15 Roland Wolters 2008-12-08 00:52:25 UTC
Marcel,

I removed the collection and re-added it via the configuration window, and now the collection is shown!
The db file says now:

sqlite> select * from AlbumRoots;
          id = 1
       label = Photoalben
      status = 0
        type = 1
  identifier = volumeid:?uuid=41e057b3-f84b-4f17-bf17-10468d744f50
specificPath = /home/liquidat/Bilder/Photoalben

Btw., most of my tags and so on are lost now (yes, I have backuped the data before), but I guess it should be enough to replace the volume id in the backuped file?
Comment 16 Marcel Wiesweg 2008-12-08 22:00:52 UTC
Yes it should.
I dont understand now where this UUID comes from (not in the output from Solid, updated HAL?) and why tags are lost, but yes try to replace the uuid in the db file and I am happy if everything works for you.
Comment 17 Roland Wolters 2008-12-15 19:06:12 UTC
The unknown number is my fault - I had to re-format my partition before I deleted and re-added the database. The setup changed, there is no VM anymore, but a normal '/' and an encrypted '/home/' partition. The new number is (!) listed in lshal and solid-hardware correctly:
$ lshal|grep 10468d744f50
udi = '/org/freedesktop/Hal/devices/volume_uuid_41e057b3_f84b_4f17_bf17_10468d744f50'
  info.udi = '/org/freedesktop/Hal/devices/volume_uuid_41e057b3_f84b_4f17_bf17_10468d744f50'  (string)
  volume.uuid = '41e057b3-f84b-4f17-bf17-10468d744f50'  (string)
$ solid-hardware list|grep 10468d744f50
udi = '/org/freedesktop/Hal/devices/volume_uuid_41e057b3_f84b_4f17_bf17_10468d744f50'

This corresponds to my main hard disk partition, /dev/sda1, which corresponds to my root partition.

I'm sorry that I forogt to add this, I totally forgot about the re-format.

But the good news is: I re-created the old database, changed the value to the new uuid (which is not the one the database is actually on, but it is at least one of the system) and it works now.

However, I wonder why you need the uuid at all for such an album: what will happen when I buy a new machine and restore the backup, will my database still work?
Comment 18 Marcel Wiesweg 2008-12-16 17:41:28 UTC
Yes you are right, we need at least provide a simple tool to change the uuid in cases of full backup restoration.

For a plain standard use case using mount path is sufficient. With UUID mobile device identification comes for free, and as soon you think of sharing a database file, using it from Windows and Linux (as soon as KDE Windows implements Solid properly).
Note that there is still support for identifying by mount path, it is just that digikam will not use it if it finds a better way.
Comment 19 Marcel Wiesweg 2009-01-18 22:42:56 UTC
Functionality will not be implemented for 0.10.0. On my TODO list for post 0.10.0.
Comment 20 caulier.gilles 2009-01-19 08:49:11 UTC
MArcel,

I just created 0.10.1-svn release tag in bugzilla.

Gilles
Comment 21 Marcel Wiesweg 2009-04-11 18:04:59 UTC
*** Bug 189362 has been marked as a duplicate of this bug. ***
Comment 22 Marcel Wiesweg 2009-07-02 17:54:34 UTC
SVN commit 990542 by mwiesweg:

Implement automatic migration for a case when a "hard-wired" collection is no longer found.
This may happen if the user restores a backup, changes the file system etc.
In this case, we scan the available volumes for migration candidates, where the old
relative path exists. These candidates are presented for choice.
In the most common case (one data partition, UUID changed) there will be exactly one
candidate here.
The second option is to mark a collection as removable, if autodetection failed or
the user likes to use his screwdriver a lot.
Third option is to do nothing ("I would like to solve the problem later using the setup dialog").
In this case, however, the message will just popup at every startup.

BUG: 175923

 M  +2 -1      NEWS  
 M  +108 -3    digikam/albummanager.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=990542
Comment 23 Milan Knížek 2009-10-10 21:00:43 UTC
This bug is marked as fixed, but I have experienced the same troubles with 1.0.0beta5 (rev. 1022810).

I have migrated my disk to LVM2, which resulted in changed UUIDs. On startup, digiKam did not find the existing collection (in the same path as before: /media/Data/Photo/Images - it is not a removable partition, even that is in /media directory) and offered to mark it as "on removable drive" or "do nothing". I chose the latter option.

Then, I manually edited digikam4.db UUID value in RootAlbum for the partition having the actual images, but no go.

I had a first success with setting UUID of the first partition in the system (/boot on sda2). digiKam opened fine and showed the albums/thumbnails in the collection.

However, when I rescanned the collection for new images, the collection disappeared immediately again (nothing in album list). During scanning, the popup window showed two steps (Preparing for scanning, Scanning for removed albums). The settings showed a different path (with /boot prefixed: the path does not exist), while db still shows the original (and existing) path.

After that, I had no success changing the UUIDs in db.

Finally, I solved that by creating a symlink /boot/media to /media. On startup, digiKam offered to set the lost collection as the one existing on /boot/media/Data/Photo/Images.

Note that on each trial, I started with the original db restored from a backup.

$ ls -la /dev/disk/by-uuid/
drwxr-xr-x 2 root root 100 2009-10-10 21:20 .
drwxr-xr-x 6 root root 120 2009-10-10 21:20 ..
lrwxrwxrwx 1 root root  10 2009-10-10 21:20 d2c4afc9-c7e0-4ce0-89a5-d3989e14c5c3 -> ../../sda2
lrwxrwxrwx 1 root root  20 2009-10-10 21:20 6425bcd4-3bc1-4ff1-a167-e0afe92d1a00 -> ../../mapper/vg-swap
lrwxrwxrwx 1 root root  20 2009-10-10 21:20 6954fd86-3273-4361-9583-b46841303c2f -> ../../mapper/vg-root

The collection is on "vg-root" (which is mounted as /).

$ mount (I removed extra lines of tmpfs, etc)
/dev/mapper/vg-root on / type ext4 (rw,relatime)
/dev/sda2 on /boot type ext3 (rw)
/home on /home-bind type none (ro,bind)
/home/modeluser/.Private on /home/modeluser type ecryptfs (blabla...)

So, I can see two problems:
x Bug re. UUID from another partition still exists
x Scanning for existing collections on startup is done only on one partition (sda2) and not on the other (vg-root).
Comment 24 Marcel Wiesweg 2009-10-10 21:41:31 UTC
Can you attach the output of "solid-hardware list details" and give me the contents of the AlbumRoots table of the DBs before and after?
Comment 25 Milan Knížek 2009-10-11 08:27:26 UTC
Created attachment 37505 [details]
$ solid-hardware list details
Comment 26 Milan Knížek 2009-10-11 08:33:00 UTC
db contents of db backup:
$ sqlite3 "digikam4 (before_lvm_migration).db"
SQLite version 3.6.10
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite> select * from AlbumRoots;
1|Images|0|1|volumeid:?uuid=cad56872-889c-4f3b-ae9c-a5367acbcecd|/media/Data/Photo/Images

db contents of a working db (collection on lvm disk, using the /boot/media symlink workaround):
$ sqlite3 digikam4.db 
SQLite version 3.6.10
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite> select * from AlbumRoots;
1|Images|0|1|volumeid:?uuid=d2c4afc9-c7e0-4ce0-89a5-d3989e14c5c3|/media/Data/Photo/Images

In the Settings, digikam shows the path to the collection as "/boot/media/Data/Photo/Images"
Comment 27 Marcel Wiesweg 2009-10-16 21:42:11 UTC
The solid-hardware dump shows that HAL recognizes only the /boot partition (probably non-LVM2). This explains why your hack is needed.

This is not a digikam problem; you probably need to upgrade HAL. I short googling on HAL and LVM2 showed this:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=510546
Comment 28 Harald Nikolisin 2010-09-09 23:39:46 UTC
Created attachment 51491 [details]
HAL does not list md's
Comment 29 Harald Nikolisin 2010-09-09 23:40:33 UTC
unfortunately I got the same problem.
background was replacing a RAID1 system with a new RAID1 system - all files were rsync'ed.

problem is that even if I want to change the UUID, I have no idea which one should I take..

The fotos are stored on /dev/md4 which consists of /dev/sda7 and /dev/sdb7.
HAL gives me no UUID of the md's - should I take on of the sd?7 devices?

output of "solid-hardware list details" attached (previous comment)
Comment 30 Marcel Wiesweg 2010-09-10 14:01:18 UTC
I can only confirm what you've already found out, that HAL is completely broken for your setup.
Remains the "traditional" fallback solution:
identifier "volumeid:?path=/home/xy/complete/path", specificPath "/" should work IIRC.
This would also be used if you'd newly add your collection now.
You know your way with the sqlite3 tool?
Comment 31 Harald Nikolisin 2010-09-12 23:27:49 UTC
marcel,

thanks for the comment. If I understand you I can change volumeid also with a path (instead of an UUID)?

and no, I'm not familiar with the sqlite3 command line. After opening digikam4.db and "select * from AlbumRoots;" I can see my 4 collections with the uuid's

Can you give me the command syntax for changing these entries?
The first entry is:

1|foto|0|1|volumeid:?uuid=0aafe707-32e8-4258-adb5-38561c282490|/foto

and it is actually located in /somewhere/foto
Comment 32 Marcel Wiesweg 2010-09-13 18:30:03 UTC
UPDATE AlbumRoots SET identifier='volumeid:?path=/somewhere/photo', specificPath='/' WHERE id=1;

did not test right now but it should work. "id" is the first number you get in the table dump.
Comment 33 Harald Nikolisin 2010-09-13 19:49:43 UTC
it did the job :)
thanks for the quick response!