Version: 1.7.0 (using KDE 4.4.5) OS: Linux One raw file processed multiple times by ufraw, output as tifs with different names. These images are very different renditions visually. Also they all have different md5sums when running md5sum at the command line. In the digikam database, most of the renditions have the wrong thumb. So I created a test database with only 8 images, 2 raw files, one tiff from one of the raw files, several tiffs (visually very different renditions from each other) from the other raw file, and one jpeg from the raw file (probably not produced by ufraw). In digikam4.db there are 8 entries in the Images table, 5 of which have the same uniqueHash. in thumbnails-digikam.db there are only 4 thumbs. Right-clicking on the thumbs and selecting "edit" does open the correct image file, as does opening the preview. So I used ufraw to produce 3 tifs and 2 jpegs from the other raw file. The jpegs got different uniqueHashes, the tifs all share the same uniqueHash, giving me 13 images in the database, and only 7 uniqueHashes. Reproducible: Always Steps to Reproduce: Put a raw file into a directory. Open the raw file with ufraw. produce a tif. do it a couple more times, make the images look wildly different, so there is no question that the images are not the same. Save each time under a different name. Then open digikam and rescan the directory (or import a new collection if a different root). Actual Results: Use SQLite database browser to inspect the digikam data and thumbs databases. You'll see an entry in the images table for each tif, but they'll all share the same uniqueHash. Initially the images may or may not have different thumbs, but play around, the thumbs will collapse, so that all the images with the same uniqueHash now have the same thumb. Expected Results: I'd expect each tif-rendition/version of the original raw file, saved under different names, would have truly unique uniqueHashes, and would have their own correct thumbs. jpegs from ufraw don't seem to have this problem. I haven't checked other tif-producing software (but I will). Using exiftool to inspect a couple of the ufraw-produced tifs, it looks like ufraw 0.16 copies all the raw file data over to the tiff, so all the metadata in the two images looks (upon quick glance) to be identical. If uniqueHash is depending on metadata to generate uniqueHashes, then that could be the source of the problem. As md5 of itself is subject to hash collisions, it seems to me that in a large image database, using only a part of the image to calculate md5 hashes is not such a good idea, even apart from the current issue. As already stated, the actual md5 hashes of the images, as calculated by md5sum at the command line, are all different. (Probably a move to sha1 (over the whole image) would be overkill. And probably I don't know enough about hashes to even make these statements.)
Thanks a lot for your research, indeed this is a problem, known and solved (for the future). 1) This happens usually with TIFF images without metadata. The header of such files contains several kilobytes of (pretty useless) line offsets. I have not seen a JPEG which is affected 2) Computing the hash over the whole file is a major performance problem - scanning would take much longer. The old hash covered 99.9% of cases, we'll see what the new algorithm brings. 3) Some other problems in context of renaming are probably unrelated *** This bug has been marked as a duplicate of bug 210353 ***
Hi Marcel, Regarding, "This is a problem, known and solved (for the future). 1) This happens usually with TIFF images without metadata." In fact the affected images, tiffs output by UFRaw 0.16 and 0.17, have a LOT of metadata, all the metadata that was in the raw file (.cr2). If one were to use exiftool to add eg copyright information, keywords, contact information, location, etc.to one's raw files (which I do, in fact) there could be a whole lot of metadata in a raw file. Suspecting that a wealth of metadata could be the problem, I used exiftool to strip out all the metadata in the UFRaw-produced tiffs, and when I added the stripped tiffs to the digikam database, the stripped tiffs all had unique hashes and proper thumbs. Is the future solved bug version of digikam available somewhere? Elle Stone On 1/7/11, Marcel Wiesweg <marcel.wiesweg@gmx.de> wrote: > https://bugs.kde.org/show_bug.cgi?id=262452 > > > Marcel Wiesweg <marcel.wiesweg@gmx.de> changed: > > What |Removed |Added > ---------------------------------------------------------------------------- > Status|UNCONFIRMED |RESOLVED > Resolution| |DUPLICATE > > > > > --- Comment #1 from Marcel Wiesweg <marcel wiesweg gmx de> 2011-01-07 > 23:39:33 --- > Thanks a lot for your research, indeed this is a problem, known and solved > (for > the future). > > 1) This happens usually with TIFF images without metadata. The header of > such > files contains several kilobytes of (pretty useless) line offsets. I have > not > seen a JPEG which is affected > > 2) Computing the hash over the whole file is a major performance problem - > scanning would take much longer. The old hash covered 99.9% of cases, we'll > see > what the new algorithm brings. > > 3) Some other problems in context of renaming are probably unrelated > > *** This bug has been marked as a duplicate of bug 210353 *** > > -- > Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email > ------- You are receiving this mail because: ------- > You are on the CC list for the bug. > You reported the bug. >
Elle, Because Marcel work current on Google Summer of Code 2010 branch, i think it's fixed to 2.0.0 Gilles
Gilles, thanks. Can 2.0.0 be run alongside rather than in place of current digikam? Elle On 1/8/11, Gilles Caulier <caulier.gilles@gmail.com> wrote: > https://bugs.kde.org/show_bug.cgi?id=262452 > > > Gilles Caulier <caulier.gilles@gmail.com> changed: > > What |Removed |Added > ---------------------------------------------------------------------------- > CC| |caulier.gilles@gmail.com > > > > > --- Comment #3 from Gilles Caulier <caulier gilles gmail com> 2011-01-08 > 10:16:59 --- > Elle, > > Because Marcel work current on Google Summer of Code 2010 branch, i think > it's > fixed to 2.0.0 > > Gilles > > -- > Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email > ------- You are receiving this mail because: ------- > You are on the CC list for the bug. > You reported the bug. >
1.x does not know the new hash, so it will not open the database once you converted it to use the new hash with 2.0. You need to convert explicitly for this reason, there is an Update button at the bottom of the Database panel in the Settings dialog. Without this conversion, both version can operate on the same db, but your problem is not fixed.
Fixed with https://bugs.kde.org/show_bug.cgi?id=210353