Version: (using KDE 3.5.10) Compiler: gcc 4.2.4 (Ubuntu 4.2.4-1ubuntu3) OS: Linux Installed from: Compiled From Sources When I download images, I take the CF card from my camera, plug it into the cardreader of my PC and then I get a pop-up asking me what to do (open in window, download pictures with digikam, etc.). This doesn't happen anymore! (I don't know why) However, that isn't the real problem, I think. When I open the medium using the menu in digikam (import -> browse /dev/sde1, or similar), I get a list of all the pictures, but not actually all of them. I shoot RAW+JPEG, so I have f.arw and f.jpg files on them, but digikam only shows me the .arw files at first. I then download these, delete them all and then, take out the card (after unmount), re-insert it and THEN I can see all the .jpg files, which I can then also download and delete. This is totally different behaviour from before, when I used to see them both at the same time in the import window. I guess this is a regression! (Did someone think having the same first part with two extensions was a bug and only the first should be shown?) Cheers Simon
Oh and BTW, I now see that all of my .arw files are smaller and corrupted when downloaded! I'm using the KDE3 svn version of 0.9.5 (with my ratiocrop patch, but that shouldn't matter)
I have this same error using 0.10.x from svn on kde 4.2rc1 (kubuntu intrepid packages from ppa) The card I'm using is working fine and the actual image files are not corrupted on the card, only after downloading by digikam... (This would seem like a blocker bug if confirmed...) Cheers Simon Please let me know if you need more info!
Choose a file. Access the mountpoint with dolphin and copy the file to your harddisk. Confirm that it is not corrupted. Then access the card with digikam, download the same one file, and confirm that it is corrupted. Watch for any error messages on the console.
I've done just this, and via dolphin there's no problem, the file is 100% ok, it's around 11 MB large and loads fine in ufraw (or digikam/editor for that matter) digikam gives some output on the console, but none very spectacular I think: QGridLayout: Multi-cell fromRow greater than toRow QGridLayout: Multi-cell fromRow greater than toRow QGridLayout: Multi-cell fromRow greater than toRow QGridLayout: Multi-cell fromRow greater than toRow Starting to load Plugins. Files: "CompassFloatItem.so" Files: "MapScaleFloatItem.so" Files: "MarbleCrosshairsPlugin.so" Files: "MarbleGeoDataPlugin.so" Files: "MarbleOverviewMap.so" Files: "MarbleStarsPlugin.so" Files: "NavigationFloatItem.so" === MarbleDirs: === Local Path: "/home/simon/.marble/data" Plugin Local Path: "/home/simon/.marble/plugins" Marble Data Path (Run Time) : "" Marble Data Path (Compile Time): "/usr/share/kde4/apps/marble/data" Marble Plugin Path (Run Time) : "" Marble Plugin Path (Compile Time): "/usr/lib/kde4/plugins/marble" System Path: "/usr/share/kde4/apps/marble/data" Plugin System Path: "/usr/lib/kde4/plugins/marble" =================== "CompassFloatItem.so" - "/usr/lib/kde4/plugins/marble/CompassFloatItem.so" "MapScaleFloatItem.so" - "/usr/lib/kde4/plugins/marble/MapScaleFloatItem.so" "MarbleCrosshairsPlugin.so" - "/usr/lib/kde4/plugins/marble/MarbleCrosshairsPlugin.so" "MarbleGeoDataPlugin.so" - "/usr/lib/kde4/plugins/marble/MarbleGeoDataPlugin.so" "MarbleOverviewMap.so" - "/usr/lib/kde4/plugins/marble/MarbleOverviewMap.so" "MarbleStarsPlugin.so" - "/usr/lib/kde4/plugins/marble/MarbleStarsPlugin.so" "NavigationFloatItem.so" - "/usr/lib/kde4/plugins/marble/NavigationFloatItem.so" Time elapsed: 5 ms Use workaround: 0 loadMapTheme "earth/citylights/citylights.dgml" loadMapTheme "earth/bluemarble/bluemarble.dgml" MapThemeId "earth/srtm/srtm.dgml" loadMapTheme "earth/srtm/srtm.dgml" DGML2 Name : "Atlas" adding container: "cityplacemarks.kml" false starting parser for "cityplacemarks" "Loading Default Placemark Cache File:/usr/share/kde4/apps/marble/data/placemarks/cityplacemarks.cache" adding container: "baseplacemarks.kml" false starting parser for "baseplacemarks" "Loading Default Placemark Cache File:/usr/share/kde4/apps/marble/data/placemarks/baseplacemarks.cache" Loading ended true adding container: "elevplacemarks.kml" false adding container: "otherplacemarks.kml" false starting parser for "elevplacemarks" starting parser for "otherplacemarks" adding container: "boundaryplacemarks" true "Loading Default Placemark Cache File:/usr/share/kde4/apps/marble/data/placemarks/otherplacemarks.cache" starting parser for "boundaryplacemarks" Loading ended true Loading ended true "Loading Default Placemark Cache File:/usr/share/kde4/apps/marble/data/placemarks/elevplacemarks.cache" "Loading Default Placemark Cache File:/usr/share/kde4/apps/marble/data/placemarks/boundaryplacemarks.cache" Loading ended true Loading ended true THEME CHANGED: *** "earth/srtm/srtm.dgml" SunLocator::setBody( QString ) DatasetProvider "1:3:0" DatasetProvider "1:0:0" DatasetProvider "1:2:0" DatasetProvider "1:1:0" DatasetProvider "1:1:1" DatasetProvider "1:2:1" Style reset requested. Style reset requested. Style reset requested. Style reset requested. start generate indexes generated indexes MarblePlacemarkModel (generateIndex): Time elapsed: 270 ms Style reset requested. Style reset requested. Error: Directory SubImage1 with 33639 entries considered invalid; not read. Error: Directory SubImage1 with 33639 entries considered invalid; not read. Error: Directory SubImage1 with 33639 entries considered invalid; not read. RESET started RESET stopped Detecting maxLabelHeight ... Error: Directory SubImage1 with 33639 entries considered invalid; not read. timeChanged void SunLocator::update() Error: Directory SubImage1 with 33639 entries considered invalid; not read. Error: Directory SubImage1 with 33639 entries considered invalid; not read. Error: Directory SubImage1 with 33639 entries considered invalid; not read. Error: Directory SubImage1 with 33639 entries considered invalid; not read. Error: Directory SubImage1 with 33639 entries considered invalid; not read. Error: Directory SubImage1 with 33639 entries considered invalid; not read. Error: Directory SubImage1 with 33639 entries considered invalid; not read. The file downloaded by digikam is 34.3 KiB, the original is 11.8 MB I'll attach the corrupted file. Cheers Simon
Created attachment 30648 [details] corrupted sony a100 .arw raw image file this was downloaded using digikam 0.10 svn r916749 from a cardreader mounted CF card of a sony a100. It's supposed to be a sony raw file, the original size is file is 11.8MB
Which digiKam version you use ? for KDE3 or KDE4 ? Which options you use in camera interface to download file ? Which camera driver you use (look Camera informations dialog) Gilles Caulier
Start digiKam froma console. Download some ARW files and look all messages printed. Gilles Caulier
Gilles: http://bugs.kde.org/show_bug.cgi?id=180065#c4
Strange ther eis no messages relevant of camera interface in #4 The string : "Error: Directory SubImage1 with 33639 entries considered invalid; not read." ... come from Exiv2 libary used to manage metadata. But with ARW files (all raw files in fact), we don't touch anything. The file is copied as well. No more. Here, i have a Minolta Dynax 5D. I have no problem to download MRW raw files. Which Exiv2 library you use on your computer ? Go to Help Components Info dialog for details. Also, try to recompile digiKam using full debug option and try again. Look in README file for details. Gilles Caulier
AFAIK the wiki instructions for compiling have fulldebug in all the options of the cmake line, so I assume that it's already compiled with full debugging. I didn't change anything in the lines from the instructions. This is the link where I got the instructions and I followed them to the letter. http://wiki.kde.org/tiki-index.php?page=Digikam+Compilation+on+Kubuntu+Intrepid The exiv2 version I use is the released exiv2-0.18 (file dates around 16-12-2008) from a tarball.
FYI: from the help -> component info digiKam version 0.10.0-rc2 (rev.: 916823) Exiv2 can write to Jp2: Yes Exiv2 can write to Jpeg: Yes Exiv2 can write to Png: Yes Exiv2 can write to Tiff: Yes Exiv2 support XMP metadata: Yes LibCImg: 129 LibExiv2: 0.18 LibJPEG: 62 LibJasper: 1.900.1 LibKDE: 4.1.96 (KDE 4.1.96 (KDE 4.2 RC1)) LibKExiv2: 0.6.0 LibKdcraw: 0.5.0 LibLCMS: 116 LibPNG: 1.2.27 LibQt: 4.4.3 LibRaw: 0.7.0-Alpha4 LibTIFF: LIBTIFF, Version 3.8.2 Copyright (c) 1988-1996 Sam Leffler Copyright (c) 1991-1996 Silicon Graphics, Inc. Marble widget: pre 0.7 SVN LibGphoto2: 2.4.2 LibKipi: 0.4.0
Do you have any of the options on the camera gui right sidebar activated? I see the string "digikam-0.10.0-rc2" when I open the attached file in a binary editor, and this should certainly not happen when the file is untouched straight from the camera. Switch off all "File Renaming" and "On-the-fly" operations.
All these operations that you mention should not apply to RAW files, so whether I've set these or not, it shouldn't matter when I download .ARW files from my memory card. I do believe I've set options like case setting (all uppercase files from my camera) and set author, but they do not apply to raw files and if digikam tries to apply them to raw files, that is a bug, IMO. Would it be useful to upload my configuration file of digikam? Which files should I upload? Cheers Simon
Simon, Perhaps your problem is relevant of a stupid bug in libkexiv2. Look here : http://bugs.kde.org/show_bug.cgi?id=182412 Please, checkout libkexiv2 code from svn trunk and recompile, install it, and try again Gilles Caulier
Simon, just try to switch off all options. If this fixes the bug, then we found the bug. This is the idea behind this. Of course it is a bug if digikam applies options to RAW files to which it cannot write, but we are hunting for a bug here.
Gilles, Marcel, I'll try both suggestions, first the update of libkexiv2 and then turn off "all" options, however could you be specific about what is in "all" options? Cheers Simon
I think I found it, in the jpeg options of the download dialogue, the setting "set default photographer identity" seems to attempt modification of the RAW image file and break it in the process...
It still valid after to have updated libkexiv2 ? Gilles Caulier
Markus, HAs you have a Sony camera and you shot ARW photo, can you reproduce the problem ? Gilles Caulier
Yes Gilles, it is still valid after the upgrade, unless I did something wrong ;-) I ran svn update of kdegraphics and rebuilt libkexiv2, installed them and then started digikam. Or should I have updated digikam as well? Cheers Simon
Just tried with 0.10.0rc2, SVN 883547 under Kubuntu Intrepid Ibex, KDE 4.1 Importing via digikam Sony .arw and .jpg files via a USB CF cardreader (vivanco brand) imports all files without and corruption. jpg files display without failures in digikam editor or gimp, arw files can be opened and converted in bibble. Not a single hint of a problem here on my side.
In theory, yes, because binary compatibility is respected... But to be sure, try to recompile and install digiKam. Gilles
Markus, Are you set the same downloading options than Simon ? Gilles
OK, my first test was with no changing options in the sidebar dialogue of the download window active, i.e. no renumbering, no writing of photographers identity etc. there were no corruptions visible. A second test with active writing of a photographers identity during download, set from the download window options dialog, indeed resulted in writing of the raw and jpeg files. Now neither the jpg nor the raw files create thumbnails in digikam, they are displayable outside of digikam in gimp (jpg) resp. bibble (arw), but again the metadata of the arw files are corrupted and create problems in bibble. A cross-check showed that with this options "off", the files are imported again without problems. I guess its a problem in exiv2 (I am using 0.18pre2) and digikam - at least the raw files should not be modified.
I updated and recompiled/installed digikam svn, same problem and I see Markus also confirmed this bug. Corrupted file when setting the option "Set default photographer identity", no corruption when un-setting this option. /Simon
NB: the setting that affects my raw files is under "On the fly operations (JPG only!)", which makes it harder to track down when it affects RAW files...
Marcel, Look in code: http://lxr.kde.org/source/extragear/graphics/digikam/utilities/cameragui/cameracontroller.cpp#523 This is camera controller code which play with image metadata. we call DMetadata::applyChanges() which call Kexiv2:save(). In this method we test if Raw files is supported : http://lxr.kde.org/source/KDE/kdegraphics/libs/libkexiv2/libkexiv2/kexiv2.cpp#308 So in case of ARW, nothing must be changed in metadata. Note : we can test if it's really a JPEG file in camera controller before to play with DMetadata. Marcus, Can you give the console trace when an ARW file is downloaded (and touched of course) Gilles Caulier
First: "JPEG-only" is obviously wrong, at least the exiv2 related options are done for all supported files. And now this is the problem here: digikam(7684)/KEXIV2 KExiv2Iface::KExiv2::save: File Extension: "digikam-camera-tmp1-7684" is supported for writing mode We operate on a temp file which does not have any file suffix.
Yes, Marcel, i just discovered this problem. A fix is on my computer... I will extend "JPEG only" operation to PNG/TIFF/JPEG2000 later 0.10.0. Gilles
SVN commit 919053 by cgilles: digiKam from trunk: Camera GUI : append orginal file name to temp url used to download file from camera (not prepend). Like this, original file type mime is not lost and JPEG operation during download are not applied to RAW files. CCBUGS: 180065 M +1 -1 cameracontroller.cpp WebSVN link: http://websvn.kde.org/?view=rev&revision=919053
Great that you found the fix! however I must note that mime types should not be inferred from filenames or their extensions, this may be ok for 99% of all situations, but it's safer to test a file based on the internal file structure (like the file command does). Anyway, I'm already quite happy to know that the problem should now be fixed... Cheers Simon
Well, try to detect type mime use file content is impossible to do... with RAW file. Why??? because camera maker have the good idea to use a derivated TIFF/EP format to make RAW files... For example KDELib typemime detection method find TIFF for ARW. Do you feel the problem ? This is why in digiKAm we only use file extension... (else it will be easy of course) I would to thanks camera maker to increase complexity of photo workflow !!! This is why i'm a support DNG format (it's not perfect of course, but it's a first start) Gilles Caulier
SVN commit 919064 by cgilles: digiKam from trunk : to prevent any problem, only apply JPEG fix to JPEG files BUG: 180065 M +42 -39 cameracontroller.cpp WebSVN link: http://websvn.kde.org/?view=rev&revision=919064