Created attachment 159418 [details] screen capture SUMMARY Difficult to update an image. STEPS TO REPRODUCE 1. Outside digiKam 8.0.0, modify some information about some images. For example, use exiftool directly to add geo‐location to a photograph. Or change the timestamp of a file (I’m not speaking about the Exif time, although the result should be the same). 2. Rescan your image (after a digiKam restart or manually refresh a folder). OBSERVED RESULT Metadata is usually not updated in digiKam. After the metadata update, I tried many things to force digiKam to update the metadata: - restart digKam to force it to update the files, - delete the files then restore them, - rescan the folder, - rename the files, - have a new identical copy in the system file manager. They all fail. In the screen capture, you can see that digiKam actually updates something, but not all. − The geo‐location appears in the Metadata pane, but the globe icon is not presented for any of these photographs. - I can’t upload these files to iNaturalist because they are “not located”. - However the Geolocation Editor can pin the photograph where it belongs. EXPECTED RESULT The metadata should be updated. SOFTWARE/OS VERSIONS Windows: 7 ADDITIONAL INFORMATION
What do you seen in debugview while you process the operations ? https://www.digikam.org/contribute/ Gilles Caulier
In a test here on Windows, I can't reproduce any problem, the metadata is updated in the database. In addition to the DebugView log, a sample image from which no GPS information is read might also be important. Maik
So, I started a new digikam profile and a test folder with one geo‐located (see the globe for that one) and one not. I worked on a copy of the latter, added geo‐location externally with exiftool, then rescanned the folder. The globe is not present, but the GPS data is somehow detected. Then the other consequences should be as in the original description. Attached the sample working image and the debug log as requested.
Created attachment 159439 [details] Debug log
Created attachment 159440 [details] Sample image not geo‐located
Created attachment 159441 [details] relevant screen capture
Your sample image definitely has no GPS information, just a GPSVersionID. Exiv2 and ExifTool agree on that. Maik
That’s what I said I’ve done. Feel free to add GPS data with exiftool and rescan it.
I suspect you uploaded the wrong image, in your log and screenshot the image has the addition "Copie". By the way, the log shows no problems. One more note about screenshot, until before digiKam-8.0.0 digiKam would not have accepted the GPS information without AltitudeRef. Please upload the correct sample image. Maik
No, as I said, I modified the copy myself, so if you want to retest the new metadata you have to work from the original again.
Maybe the log shows no problem, but the screen shots each show the same problem, including the original one with altitude. The last test was a quick test with dummy GPS data, but the original screen capture is as complete as possible. Both show the same problem.
(In reply to Maik Qualmann from comment #9) > I suspect you uploaded the wrong image, in your log and screenshot the image > has the addition "Copie". By the way, the log shows no problems. One more > note about screenshot, until before digiKam-8.0.0 digiKam would not have > accepted the GPS information without AltitudeRef. Please upload the correct > sample image. > > Maik Did you try to reproduce the external geo‐location addition and rescan?
When I add coordinates with this ExifTool command there are no problems: exiftool Minuartia\ viscosa\ 5.JPG -gpslatitude=53.86 -gpslongitude=11.19 -gpslatituderef=N -gpslongituderef=E I assume you don't define GPSLatitudeRef and GPSLongitudeRef. But without these two details, they are not valid coordinates and cannot be decoded. Now the question arises whether we really have to interpret here and whether we have to accept positive GPS Latitude and GPS Longitude as N and E and vice versa for negative values S and W. Or whether we insist on the Ref information... Maik
Read this text in the ExifTool Doc: https://exiftool.org/TagNames/GPS.html ------------------- When adding GPS information to an image, it is important to set all of the following tags: GPSLatitude, GPSLatitudeRef, GPSLongitude, GPSLongitudeRef, and GPSAltitude and GPSAltitudeRef if the altitude is known. ------------------- Also read this thread: https://exiftool.org/forum/index.php?topic=8443.0 Without this information it is not a valid GPS. digiKam does nothing wrong here. Maik
Why no GPS icon is displayed in your first screenshot because it contains Ref values cannot be judged without the sample image. Maik
(In reply to Maik Qualmann from comment #15) > Why no GPS icon is displayed in your first screenshot because it contains > Ref values cannot be judged without the sample image. > > Maik Thanks for checking. Again, the last test was just a quick test and can’t draw detailed conclusions. Going back at the original example, thanks for trying, but I still think we are not on the right track yet. As I said, it’s not the specific file, it’s how digiKam 8.0.0 manages the file updates. I didn’t notice this flaw in 7.* versions. Here are two captures from my main profile and two GPS locations. They have the same data structure, but one has been updated before and the one that has been updated while being known by digiKam makes digiKam not handle its metadata correctly. The proof that it’s not the file itself is that if I start a new profile from scratch, then every file is new and then the same files are considered valid (last screen capture).
Created attachment 159450 [details] Screen capture with main software profile, a file that has been accepted
Created attachment 159451 [details] Screen capture with main software profile, a file that digiKam doesn’t update because it’s been updated externally
Created attachment 159452 [details] Screen capture with main software profile, a file that digiKam doesn’t update because it’s been updated externally
Created attachment 159454 [details] Screen capture with a new digikam profile, therefore every file is new, therefore file updates haven’t been performed, therefore the same “faulty” files are not a problem at all
(In reply to Maik Qualmann from comment #14) > When adding GPS information to an image, it is important to set all of the > following tags: GPSLatitude, GPSLatitudeRef, GPSLongitude, GPSLongitudeRef, > and GPSAltitude and GPSAltitudeRef if the altitude is known. > ------------------- > > Also read this thread: > > https://exiftool.org/forum/index.php?topic=8443.0 > > Without this information it is not a valid GPS. digiKam does nothing wrong > here. As an irrelevant reply for this side subject, I update my files externally with exiftool because I geo‐locate them with a .nmea file which is not supported by digiKam (only gpx, although it shouldn’t make any difference for digiKam if it only transfers it to exiftool in the back). For someone who doesn’t know, nmea is the most complete GPS capture, therefore all the possible information is there and the relevant ones are then transferred to the photographs. OK, let’s go back to the bug.
Reread the metadata in the item menu still doesn't show a geolocation icon? If not, please create a log with this reread metadata action. Note that the metadata option "Rescan file when files are modified" should be enabled if you want digiKam's initial start scan to already update the GPS information. Maik
Yes we are coming back to the same report: remember I used many ways to force digiKam to update its data (probably all the possible imaginable ways). Remember also that it did read the metadata after the external update since it presents it in the panel. But it only half updates it, because there is no globe and it confuses other functions (like update to iNaturalist), by letting them think there is no geo‐location. And the log provided didn’t seem to show where in digiKam the bug is located.
Are sidecars possibly involved? Do you have sidecar reading enabled? Note that GPS coordinates from XMP in digiKam take precedence over EXIF metadata. Maik
No, I don’t use extra metadata files. Most of the photographs are JPEG, therefore there is no need for xmp files since metadata can stay in the very file.
So, after various tests on Windows, I can't reproduce the problem. There is one more thing, since the last versions of Exiv2 (also 0.27.5) the Unicode support in Windows is broken. If your image path contains characters that are outside of the current Windows code page, problems may arise. Check this out, please. Otherwise we need the image with the problem and a real log (not a log from a test collection) when updating the metadata of the corresponding image. If not publicly possible, to my private mail. Maik
Hi Maik, This means that if we go back to the Exiv2 0.27-maintenance branch under Windows, will this problem also remain ? Gilles
Maik, I double check and the Exiv2 0.27-maintenance branch still have the EXV_UNICODE_PATH support. This problem must only touch Exiv2 0.28 Gilles
The explanations given in this thread, even if they are a little bit old, explain well the mess under Windows about string encoding: https://stackoverflow.com/questions/402283/stdwstring-vs-stdstring Gilles
From Exiv2 git/main branch (0.28 and later), the Exiv2::Image API provides : std::string support : https://github.com/Exiv2/exiv2/blob/main/src/image.cpp#L794 --> do not work with wide string char under Windows. byte* support: https://github.com/Exiv2/exiv2/blob/main/src/image.cpp#L801 It use internally a post conversion of the open(BasicIo::UniquePtr io). I have no idea if a wide string char support under Windows will work here... Gilles
In Exiv2 main/branch, BasicIO is only based on std::string. So the Unicode support under Windows is completely broken in Exiv2 0.28 as far i understand... Gilles
It was removed with this commit because of supposedly "dead" code...well, "dead" code was extremely important to Windows. https://github.com/Exiv2/exiv2/commit/7933ff401ddb9768502c36b1febea06b854e176a The MSVC build has Exiv2-0.27.5 and the unicode support didn't work either, but that's probably because Exiv2 wasn't compiled with unicode support. Maik
yes probably compilation option was not passed to the MSVC build to support UNICODE. I will switch Windows build to the 0.27-maintenance branch until this situation is fixed to Exiv2 main branch. so, now we have a Q : why this report is for digiKam 8.0.0, where we use Exiv2 0.27-maintenance ? Note: how Exiv2 team can decide to drop this important unicode support code in the library ? I comment the Exiv2 report, wait and see a feedback, if any... Gilles
I want mean this UPSTREAM bug from Exiv2 : https://github.com/Exiv2/exiv2/issues/2637
Ok, got the last answer from: https://github.com/Exiv2/exiv2/issues/2637 So Windows7 is out (obsolete anyway) and probably only works with an MSVC build? https://stackoverflow.com/questions/67848972/differences-between-msvcrt-ucrt-and-vcruntime-libraries Maik
UPSTREAM comment posted : https://github.com/Exiv2/exiv2/issues/2637#issuecomment-1577969708
I suspect that the problem here has nothing to do with the unicode problem either, but without a real log and the corresponding image, it's just poking around in the dark. Maik
(In reply to Maik Qualmann from comment #26) > So, after various tests on Windows, I can't reproduce the problem. There is > one more thing, since the last versions of Exiv2 (also 0.27.5) the Unicode > support in Windows is broken. If your image path contains characters that > are outside of the current Windows code page, problems may arise. Check this > out, please. > Otherwise we need the image with the problem and a real log (not a log from > a test collection) when updating the metadata of the corresponding image. If > not publicly possible, to my private mail. > > Maik Thanks Maik for trying to reproduce. I’m convinced it’s not a problem with the image files (they are scanned fine if they are new to a new digiKam profile) and it’s not because of Unicode characters (some of them do have Unicode and some don’t, and again, both are scanned fine if digiKam doesn’t know them in advance). The Unicode exiv2 discussion might be a different subject than this bug — to be confirmed. But because you’re asking I’m going to send you the files. BTW, I still use Windows 7 because I’ve never had the time to reinstall all my software on Windows 10. Maybe next winter.
Git commit 37c680073ffa9ac53e056c71dd8fbc0b7178c455 by Gilles Caulier. Committed on 06/06/2023 at 08:37. Pushed by cgilles into branch 'master'. under Windows, we needs to use 0.27-maintenance branch for the moment. Related: bug 469372 M +2 -2 project/bundles/3rdparty/ext_exiv2/CMakeLists.txt https://invent.kde.org/graphics/digikam/-/commit/37c680073ffa9ac53e056c71dd8fbc0b7178c455
It is probably possible to compile MinGW with UCRT: https://stackoverflow.com/questions/57528555/how-do-i-build-against-the-ucrt-with-mingw-w64 Maik
This is also interesting with small patches for MXE: https://octave.discourse.group/t/windows-utf-8-and-ucrt-yet-another-variant-of-octave-on-windows/1821 Maik
Hum, question must be posted on MXE github issue... Gilles
This bug is not only about geo‐location. I’ve just discovered a few files that happened to have no CreateDate field (which forced digiKam to use the DateTimeCreated field). I updated them through the -csv=csv file option from exiftool. Then I rescanned the folder in digiKam. Surprise (or not so much anymore): these jpegs are still presented at the wrong date in the Date selection tree and keep that date underneath in Digikam. So I tried the brutal method again: I created a new digiKam profile. Surprise (or not): these files now have the date they deserve in digiKam. As a side note, these files are named nothing fancy, like 24.jpg. No extra Unicode. Maybe more importantly I should add, my exiftool pre‐made commands bear the flags -overwrite_original -P , which shouldn’t be a problem for digiKam as it used to rescan the files correctly on demand in the past.
Created attachment 159498 [details] Log for comment 43 Here is the log for the last updated files. I opened the main digiKam profile which automatically updates files at startup, but I also waited 3 minutes and I forced a rescan of the folder which contains the files with updated CreateDate.
The Qt debug variable is probably not set, or internal debugging is not activated in the settings. The log does not show any digiKam debug messages. If the metadata is re-read, the date found is output, none of it is in the log. Maik
Sorry, I forgot to set the option in the main profile (too much profile fiddling). I sent you the new full debug log privately. I hope this finally helps.
Your log doesn't show any errors. However, it seems that you have not activated the metadata option "Rescan file when files are modified". This is important in your case, however, so that a changed file is completely rescanned. This also works with the ExifTool "-P" option, since the file size is always changing. Then they probably pressed F5. And this is where many people have a misunderstanding. The "F5" function would find new items and always recreates all thumbnails. However, it does not update changed files. To re-read changed files Menu::Album->Reread Metadata From Files - or - Menu::Item->Reread Metadata From File. Maik
(In reply to Maik Qualmann from comment #47) > Your log doesn't show any errors. However, it seems that you have not > activated the metadata option "Rescan file when files are modified". This is > important in your case, however, so that a changed file is completely > rescanned. This also works with the ExifTool "-P" option, since the file > size is always changing. > > Then they probably pressed F5. And this is where many people have a > misunderstanding. The "F5" function would find new items and always > recreates all thumbnails. However, it does not update changed files. > To re-read changed files Menu::Album->Reread Metadata From Files - or - > Menu::Item->Reread Metadata From File. > > Maik Thank you Maik for your analysis. Indeed, I didn’t know about that option that is not active by default. With it, the files now have the full detail. I still think that it’s very strange that the data was correct in the metadata panel but not in the main view. Something to reconsider probably. Indeed, when I asked digiKam to refresh, I right clicked on the file folder and selected the refresh option. The nuance with the Reread option was too subtle. For clarity, maybe call that option Scan for new files?
Also as I said, I don’t think I encountered the metadata not refreshing in digikam 7, so it might be something new in v. 8.
@maison, What the status of this entry ? Best regards Gilles Caulier
We can close this bug for now, unless other people have the same needs for clarification.