Bug 480606 - Digikam git doesn't display thumbnails from tif images cropped with Rawtherapee
Summary: Digikam git doesn't display thumbnails from tif images cropped with Rawtherapee
Status: RESOLVED FIXED
Alias: None
Product: digikam
Classification: Applications
Component: Plugin-DImg-TIFF (show other bugs)
Version: 8.3.0
Platform: Compiled Sources Linux
: NOR normal
Target Milestone: ---
Assignee: Digikam Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2024-01-31 16:12 UTC by maderios
Modified: 2024-02-03 14:17 UTC (History)
2 users (show)

See Also:
Latest Commit:
Version Fixed In: 8.3.0
Sentry Crash Report:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description maderios 2024-01-31 16:12:10 UTC
Digikam git version doesn't display thumbnails from tiff images cropped with Rawtherapee.
To reproduce:
- In  Digikam open tif file "test.tif" with Rawtherapee
- In Rawtherapee, crop test.tiff  image, save  new image "test-cropped.tif" as tif 8bit in same folder than original
- Update folder
- "test.cropped.tif" thumbnail is displayed but image is not cropped
- click on "test.cropped.tif", full cropped image is displayed now
This issue doesn't happen with jpeg files. The issue doesn't come from Rawtherapee because Gimp displays  this tiff cropped image normally.
This issue doesn't happen with cropped tif image created with Gimp
Cropped file and original file here:
https://www.dropbox.com/scl/fi/ds5ukxx8irgthmswkh4oy/moose-tracks-cropped-RT.tif?rlkey=a05tcqf1kox66xv8ywse8bhn0&dl=0
https://www.dropbox.com/scl/fi/i0obrab8cu0ql6zvbz7au/moose-tracks.tif?rlkey=41fbcf90f2kkpkst18tnsjx1r&dl=0
Arch Linux system
Qt5
rawtherapee-git 1:5.9.r474.g255a0c708-1
libtiff 4.6.0-2
Comment 1 Maik Qualmann 2024-01-31 16:58:19 UTC
At the moment I only have the Windows version of digiKam-8.3.0, no problems, both TIFF files have thumbnails and can be opened.

Maik
Comment 2 caulier.gilles 2024-01-31 17:19:16 UTC
TIFF can use special compress algorithm as ZIP. libtiff must be compiled with this kind of support (optional). If RawTherapee enable an exotic compression method, this is not portable.
Comment 3 caulier.gilles 2024-01-31 17:25:01 UTC
Look tiff compresion notes in wikipedia page :

https://en.wikipedia.org/wiki/TIFF
Comment 4 Maik Qualmann 2024-01-31 17:33:22 UTC
Both TIFF files are also displayed here under my Linux version. The original is uncompressed, the cropped is compressed with Adobe Deflate.

Maik
Comment 5 Maik Qualmann 2024-01-31 17:40:22 UTC
If you're concerned about the appearance/size of the cropped image, it's because there's a preview image included that Rawtherapee hasn't adjusted when I compare the bytes. Now digiKam uses the preview image for performance reasons if it is available.

Maik
Comment 6 Maik Qualmann 2024-01-31 17:52:21 UTC
Git commit e59b62583ab0cdb48fd76bd5de4e4aca02094b22 by Maik Qualmann.
Committed on 31/01/2024 at 17:51.
Pushed by mqualmann into branch 'master'.

add remove IPTC preview image to the BQM tool

M  +11   -3    core/dplugins/bqm/metadata/removemetadata/removemetadata.cpp

https://invent.kde.org/graphics/digikam/-/commit/e59b62583ab0cdb48fd76bd5de4e4aca02094b22
Comment 7 Maik Qualmann 2024-01-31 17:56:15 UTC
The error here is with Rrawtherapee that the IPTC preview image is not updated. With the last commit you have the option to remove it.
Alternatively, save the image again with the digiKam editor, we update the preview image.

Maik
Comment 8 maderios 2024-01-31 18:06:32 UTC
(In reply to Maik Qualmann from comment #1)
> At the moment I only have the Windows version of digiKam-8.3.0, no problems,
Issue happens with Rawtherapee git version
> both TIFF files have thumbnails and can be opened.
To be clear:
I can open both files and display thumbnails too but Digikam doesn't display image cropped thumbnail, it displays two same thumbnails, as if one of these images has not been cropped.  
I tested with Rawtherapee 5.9, issue is not present, cropped image thumbnail is displayed normally. Something has changed with Rawtherapee git version. 
It seems RT uses deflate  compression https://github.com/Beep6581/RawTherapee/blob/dev/rtengine/imageio.cc

> 
> Maik
Comment 9 maderios 2024-01-31 18:09:35 UTC
(In reply to caulier.gilles from comment #2)
> TIFF can use special compress algorithm as ZIP. libtiff must be compiled
> with this kind of support (optional). If RawTherapee enable an exotic
> compression method, this is not portable.

Rawtherapee use deflate compression (see the code in my post below)
Comment 10 Maik Qualmann 2024-01-31 19:25:19 UTC
I have Rawtherapee 5.9 here, this version removes the IPTC preview, so the thumbnail in digiKam is correct. If the Rawtherapee git version keeps the IPTC preview but does not update it, that is a bug.

Maik
Comment 11 maderios 2024-01-31 20:26:30 UTC
Same issue in Digikam with tiff files cropped with Rawtherapee git version.  
No issue with two file managers, Dolphin 23.08.4-1 and Pcmanfm-Qt  1.4.0.  They display cropped cropped file/ tiff thumbnails normally. 
It seems bug comes from Digikam
Comment 12 Maik Qualmann 2024-01-31 20:33:41 UTC
Do not you understand? digiKam uses the embedded preview image in the TIFF image, if it exists, to create thumbnails faster. Rawtherapee removed this thumbnail in older versions, so digiKam created the thumbnail from the full TIFF - all good. Now the “old” preview image remains in the TIF, which of course we then use and the display is not correct.

So either Rawtherapee removes the preview image as before or saves a correct one.

Maik
Comment 13 maderios 2024-01-31 20:46:36 UTC
I understand what you mean, I have to investigate and may be, report issue upstream.
Thanks for these explanations
Comment 14 maderios 2024-02-01 11:32:15 UTC
I get same issue after rotating image with Gimp 2.10. 
Open tiff image in Digikam with Gimp
In Gimp, rotate and save it
New tiff image/rotated/thumbnail is not displayed in Digikam, only original tiff image/thumbnail is displayed.
Clicking on this new thumbnail/image displays it normally/rotated
Comment 15 Maik Qualmann 2024-02-01 11:41:08 UTC
As I said, we use the embedded preview image to create the thumbnail. If it is not updated by Gimp and other programs, it will display incorrectly. We can disable that. But then with large TIFF files created by digKam, the creation of new thumbnails will be significantly slower.
However, I am of the opinion that the corresponding programs, if a preview is available, also have to update it.

With digiKam-8.3.0 you can remove the preview image with the BQM, then everything will be correct again.

Maik
Comment 16 Maik Qualmann 2024-02-01 11:55:26 UTC
Another fact is that if programs like Gimp or Rawtherapee do not update the embedded preview image, you think you have removed sensitive information with a crop, but it is still in the preview image.

Maik
Comment 17 maderios 2024-02-01 12:42:43 UTC
(In reply to Maik Qualmann from comment #15)
> As I said, we use the embedded preview image to create the thumbnail. If it
> is not updated by Gimp and other programs, it will display incorrectly. We
> can disable that. But then with large TIFF files created by digKam, the
> creation of new thumbnails will be significantly slower.
Slowness is not a problem if it solves issue
> However, I am of the opinion that the corresponding programs, if a preview
> is available, also have to update it.
Reasonably, I can't report that to Rawtherapee and Gimp upstream, it concerns Digikam
> 
> With digiKam-8.3.0 you can remove the preview image with the BQM, then
> everything will be correct again.
In BQM/batch queue manager,  I don't see any option or function  to remove the embedded preview image
Comment 18 Maik Qualmann 2024-02-01 12:49:48 UTC
The function was just added yesterday, it is Comment 6. In the Metadata Remover in the IPTC section.

Maik
Comment 19 maderios 2024-02-01 13:19:52 UTC
Yes, remove IPTC metadata works in BQM but i see XMP tags are no more displayed in thumbnail after removing IPTC
To reproduce, in BQM:
Metadata
remove metadata
IPTC/preview image is checked
XMP and exif are not checked
Comment 20 Maik Qualmann 2024-02-01 17:45:39 UTC
I can't reproduce it, I used your sample image for testing, XMP remains. Please test it again.

Maik
Comment 21 maderios 2024-02-01 19:09:30 UTC
Strangely, it keeps metadata now, it works. I don't understand what happened, sorry for the noise.......
Comment 22 maderios 2024-02-01 19:12:39 UTC
Because the problem comes from IPTC, not using IPTC would be a good thing but i did not find a way to disable it. Is it possible?
Comment 23 maderios 2024-02-02 13:36:31 UTC
I see the new BQM tool doesn't remove all IPTC metadata
To reproduce:
select tiff image 
In BQM choose remove image IPTC metadata "completely"
remove IPTC 
In metadata editor
item -> edit metadata -> IPTC -> keyword 
One keyword tag is still there. I tested with other images, same issue, each time, one keyword is not deleted (only one each time)
Comment 24 Maik Qualmann 2024-02-02 19:33:57 UTC
I'm guessing you're using sidecars. There is an interaction between XMP and EXIF/IPTC. Check out the XMP conversion table:

https://exiv2.org/conversion.html

To prevent this, you would have to deactivate the sidecar.

Maik
Comment 25 maderios 2024-02-03 14:17:20 UTC
Yes, I use sidecars, I never use IPTC and EXIF. Cameras creates EXIF but I don't need this kind of metadata. 
May be I'm wrong but I think converting IPTC to XMP is useless because Digikam never ends recreating interaction issue between  XMP and EXIF/IPTC. 
I tested it with 11 images this way:
In images test directory, I deleted .xmp files then, I deleted all images metadata in this directory :
find . -type f -iname '*.tif' | xargs exiv2 -v -k delete
After launching Digikam, I see metadata have disappeared, good.
Then I tagged my 11 images normally, with XMP settings, only one tag 'animaux'
In metadata editor, all is right in XMP keyword section but  i see tag/keyword 'animaux' is present in IPTC section.