Bug 466658 - EXIF metadata is used when available for JPEG thumbnails, resulting to low resolution thumbnails
Summary: EXIF metadata is used when available for JPEG thumbnails, resulting to low re...
Status: RESOLVED FIXED
Alias: None
Product: kio-extras
Classification: Frameworks and Libraries
Component: Thumbnails and previews (show other bugs)
Version: 22.08.2
Platform: Kubuntu Linux
: NOR normal
Target Milestone: ---
Assignee: Plasma Bugs List
URL:
Keywords: regression
Depends on:
Blocks:
 
Reported: 2023-03-01 14:35 UTC by Stéphane Tréboux
Modified: 2023-12-12 02:50 UTC (History)
3 users (show)

See Also:
Latest Commit:
Version Fixed In: 23.08.2


Attachments
Provided by the user drinktheocean on https://forum.endeavouros.com/t/any-way-to-make-dolphin-preview-ignore-jpeg-exif-thumbnails/37091 (662.90 KB, image/jpeg)
2023-03-01 14:35 UTC, Stéphane Tréboux
Details
Expected thumbnail quality (left) vs EXIF thumbnail quality (right) (215.06 KB, image/png)
2023-03-01 14:54 UTC, Stéphane Tréboux
Details
Correct thumbnail of 7n36d9.jpg with Kubuntu 20.04, Dolphin 19.12.03 (113.55 KB, image/png)
2023-03-01 15:00 UTC, Stéphane Tréboux
Details
Correct thumbnail of 7n36d9.jpg with Kubuntu 22.04, Dolphin 21.12.03 (116.69 KB, image/png)
2023-03-01 15:02 UTC, Stéphane Tréboux
Details
Incorrect thumbnail of 7n36d9.jpg with Kubuntu 22.10, Dolphin 22.08.2 (125.59 KB, image/png)
2023-03-01 15:03 UTC, Stéphane Tréboux
Details
Thumbnails from EXIF of Nikon D800e showing black bars on long edge and wrong orientation. (118.93 KB, image/jpeg)
2023-07-07 09:08 UTC, Paul
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Stéphane Tréboux 2023-03-01 14:35:58 UTC
Created attachment 156871 [details]
Provided by the user drinktheocean on https://forum.endeavouros.com/t/any-way-to-make-dolphin-preview-ignore-jpeg-exif-thumbnails/37091

SUMMARY

Since at least Kubuntu 22.10 and Dolphin 22.08.2 EXIF metadata is used when available for JPEG thumbnails and the thumbnail creation from the JPEG data is skipped, resulting into low quality thumbnails.

STEPS TO REPRODUCE
1. Save the image 7n36d9.jpg (attached to this bug report) to any folder.
2. Show the folder in Dolphin 22.08.2 or newer.

OBSERVED RESULT
The image preview shows an orange (the thumbnail embedded inside the EXIF metadata).
This serves only as an example, in real life cases the thumbnail embedded inside the EXIF metadata will be correct but low resolution and not adequate for a preview in a file explorer on a personal computer.

EXPECTED RESULT
The image preview should show an apple (the actual content of the image).
The resolution should be sufficient to use on a high resolution display commonly used with modern personal computers.
This is mostly not the case with the thumbnail embedded inside the EXIF metadata.

SOFTWARE/OS VERSIONS
Linux: 5.19.0-31-generic
KDE Plasma Version: 5.25.5
KDE Frameworks Version: 5.98.0
Qt Version: 5.15.6

ADDITIONAL INFORMATION
This was recently reported in different forums:
https://forum.manjaro.org/t/bad-quality-thumbnails-in-dolphin/126378
https://forum.endeavouros.com/t/any-way-to-make-dolphin-preview-ignore-jpeg-exif-thumbnails/37091
Comment 1 Stéphane Tréboux 2023-03-01 14:54:32 UTC
Created attachment 156872 [details]
Expected thumbnail quality (left) vs EXIF thumbnail quality (right)

Comparison.png was created by stripping a camera picture from its EXIF data to demonstrate what would be the expected thumbnail quality (left) vs EXIF thumbnail quality (right).
This was achieved by stripping a camera picture from its EXIF data, which of course is not a desirable/scalable solution.
Comment 2 Stéphane Tréboux 2023-03-01 15:00:04 UTC
Created attachment 156873 [details]
Correct thumbnail of 7n36d9.jpg with Kubuntu 20.04, Dolphin 19.12.03

Kubuntu 20.04, Dolphin 19.12.03 generate a correct thumbnail of the example file 7n36d9.jpg.
Comment 3 Stéphane Tréboux 2023-03-01 15:02:29 UTC
Created attachment 156874 [details]
Correct thumbnail of 7n36d9.jpg with Kubuntu 22.04, Dolphin 21.12.03

Kubuntu 22.04, Dolphin 21.12.03 generate a correct thumbnail of the example file 7n36d9.jpg.
Comment 4 Stéphane Tréboux 2023-03-01 15:03:51 UTC
Created attachment 156875 [details]
Incorrect thumbnail of 7n36d9.jpg with Kubuntu 22.10, Dolphin 22.08.2

Kubuntu 22.10, Dolphin 22.08.2 generate an incorrect thumbnail of the example file 7n36d9.jpg. It should show an apple and not an orange.
Comment 5 Paul 2023-07-07 09:08:10 UTC
Created attachment 160117 [details]
Thumbnails from EXIF of Nikon D800e  showing black bars on long edge and wrong orientation.

Whilst no doubt gaining speed in thumbnail production, the loss of quality can be quite significant. Additionally, the exif thumbnails from my Nikon D800e, (direct from camera, no post processing), show black bars along the long edge; and on portrait orientation, the orientation is wrong. See attached screen shot.

IMHO with regard to this, quality should take precedence over speed.


@Stéphane Tréboux

A temporay workaround is to use exiftool to strip out only the thumbnail from the exif data:

exiftool -ThumbnailImage= -ifd1:all= filename.jpg
Comment 6 Stéphane Tréboux 2023-07-07 10:14:58 UTC
> IMHO with regard to this, quality should take precedence over speed.
Ideally this should be configurable in the thumbnailer and the decision should be up to the user.

> A temporary workaround is to use exiftool to strip out only the thumbnail from the exif data
Well this is not really temporary, this is a permanent change to the pictures.
Even assuming someone is willing to do this permanent change, it can easily cause issues when manipulating very large amounts of pictures like private photo collections of several gigabytes.
But thanks for the hint :-)
Comment 7 Mirco Miranda 2023-07-31 10:09:33 UTC
Since framework 5.99 the raw plugin has been included in the kimageformats-plugin. The plugin always renders thumbnails from the image and should fix the problem.
Comment 8 Mirco Miranda 2023-07-31 10:13:31 UTC
(In reply to Mirco Miranda from comment #7)
> Since framework 5.99 the raw plugin has been included in the
> kimageformats-plugin. The plugin always renders thumbnails from the image
> and should fix the problem.

This only applies to Nikon D800 previews.
Comment 9 Méven Car 2023-09-08 17:29:32 UTC
MR in progress https://invent.kde.org/network/kio-extras/-/merge_requests/279/diffs
Comment 10 Méven 2023-09-09 12:04:49 UTC
Git commit 7218e3648c78dbde2b6381e0e6c72d0a30a25f64 by Méven Car.
Committed on 09/09/2023 at 10:59.
Pushed by meven into branch 'master'.

thumbnail Jpegcreator: skip exif thumbnail when too small
Related: bug 465336

M  +-    --    thumbnail/autotests/data/boxes_thumb@2.png
A  +-    --    thumbnail/autotests/data/castle_thumb_256.png
M  +5    -0    thumbnail/autotests/thumbnailtest.cpp
M  +6    -0    thumbnail/jpegcreator.cpp

https://invent.kde.org/network/kio-extras/-/commit/7218e3648c78dbde2b6381e0e6c72d0a30a25f64
Comment 11 Stéphane Tréboux 2023-09-12 14:28:40 UTC
Hello Méven,

Thanks for the fix.

Looking at https://invent.kde.org/network/kio-extras/-/commit/7218e3648c78dbde2b6381e0e6c72d0a30a25f64 the new behavior is:
- if both width and height of the exif thumbnail are smaller than the requested width and height: generate a thumbnail from the full size original picture
- if either width or height of the exif thumbnail are larger than or same as the requested width or height: use the exif thumbnail and skip thumbnail generation

Looking at the exif thumbnails generated by my Canon camera, the size is 120x80 pixels. 

The zoom slider in Dolphin lets me chose the thumbnail size from the following values: 16 pixels, 22 pixels, 32 pixels, 48 pixels, 64 pixels, 80 pixels, 96 pixels, 112 pixels, 128 pixels, 144 pixels, 160 pixels, 176 pixels, 192 pixels, 208 pixels, 224 pixels, 240 pixels and 256 pixels.

The folder  ~/.cache/thumbnails/ contains four folders:
- normal: 128x96, sometimes 120x80 pixels
- large: 256x192 pixels, sometimes 160x120 pixels
- x-large and xx-large are empty, I assume the thumbnails would be even larger

Hence in my case to avoid using the exif thumbnails generated by my Canon camera, I would have to set the zoom slider in Dolphin to at least 128 pixels (just my guess). I like big icons and I use 256 pixels anyway so not an issue for me :-)

Thanks again for the fix!
Comment 12 Méven Car 2023-09-12 17:17:55 UTC
(In reply to Stéphane Tréboux from comment #11)
> Hello Méven,
> 
> Thanks for the fix.
> 
> Looking at
> https://invent.kde.org/network/kio-extras/-/commit/
> 7218e3648c78dbde2b6381e0e6c72d0a30a25f64 the new behavior is:
> - if both width and height of the exif thumbnail are smaller than the
> requested width and height: generate a thumbnail from the full size original
> picture
> - if either width or height of the exif thumbnail are larger than or same as
> the requested width or height: use the exif thumbnail and skip thumbnail
> generation
> 
> Looking at the exif thumbnails generated by my Canon camera, the size is
> 120x80 pixels. 
> 
> The zoom slider in Dolphin lets me chose the thumbnail size from the
> following values: 16 pixels, 22 pixels, 32 pixels, 48 pixels, 64 pixels, 80
> pixels, 96 pixels, 112 pixels, 128 pixels, 144 pixels, 160 pixels, 176
> pixels, 192 pixels, 208 pixels, 224 pixels, 240 pixels and 256 pixels.
> 
> The folder  ~/.cache/thumbnails/ contains four folders:
> - normal: 128x96, sometimes 120x80 pixels
> - large: 256x192 pixels, sometimes 160x120 pixels
> - x-large and xx-large are empty, I assume the thumbnails would be even
> larger

That's mostly useful for hidpi cases.

FYI, this is defined in specification https://specifications.freedesktop.org/thumbnail-spec/thumbnail-spec-latest.html

> 
> Hence in my case to avoid using the exif thumbnails generated by my Canon
> camera, I would have to set the zoom slider in Dolphin to at least 128
> pixels (just my guess). I like big icons and I use 256 pixels anyway so not
> an issue for me :-)

Yes and once your thumbnail size is smaller than 128, the exif thumbnails should be good for you need.
> 
> Thanks again for the fix!

You are welcome !

I am think about backporting the KF 5 so the fix can reach users earlier.
Comment 13 Stéphane Tréboux 2023-09-13 13:26:08 UTC
If my understanding is correct the latest release of kio-extras is 23.08.0 which is the version targeted by the upcoming version 23.10 of Ubuntu (mantic).
https://invent.kde.org/network/kio-extras/-/tags (23.08.0 is the latest tag at the time of this comment, created three weeks ago)
https://packages.ubuntu.com/mantic/kio-extras (version is 23.08.0a at the time of this comment, also three weeks ago)

So if additionally to the forward path kio-extras 23.08 could receive this change maybe it could find its way in Ubuntu 22.10 (mantic) and that would save half a year of waiting for the users, which would be nice :-)
Comment 14 Méven Car 2023-09-13 13:49:05 UTC
(In reply to Stéphane Tréboux from comment #13)
> If my understanding is correct the latest release of kio-extras is 23.08.0
> which is the version targeted by the upcoming version 23.10 of Ubuntu
> (mantic).
> https://invent.kde.org/network/kio-extras/-/tags (23.08.0 is the latest tag
> at the time of this comment, created three weeks ago)
> https://packages.ubuntu.com/mantic/kio-extras (version is 23.08.0a at the
> time of this comment, also three weeks ago)
> 
> So if additionally to the forward path kio-extras 23.08 could receive this
> change maybe it could find its way in Ubuntu 22.10 (mantic) and that would
> save half a year of waiting for the users, which would be nice :-)

It depends on K/Ubuntu mostly.

The next KDE Gear 23.08.1 release should have the fix, or the 23.08.2 depending if MR is merged on time https://invent.kde.org/network/kio-extras/-/merge_requests/283

Often kubuntu provides a ppa to get more recent plasma packages: https://kubuntu.org/news/plasma-5-27-2-for-kubuntu-22-10-available-via-ppa/ They haven't updated the package for 23.04 yet. You should have the fix for 23.10 or later with the ppa.
Comment 15 Méven 2023-09-14 05:24:15 UTC
Git commit 8a0c0d9702b718defb89652762a373dfd8c94979 by Méven Car.
Committed on 12/09/2023 at 19:19.
Pushed by meven into branch 'release/23.08'.

thumbnail Jpegcreator: skip exif thumbnail when too small
Related: bug 465336
(cherry picked from commit 7218e3648c78dbde2b6381e0e6c72d0a30a25f64)

M  +-    --    thumbnail/autotests/data/boxes_thumb@2.png
A  +-    --    thumbnail/autotests/data/castle_thumb_256.png
M  +5    -0    thumbnail/autotests/thumbnailtest.cpp
M  +6    -0    thumbnail/jpegcreator.cpp

https://invent.kde.org/network/kio-extras/-/commit/8a0c0d9702b718defb89652762a373dfd8c94979
Comment 16 Stéphane Tréboux 2023-12-12 02:50:24 UTC
kio-extras 23.08.3 including Git commit 8a0c0d97 is now available as a Ubuntu package targeting the upcoming Ubuntu 24.04 (noble).

If you use Ubuntu 23.10 (mantic) on AMD64 you can download and install at your own risk, please only use if you know what you are doing!
https://packages.ubuntu.com/noble/amd64/kio-extras/download
https://packages.ubuntu.com/noble/all/kio-extras-data/download
https://packages.ubuntu.com/noble/amd64/libkf5kexiv2-15.0.0/download

Here are the direct download links I used due to https://packages.ubuntu.com/ being often down lately:
https://cz.archive.ubuntu.com/ubuntu/pool/universe/k/kio-extras/kio-extras_23.08.3-0ubuntu1_amd64.deb
https://cz.archive.ubuntu.com/ubuntu/pool/universe/k/kio-extras/kio-extras-data_23.08.3-0ubuntu1_all.deb
https://cz.archive.ubuntu.com/ubuntu/pool/universe/libk/libkf5kexiv2/libkf5kexiv2-15.0.0_23.08.3-0ubuntu1_amd64.deb

Once you have downloaded the three packages (kio-extras, kio-extras-data and libkf5kexiv2-15.0.0) in the new version you can use "dpkg -i *.deb" to install them in one batch from a command line.
After installation go to ~/.cache and rename the folder "thumbnails" to "thumbnails_bak". If you are happy with the result you can erase the folder later on.

No restart is needed and applications like the file manager Dolphin will immediately generate new thumbnails. The new thumbnail will not be the embedded exif thumbnail when it is too small to be used. The criteria applied is that the exif thumbnail will be used if at least one of the edges is long enough. In other words, both the width and height of the exif thumbnail need to be strictly smaller than the target size for the desired thumbnail for the exif thumbnail to be skipped. The target size of the thumbnail is configurable in many apps, in Dolphin there is a slider at the bottom right. If you pick a large enough value there is a good chance that the exif thumbnail will never be used.

This will not necessarily resolve issues like the wrong orientation of the thumbnail (https://bugs.kde.org/attachment.cgi?id=160117), especially if the exif thumbnail comes with a sufficient resolution (https://bugsfiles.kde.org/attachment.cgi?id=156871).

If you have to go back to stock Ubuntu 23.10 for any reason, the packages which originally come with Ubuntu 23.10 can be downloaded and installed the same way as the packages from Ubuntu 24.04.
https://packages.ubuntu.com/mantic/amd64/kio-extras/download
https://packages.ubuntu.com/mantic/all/kio-extras-data/download
https://packages.ubuntu.com/mantic/amd64/libkf5kexiv2-15.0.0/download

Here are the direct download links I used due to https://packages.ubuntu.com/ being often down lately:
https://cz.archive.ubuntu.com/ubuntu/pool/universe/k/kio-extras/kio-extras_23.08.1-0ubuntu1_amd64.deb
https://cz.archive.ubuntu.com/ubuntu/pool/universe/k/kio-extras/kio-extras-data_23.08.1-0ubuntu1_all.deb
https://cz.archive.ubuntu.com/ubuntu/pool/universe/libk/libkf5kexiv2/libkf5kexiv2-15.0.0_23.08.1-0ubuntu1_amd64.deb

Again, once you have downloaded the three packages (kio-extras, kio-extras-data and libkf5kexiv2-15.0.0) in the original version you can use "dpkg -i *.deb" to install them in one batch from a command line.
After installation go to ~/.cache, erase the folder "thumbnails" and rename the folder "thumbnails_bak" to "thumbnails".