Bug 467675

I suspect that on macOS the KImageFormats plugins are not compiled with webp support.
Please have a look at the digiKam Settings->Plugins->Image loader modules.
Is the webp format listed in the QImage Loader?

Maik

Hi Maik,  
I see WBMP and WEBP listed as Type-Mimes under ImageMagick loader and QImage loader.  

Geoff

Maik,

WEBP is well supported in the QImage loader under MacOS.

https://i.imgur.com/jtCzPxp.png

KF5::KImageFormat version 5.102 is used to compile the MacOS PKG.

Gilles

It works here on Linux with no problems. Then we need the debug output from the terminal, as described here for macOS:

https://www.digikam.org/contribute/

Maik

Created attachment 157579 [details]
terminal log of error

Debug output as requested.

Git commit 410fd4a0e095eac26d3fb8fb398cb67e6d076a42 by Maik Qualmann.
Committed on 26/03/2023 at 07:51.
Pushed by mqualmann into branch 'master'.

various fixes for newer image formats
- use QImage for Webp
- fix quality settings

M  +2    -16   core/dplugins/dimg/imagemagick/dimgimagemagickplugin.cpp
M  +4    -5    core/dplugins/dimg/qimage/dimgqimageloader_save.cpp
M  +42   -0    core/utilities/imageeditor/core/editorcore_p.h

https://invent.kde.org/graphics/digikam/commit/410fd4a0e095eac26d3fb8fb398cb67e6d076a42

I will restart the 8.0.0 MacOS PKG installer this morning to see if the last commit fix the problem.

Gilles

PKG installer is now updated :

https://files.kde.org/digikam/

Please check if problem is fixed.

Gilles Caulier

Hi,  I tried this new version.  
It is improved, but there are still issues.  
I don't have time to test more right now, but will get back to it.  
Results in short:
-I think this is related to the gmic filter at times
-can now export to webp (if make no gmic edits to image, other non-gmic edits cause no issue).  
  -see screen capture
   _test1 -export as webp lossless 3, no edits to original file
   _test2 export as webp quality 65, no edits to original file
   _test 4 gmic.jp2 - okay, edited in gmic with black and white filter before exporting
   _test 5 gmic.webp - small file created, and shows transparent thumbnail
-exporting to wbmp now creates a file, but that cannot be opened.  The thumbnail shown is a generic WBMP icon.
    _test3 export as wbmp -   corrupt, but 1.3 MB, no edits to original file. 
-If I edit image using gmic plugin before exporting the webp file is exported, but entirely transparent and small file size.  

See screenshot and text file.  

I can test some more if needed.  
Thanks.

Created attachment 157593 [details]
screenshot showing issues

Created attachment 157594 [details]
debug log for _test1 to _test 5

Can you share the test files to try to reproduce ?

The webmp format is also broken here on Linux. There should be a bug report for the KImageFormats plugins.

Maik

Created attachment 157614 [details]
_test0 image

I am sharing the files from my tests and some new observations below. 
This is on digiKam-8.0.0-20230326T082403-MacOS-x86-64-debug.pkg with MacBook Pro Intel.
-new issue noticed - metadata not being carried over to exported files, including comments and gps on Mac, I also tried on Ubuntu and the metadata does carry over.  
-new issue noticed - linux appimage doesn't include g'mic.  
-sorry forgot to capture the debug log

_test0.jpg - original source use for all tests, opened in Image editor, and exported to the formats below.
_test1.webp - no edits to original, just export as webp lossless, seems okay except no metadata or coordinates
_test2.webp - no edits to original, just export as webp quality 65, no edits to original file, seems okay except no metadata or coordinates
_test3.wbmp - no edits to original, just export as wbmp -  corrupt, but 1.3 MB
_test4.jp2 - edited in gmic with black and white filter before exporting, jpeg2000, no metadata carried over, but otherwise looks okay, 
_test5.webp - edited in gmic with black and white filter before exporting, small file created, image is missing, and shows transparent checkerboard pattern thumbnail
_test6.jpg, edited in gmic with black and white filter before exporting, missing gps metadata, other metadata is present

Created attachment 157615 [details]
_test2 image

Created attachment 157616 [details]
_test3 image

Created attachment 157617 [details]
_test5 image

> -new issue noticed - linux appimage doesn't include g'mic. 

==> already fixed with last bundle pushed at usual place...

Gilles

Here all metadata is available in webp in my native Linux version, both with Exiv2 writing engine or with ExifTool.

Maik

Maik, Geof, 

I discovered that libheif and libaom are now handled by Macports. These libs was overwritten by my scripts with older versions in current PKG. I will fix it to rebuild whole PKG from scratch.

I will also upgrade KF5 to last 5.104 for the KImageFormat component supporting WEBP Qt plugin.

For the G'Mic-qt bug, i will try to reproduce the problem.

Gilles

Geoff,

It miss the test04.jp2 file...

Gilles

It miss also test06.jpg

I tried to save in WEBP under MacoOS, and editing or not in gmic do not change anything: editor save to a new WEBP file without error. Metadata are visible by ExifTool, and Exiv2 (Exif and XMP, as IPTC is not supported by WEBP)

https://digikamdevelopper.imgur.com/all

Gilles Caulier

oups, wrong link : https://i.imgur.com/4APTsab.png

Gilles

New test, with test01.jpg open in editor, edited with gmic/B&W, and exported to webp, jp2, and wbmp.

All is fine expected for wbmp which is broken due to a bug in KImageFormats codecs.

https://i.imgur.com/Jsd8Gp9.png

Exif, makernotes, and XMP are here for webp, jp2. The original JPEG file do not have any GPS data, so...

Gilles

When export JPEG to WBMP from image editor, i can see that from the Terminal :

digikam.general: Trying to find a saving format from targetUrl =  QUrl("file:///Users/gilles/Pictures/BUG 467675/_test0.wbmp")
digikam.general: Qt Offered types:  "*.avif *.bmp *.bw *.cur *.eps *.epsf *.epsi *.icns *.ico *.jxl *.pbm *.pcx *.pgm *.pic *.png *.ppm *.rgb *.rgba *.sgi *.tga *.wbmp *.webp *.xbm *.xpm *.tiff *.tif *.jpg *.jpeg *.jpe *.jp2 *.j2k *.jpx *.pgx *.pgf *.heic *.heif *.fts *.fit *.fits "
digikam.general: Writable formats:  ("avif", "bmp", "bw", "cur", "eps", "epsf", "epsi", "icns", "ico", "jxl", "pbm", "pcx", "pgm", "pic", "png", "ppm", "rgb", "rgba", "sgi", "tga", "wbmp", "webp", "xbm", "xpm", "tiff", "tif", "jpg", "jpeg", "jpe", "jp2", "j2k", "jpx", "pgx", "pgf", "heic", "heif", "fts", "fit", "fits")
digikam.general: Possible format from local file:  "wbmp"
digikam.general: Using format from target url  "wbmp"
digikam.geoiface: ----
digikam.general: Saving to : "/Users/gilles/Pictures/BUG 467675/EditorWindow-NHHEVy-a0b08721.digikamtempfile.wbmp" ( "wbmp" )
digikam.general: Saving file "/Users/gilles/Pictures/BUG 467675/EditorWindow-NHHEVy-a0b08721.digikamtempfile.wbmp" at -1
digikam.dimg: Prepare Metadata to save for "/Users/gilles/Pictures/BUG 467675/_test0.wbmp"
digikam.metaengine: JPEG image preview size: ( 1280 x 960 ) pixels - 263188 bytes
digikam.dimg: Saving to  "/Users/gilles/Pictures/BUG 467675/EditorWindow-NHHEVy-a0b08721.digikamtempfile.wbmp"  with format:  "wbmp"
digikam.metaengine: Cannot load metadata from file with Exiv2 backend: /Users/gilles/Pictures/BUG 467675/EditorWindow-NHHEVy-a0b08721.digikamtempfile.wbmp  (Error # 11 :  "/Users/gilles/Pictures/BUG 467675/EditorWindow-NHHEVy-a0b08721.digikamtempfile.wbmp: The file contains data of an unknown image type"
digikam.metaengine: Check ExifTool availability: true
digikam.metaengine: ExifTool "Load Chunks" "-TagsFromFile /Users/gilles/Pictures/BUG 467675/EditorWindow-NHHEVy-a0b08721.digikamtempfile.wbmp -all:all -o -.exv"
digikam.metaengine: ExifToolProcess::readOutput(): ExifTool command completed
digikam.metaengine: ExifTool complete command for action "Load Chunks" with elasped time (ms): 5
digikam.metaengine: EXV chunk size: 0
digikam.metaengine: ExifTool parsed command for action "Load Chunks" 1 properties decoded
digikam.metaengine: ExifTool complete "Load Chunks" for "/Users/gilles/Pictures/BUG 467675/EditorWindow-NHHEVy-a0b08721.digikamtempfile.wbmp"
digikam.metaengine: Metadata chunk loaded with ExifTool
digikam.metaengine: Metadata chunk loaded with ExifTool has no data
digikam.metaengine: Loading metadata with "No Backend" backend from "/Users/gilles/Pictures/BUG 467675/EditorWindow-NHHEVy-a0b08721.digikamtempfile.wbmp"
digikam.metaengine: MetaEngine::metadataWritingMode 0
digikam.metaengine: Will write Metadata to file "/Users/gilles/Pictures/BUG 467675/EditorWindow-NHHEVy-a0b08721.digikamtempfile.wbmp"
digikam.metaengine: Cannot save metadata to image with Exiv2 backend:  (Error # 11 :  "/Users/gilles/Pictures/BUG 467675/EditorWindow-NHHEVy-a0b08721.digikamtempfile.wbmp: The file contains data of an unknown image type"
digikam.general: "/Users/gilles/Pictures/BUG 467675/EditorWindow-NHHEVy-a0b08721.digikamtempfile.wbmp" true true
digikam.databaseserver: Running 30 seconds...
digikam.general: Saved QUrl("file:///Users/gilles/Pictures/BUG 467675/_test0.jpg") to QUrl("file:///Users/gilles/Pictures/BUG 467675/_test0.wbmp")
digikam.general: was versioned false current 750 "_test0.jpg" destinations ()
digikam.metaengine: Cannot load metadata from file with Exiv2 backend: /Users/gilles/Pictures/BUG 467675/_test0.wbmp  (Error # 11 :  "/Users/gilles/Pictures/BUG 467675/_test0.wbmp: The file contains data of an unknown image type"
digikam.metaengine: Check ExifTool availability: true
digikam.metaengine: ExifTool "Load Chunks" "-TagsFromFile /Users/gilles/Pictures/BUG 467675/_test0.wbmp -all:all -o -.exv"
digikam.metaengine: ExifToolProcess::readOutput(): ExifTool command completed
digikam.metaengine: ExifTool complete command for action "Load Chunks" with elasped time (ms): 5
digikam.metaengine: EXV chunk size: 0
digikam.metaengine: ExifTool parsed command for action "Load Chunks" 1 properties decoded
digikam.metaengine: ExifTool complete "Load Chunks" for "/Users/gilles/Pictures/BUG 467675/_test0.wbmp"
digikam.metaengine: Metadata chunk loaded with ExifTool
digikam.metaengine: Metadata chunk loaded with ExifTool has no data
digikam.metaengine: Loading metadata with "No Backend" backend from "/Users/gilles/Pictures/BUG 467675/_test0.wbmp"
digikam.dimg: "/Users/gilles/Pictures/BUG 467675/_test0.wbmp" : "QIMAGE" file identified
digikam.dimg.qimage: Can not load " "/Users/gilles/Pictures/BUG 467675/_test0.wbmp" " using DImg::DImgQImageLoader!
digikam.dimg.qimage: Error message from loader: "Unsupported image format"
digikam.dimg: "/Users/gilles/Pictures/BUG 467675/_test0.wbmp" : Unknown image format !!!
digikam.database: Scanning took 8 ms
digikam.database: Finishing took 1 ms
digikam.database: Copying properties from 750 to 759
digikam.general: Event is dispatched to OSX desktop notifier
digikam.metaengine: ExifTool "Load Metadata" "-json -G:0:1:2:4:6 -l /Users/gilles/Pictures/BUG 467675/_test0.wbmp"
digikam.general: "Standard Exif Tags" decoding took 1 ms ( false )
digikam.metaengine: ExifToolProcess::readOutput(): ExifTool command completed
digikam.general: "MakerNote Exif Tags" decoding took 1 ms ( false )
digikam.general: "IPTC Records" decoding took 774 ms ( false )
digikam.general: "XMP Schema" decoding took 4 ms ( false )
digikam.metaengine: ExifTool complete command for action "Load Metadata" with elasped time (ms): 4
digikam.metaengine: ExifTool Json map size: 10
digikam.metaengine: ExifTool parsed command for action "Load Metadata" 9 properties decoded
digikam.metaengine: ExifTool complete "Load Metadata" for "/Users/gilles/Pictures/BUG 467675/_test0.wbmp"
digikam.metaengine: Cannot load metadata from file with Exiv2 backend: /Users/gilles/Pictures/BUG 467675/_test0.wbmp  (Error # 11 :  "/Users/gilles/Pictures/BUG 467675/_test0.wbmp: The file contains data of an unknown image type"
digikam.metaengine: Check ExifTool availability: true
digikam.metaengine: ExifTool "Load Chunks" "-TagsFromFile /Users/gilles/Pictures/BUG 467675/_test0.wbmp -all:all -o -.exv"
digikam.metaengine: ExifToolProcess::readOutput(): ExifTool command completed
digikam.metaengine: ExifTool complete command for action "Load Chunks" with elasped time (ms): 6
digikam.metaengine: EXV chunk size: 0
digikam.metaengine: ExifTool parsed command for action "Load Chunks" 1 properties decoded
digikam.metaengine: ExifTool complete "Load Chunks" for "/Users/gilles/Pictures/BUG 467675/_test0.wbmp"
digikam.metaengine: Metadata chunk loaded with ExifTool
digikam.metaengine: Metadata chunk loaded with ExifTool has no data
digikam.metaengine: Loading metadata with "No Backend" backend from "/Users/gilles/Pictures/BUG 467675/_test0.wbmp"
digikam.general: Trying to get thumbnail from "/Users/gilles/Pictures/BUG 467675/_test0.wbmp" ( "image" )
digikam.general: Trying to get thumbnail with Exiv2 for "/Users/gilles/Pictures/BUG 467675/_test0.wbmp"
digikam.general: Trying to get thumbnail with DImg preview for "/Users/gilles/Pictures/BUG 467675/_test0.wbmp"
digikam.general: Trying to get thumbnail from Embedded preview with libraw for "/Users/gilles/Pictures/BUG 467675/_test0.wbmp"
digikam.rawengine: Failed to load embedded RAW preview
digikam.general: Trying to get thumbnail from half preview with libraw for "/Users/gilles/Pictures/BUG 467675/_test0.wbmp"
digikam.general: Trying to get thumbnail from Embedded preview with Exiv2 for "/Users/gilles/Pictures/BUG 467675/_test0.wbmp"
digikam.metaengine: Cannot load metadata with Exiv2:  (Error # 11 :  "/Users/gilles/Pictures/BUG 467675/_test0.wbmp: The file contains data of an unknown image type"
digikam.dimg: "/Users/gilles/Pictures/BUG 467675/_test0.wbmp" : "QIMAGE" file identified
digikam.dimg.qimage: Can not load " "/Users/gilles/Pictures/BUG 467675/_test0.wbmp" " using DImg::DImgQImageLoader!
digikam.dimg.qimage: Error message from loader: "Unsupported image format"
digikam.dimg: "/Users/gilles/Pictures/BUG 467675/_test0.wbmp" : Unknown image format !!!
digikam.general: mimetype =  ""  ext =  "WBMP"
digikam.general: Cannot create thumbnail for "/Users/gilles/Pictures/BUG 467675/_test0.wbmp"
digikam.general: Thumbnail is null for  "/Users/gilles/Pictures/BUG 467675/_test0.wbmp"
digikam.metaengine: ExifTool "Load Metadata" "-json -G:0:1:2:4:6 -l /Users/gilles/Pictures/BUG 467675/_test0.wbmp"
digikam.metaengine: Cannot load metadata from file with Exiv2 backend: /Users/gilles/Pictures/BUG 467675/_test0.wbmp  (Error # 11 :  "/Users/gilles/Pictures/BUG 467675/_test0.wbmp: The file contains data of an unknown image type"
digikam.metaengine: Check ExifTool availability: true
digikam.metaengine: ExifTool "Load Chunks" "-TagsFromFile /Users/gilles/Pictures/BUG 467675/_test0.wbmp -all:all -o -.exv"
digikam.metaengine: ExifToolProcess::readOutput(): ExifTool command completed
digikam.metaengine: ExifToolProcess::readOutput(): ExifTool command completed
digikam.metaengine: ExifTool complete command for action "Load Chunks" with elasped time (ms): 5
digikam.metaengine: EXV chunk size: 0
digikam.metaengine: ExifTool parsed command for action "Load Chunks" 1 properties decoded
digikam.metaengine: ExifTool complete "Load Chunks" for "/Users/gilles/Pictures/BUG 467675/_test0.wbmp"
digikam.metaengine: Metadata chunk loaded with ExifTool
digikam.metaengine: Metadata chunk loaded with ExifTool has no data
digikam.metaengine: Loading metadata with "No Backend" backend from "/Users/gilles/Pictures/BUG 467675/_test0.wbmp"
digikam.general: Metadata loading with Exiv2 took 10 ms ( false )
digikam.metaengine: ExifTool complete command for action "Load Metadata" with elasped time (ms): 5
digikam.metaengine: ExifTool Json map size: 10
digikam.metaengine: ExifTool parsed command for action "Load Metadata" 9 properties decoded
digikam.metaengine: ExifTool complete "Load Metadata" for "/Users/gilles/Pictures/BUG 467675/_test0.wbmp"

... nothing special. KImageFormats provide well the WBMP supports and it's called, but image is a mess

Gilles

Geoff,

The WBMP problem must be reported to frameworks-kimageformats product as UPSTREAM KDE bug.

Gilles

Created attachment 157710 [details]
add missing jp2 test file from prior message

Cannot attached test1.png and test4.jpg since they are too big.

WBMP corrupt export file bug filed here:
https://bugs.kde.org/show_bug.cgi?id=467950

(In reply to caulier.gilles from comment #24)
> New test, with test01.jpg open in editor, edited with gmic/B&W, and exported
> to webp, jp2, and wbmp.
> 
> All is fine expected for wbmp which is broken due to a bug in KImageFormats
> codecs.
> 
> https://i.imgur.com/Jsd8Gp9.png
> 
> Exif, makernotes, and XMP are here for webp, jp2. The original JPEG file do
> not have any GPS data, so...
> 
> Gilles

For me there is definitely something wrong with using gmic and exporting to WEBP on MacOS.  
I have tried several times (GMIC black & white filter and others) and it gives a transparent file exporting to WEBP. 

I also tried this on Windows and it gave similar results (one quick test).  

I looked at Ubuntu also, and it doesn't have the gmic plugin in the appimage.  

If I use the same source file, process using the digikam built in black and white module, then export, it works fine. 

When exporting to WEBP with or without editing, I lose the EXIF data.
I tried several files with same results so don't think is my files.  
most webp export tests at quality 73
 
Thanks, Geoff

Git commit 1b84d0aa56f91b7df6a7fd76e42e5afef3946ded by Gilles Caulier.
Committed on 30/03/2023 at 07:23.
Pushed by cgilles into branch 'master'.

some format to save are optional, make enum depending of this compilation configurations

M  +0    -1    core/libs/widgets/files/filesaveoptionsbox.cpp
M  +5    -0    core/libs/widgets/files/filesaveoptionsbox.h

https://invent.kde.org/graphics/digikam/commit/1b84d0aa56f91b7df6a7fd76e42e5afef3946ded

As WBMP problem is releavant of KImageFormats codecs collection and is reported as UPSTREAM bug #467950, i rename title only for WEBP

I found a little problem with the export settings widget depending of the format, but it doesn't have any effect on the output file contents

Gilles

Geoff,

Here you can find the AppImage demo loading a JPEG, editing with GMIC-Qt/B&W and exporting to WEBP. As you can see all work as expected and metadata are there...

https://drive.google.com/file/d/18VxKzFfXsC02-euPluy1XmemknNfbmr_/view?usp=sharing

I reproduced the same under MacOS without problem...

Best

Gilles

The WBMP format probably comes from Qt directly. It's not relevant to us at all. It comes from old cell phones and is a 1-Bit black and white format. It has nothing to do with modern Webp.

Maik

yes WBMP is an old phone image format :

https://en.wikipedia.org/wiki/Wireless_Application_Protocol_Bitmap_Format

yes, i confirm that WBMP format is support as a low level Qt plugin. It's not provided KImageFormats module.

https://github.com/qt/qtimageformats/tree/dev/src/plugins/imageformats

Geoff, this want mean that the bug report open in KDE Bugzilla must be moved to Qt Bugzilla:

https://bugreports.qt.io/browse/QTBUG-112275?jql=project%20%3D%20QTBUG%20AND%20component%20%3D%20%22Image%20formats%22

Gilles

(In reply to Maik Qualmann from comment #33)
> The WBMP format probably comes from Qt directly. It's not relevant to us at
> all. It comes from old cell phones and is a 1-Bit black and white format. It
> has nothing to do with modern Webp.
> 
> Maik

Okay.  I misunderstood this format.

(In reply to caulier.gilles from comment #32)
> Geoff,
> 
> Here you can find the AppImage demo loading a JPEG, editing with GMIC-Qt/B&W
> and exporting to WEBP. As you can see all work as expected and metadata are
> there...
> 
> https://drive.google.com/file/d/18VxKzFfXsC02-euPluy1XmemknNfbmr_/
> view?usp=sharing
> 
> I reproduced the same under MacOS without problem...
> 
Hi Gilles, 

Here is a screencast of MacOS and Digikam 8.0.0 (3/30/2023) editing in GMIC filter and exporting to WEBP.  The resulting thumbnail shows an image, but nothing in preview or editor.  This affects WEBP and I have had no issues with other formats like PNG or JPEG or other built-in digikam effects.  This also looks to affect Windows, but not Linux.  I just picked a random photo, it happens to all (several) that I've tried.  

https://drive.google.com/file/d/1XuLik9J9X8WRuk8Xa761NdWVhAVmRogv/view?usp=share_link

I also did another test where I used exiftool to remove all metadata from the WEBP version and then the thumbnail turned transparent.  

Regards, Geoff

Hi Maik,

I can reproduce now the dysfunction with WEBP under Linux.

- Open a JPEG files in editor
- Run GMIC Qt B&W filter.
- Export to WebP

The resulting image generated by GMIC Qt is fine in editor. GMIC convert from RGBA to grayscale image. When we re-importing the image data, we re-convert to DImg as RGBA. Code is located in this function:

https://github.com/cgilles/gmic-qt/blob/master/src/Host/digiKam/editor/host_digikam.cpp#L75

Nothing special here. If i export to another format image is not broken. I tried TIFF, PGF, JPEG, PNG, etc...

But in fact it's not only relevant of any filter applied to GMIC: all image process and exported in WEBP are broken. For ex, A JPEG file post-processed in GMIC with Brightness-Contrast filter and exported as WEBP is also broken.

So something is wrong in WEBP plugin. I tried to disable the saveMetadata() in DIMg::Qimage plugin to see if it have any effect. It's not better, expected that alpha channel become the main color and no contents is displayed.

I analyzed the WEBP RGB contents generated and it sound like only the first column of the image is used to generate all the rest. Open the WEBP image export to gimp and disable the alpha channel:

https://i.imgur.com/2rL83S6.png

It's a really strange effect.

Note: an image post processed in GMIC and export in PNG and reopen in editor to be re-exported a WEBP is fine. So for me, something is badly pass to the DImg container when post processed in GMIC that WEBP codec miss-understood. 

For the moment i have no other ideas in mind.
Best

Gilles

I've now loaded and compiled the GMIC-Qt plugin, I hadn't installed it before.
I can call it up from the digiKam Editor and select modules, the preview works without any problems. But if I also click Apply or OK, the progress bar keeps scrolling back and forth, the seconds count up, but nothing else happens.

Maik

yes forget the apply button, i already seen this dysfunction. It's a race condition. Only use Ok button for the moment
Gilles

Gilles, you set the alpha channel to 0x00, but this needs to be set to 0xFF. And yes, just OK works.

Maik

Damned, you are right. Alpha channel was set to the opposite value. Now problem is not reproducible under Linux. 

But why the Alpha channel break the WebP export, it's mysterious...

Gilles

Webp support alpha channel, Jpeg not for example. DImg has an alpha flag, so it won't show up in the program when rendering the view, but the pixel data must be correct when converted to QImage.

Maik

I just tested under MacOS with last build and problem is now fixed.

Gilles