Bug 360821

Gwenview version 15.12.3 using KDE Fw 5.19.0

If possible, add a link with description of the new file format. I only found references to the old file format.

Here is a thread of the thumbnailing of the Gimp 2.9 XCF files:  https://mail.gnome.org/archives/gimp-developer-list/2016-January/msg00007.html

I can reproduce bug with:
Fedora 24 system
digikam 5.1 fedora package compiled with kde 5.24.0
installed kde version: 5.25
I use gimp-2.9.4 dev version. I can't open in Digikam 5.1 .xcf files or get  .xcf thumbnails.
DK 5.1 works normally with .xcf created with gimp 2.8.x. 
Digikam doesn't work with .xcf created with gimp 2.9.4 
It seems to be a compatibility problem with xcf version 8 created with gimp 2.9.4

Marcel, see comment #2. Without information about the new file format, there is nothing we can do.

I think it's useful
- Add LZMA2 compressed file support (.xcf.xz/.xcfxz)
- Internal tile compression for XCF is now zlib (instead of RLE) 
http://wiki.gimp.org/wiki/Release:2.10_changelog

More information about xcf here:
https://github.com/GNOME/gimp/blob/master/devel-docs/xcf.txt
Mainly:
PROP_COMPRESSION (essential)
  uint32  17       Type identification
  uint32  1        One byte of payload
  byte    comp     Compression indicator; one of
                     0: No compression
                     1: RLE encoding
                     2: (Never used, but reserved for zlib compression)
3: (Never used, but reserved for some fractal compression)

The document referenced in comment #7 describes the old (2.8) format, but does not have any information about the 2.9 additions/changes.

*** Bug 379550 has been marked as a duplicate of this bug. ***

(In reply to Christoph Feck from comment #2)
> If possible, add a link with description of the new file format. I only
> found references to the old file format.

Might help?
https://bugzilla.gnome.org/show_bug.cgi?id=782244

kimageformats 5.45.0-1 (kf5) is installed on my (Arch linux) system but .xcf thumbnails created with gimp-2.10 or 2.9 are not displayed. Thumbs created with all gimp 2.8 versions are displayed. Problem is present from gimp-2.9, when .xcf format changed. Dolphin-18.0.4 can display new .xcf format created with gimp-2.10 
https://wiki.gimp.org/wiki/Release:2.10_changelog
Substantial rewrite of layer modes and how they are stored in XCF
Add LZMA2 compressed file support (.xcf.xz/.xcfxz)
Internal tile compression for XCF is now zlib (instead of RLE)

Precision, it seems to be a specific kimageformats bug: Dolphin-18.0.4, Nautilus-3.28 and Thunar-1.6.15 can display new .xcf format created with gimp-2.10

I installed today new framework 5.46 + kimageformats-5.46 then I compiled again Digikam-5.9. It doesn't display .xcf thumbnails created with Gimp-2.10.
All others formats (ora, jpeg, png, tiff, etc) are displayed in DK.

Please provide the information requested in comment #2.

(In reply to Christoph Feck from comment #14)
> Please provide the information requested in comment #2.

Hello!
I have started updating the documentation of the XCF format.
See the last commits to see the major changes in the format since GIMP 2.8:
https://git.gnome.org/browse/gimp/log/devel-docs/xcf.txt

I think the only remaining feature that still have to be added (will do tomorrow) is about masks on group layers. And there may be an information about endianness but this is mostly a bug which touched development format of XCF and which should not be a problem if you only target stable XCF versions.

Good news. Thanks, Jehan :)

Thanks for the update; changing status.

Regarding layer blending modes, do you have a link to the code to understand the actual formulas?

(In reply to Christoph Feck from comment #17)
> Thanks for the update; changing status.
> 
> Regarding layer blending modes, do you have a link to the code to understand
> the actual formulas?

In this file:
https://git.gnome.org/browse/gimp/tree/app/operations/layer-modes/gimp-layer-modes.c
You'll see the list of layer modes, and in particular notice the op_name parameter, which is the string you want to look for the implementation.

For instance, the "Normal" layer mode is the GEGL operation "gimp:normal" whose implementation is: https://git.gnome.org/browse/gimp/tree/app/operations/layer-modes/gimpoperationnormal.c

So I spent a good part of my day updating the docs, and I think it's not too bad now. After reviewing myself the changes, I think that the main points you'll want to check are:

- of course the new layer modes algorithms, as you note, but also the new composite mode, composite space and blend mode properties.
- the fact that pointers can now be on 8 bytes (XCF 11 and over).
- zlib compression
- and color precision per component.

Ha! There's https://git.gnome.org/browse/gimp/tree/devel-docs/xcf.txt -- and it's been updated just now. See https://www.gimp.org/news/2018/05/20/gimp-2-10-2-released/

Is there no way to reuse the XCF load/save code from the gimp source for the xcf kimage format plugin? Is it really necessary to implement it from scratch? I think support for the new xcf format in the xcf kimage format now is even more important as GIMP 2.10 is released. For example, digikam now can't show many xcf files saved with GIMP 2.10.

No, that is not possible. Gimp's xcf code loads directly into gimp's internal data structures, and saves directly from them. Re-using that code basically means embedding gimp. So it would be better to update the xcftools codebase, but afaik, nobody is working on that yet.

If there is a major overhaul of the xcf reader due, please also consider bug 126072 to incorporate gzip and bzip2 compression for image collections viewed in gwenview / digikam. Thanx!

That should be completely independent -- and the patch looks like it could have been committed before in any case.

*** Bug 360806 has been marked as a duplicate of this bug. ***

Albert asked me on IRC last night whether I thought that this might be a good gsoc project, but I was asleep (01:54:53)... The answer is, yes, could very well be. Take the xcftools library, modernize it and update it to xcf for gimp 2.10, then update this kimageio plugin and Krita's xcf plugin.

Hi all in this room.

I just recompiled digiKam 6.1.0 pre-release AppImage with KImageFormat 5.46:

https://files.kde.org/digikam/

...and the problem still reproducible with XCF images generated with Gimp 2.10 (zlib compression).

This is the console trace when digiKam try to load XCF through Qt Image Loader delegate to KImageFormat :

Digikam::MetaEngine::Private::printExiv2ExceptionError: Cannot load metadata using Exiv2   (Error # 11 :  /home/gilles/Pictures/Ubaye 2018/09-07-2018/DSC05816.xcf: The file contains data of an unknown image type
Digikam::DImg::load: "/home/gilles/Pictures/Ubaye 2018/09-07-2018/DSC05816.xcf"  : QIMAGE file identified
Digikam::QImageLoader::load: Can not load " "/home/gilles/Pictures/Ubaye 2018/09-07-2018/DSC05816.xcf" " using DImg::QImageLoader!
Digikam::QImageLoader::load: Error message from loader: "Unable to read image data"

The message from the last one come from KImageFormat XCF image loader.

I tested with gimp 2.8 and KImageFormat 5.46 has no problem to generate XCF thumbnails.

Gilles Caulier

Fro info, ImageMagick is able to display Gimp 2.10 XCF files :

[gilles@localhost 09-07-2018]$ display --version
Version: ImageMagick 7.0.8-35 Q16 x86_64 2019-03-26 https://imagemagick.org
Copyright: © 1999-2019 ImageMagick Studio LLC
License: https://imagemagick.org/script/license.php
Features: Cipher DPC HDRI Modules OpenCL OpenMP 
Delegates (built-in): bzlib cairo fftw fontconfig freetype gvc heic jbig jng jp2 jpeg lcms lqr ltdl lzma openexr pangocairo png rsvg tiff webp wmf x xml zlib
[gilles@localhost 09-07-2018]$ 


Gilles Caulier

If I understand:
- For three years, Kimageformats can't diplay xcf files (except old gimp files)
- ImageMagick can display all xcf files (I tested it)
Replacing Kimageformats by ImageMagick could be a good way to solve issue.
https://github.com/ImageMagick/ImageMagick
https://en.wikipedia.org/wiki/ImageMagick
ImageMagick is already required by 91 Archlinux packages:
https://www.archlinux.org/packages/extra/x86_64/imagemagick/

You can't replace kimageformats with imagemagick, that simply makes no sense

To replace whole image format supported by KImageFormat certainly no, but to call ImageMagick C++ API to get the big advantage to render plenty of image formats, yes...

https://www.imagemagick.org/script/formats.php#supported

Gilles Caulier

@Gilles

I love that suggestion, as it will also solve the long-standing SVG issue: bug 336436

Hope I'm not the april fool now!

That used to be the way Krita imported/exported images. It was not a huge success, and we moved away to using libpng etc. directly.

ImageMagick C++ API have been a problem ? Which one exactly ? The IM C++ exception no compatible with Qt ?

Gilles Caulier

The ImageMagick API's are very unstable: there is no guarantee of source compatibility between versions. That's why GraphicsMagick was created back in the days, but that fell behind. It was a huge maintenance burden and mess. It would be much better to just check what ImageMagick has done in their XCF filter and try to port it to kimageformats. It cannot be that hard...

Ah yes, i hear about the IM vs GM story.

IM XCF codec is here :

https://github.com/ImageMagick/ImageMagick/blob/master/coders/xcf.c

Gilles Caulier

(In reply to Boudewijn Rempt from comment #35)
> The ImageMagick API's are very unstable: there is no guarantee of source
> compatibility between versions. 
In this case, why 91 softwares (see Arch IM page) depend on ImageMagick?

Either they don't care, or they have the manpower to follow the api changes, or whatever. I'm not here to argue with you: I give you the benefit of my experience with ImageMagick, but if you want to come up with a patch that rewrites kimageformats to use ImageMagick, well, we'll see.

Packages depend on the ImageMagick binary, not the libraries.

@Christoph Feck 
Libmagick is required by 16 Arch package, whose ImageMagick, Transcode, Cuneiform, Sk1, Converseen... 
https://www.archlinux.org/packages/extra/x86_64/libmagick/

(In reply to Boudewijn Rempt from comment #33)
> That used to be the way Krita imported/exported images. It was not a huge
> success, and we moved away to using libpng etc. directly.

But libpng is png only, right? OTH Krita has xcf support ... so how does Krita do it? Are they also using KImageFormats? Are they using selfwritten code? Is Krita already compatible with 2.9 and 2.10 of Gimp?

Looking at the impressive long list of supported fileformats, I'd say it could still be worthwhile to follow ImageMagick despite API changes ... I mean, how often do they overthrow it? Maybe they also learned from last time, that their changes were met with mixed reactions?

No, kimageformats (and qimagio) is only suitable as a fallback for formats we haven't implemented yet, it's not possible to offer image format specific options for export, or multiple layers of different kinds for import.

For xcf we use xcftools, the first "library" created for loading xcf files (not saving), which stopped being maintainted donkeys years ago.

In fact, the gimp people a) are of the opinion that nothing bug gimp should load or save xcf since it's their internal format and b) don't want to add a rendered image somewhere in the xcf file like Photoshop or Krita does, because that would take extra space and c) don't want to document their format other than in the source code, because of a).

Porting kimageformats to ImageMagick would be more work than just fixing the xcf filter, I'm pretty sure of that.

*** Bug 412339 has been marked as a duplicate of this bug. ***

Hi
I posted here a bug report to Gimp Team about Gimp 2.10 XCF support
https://gitlab.gnome.org/GNOME/gimp/issues/3990

This is a copy:

GIMP version: 2.10.12
Operating System: Arch Linux and all others
Is the issue reproducible? : Always
Hi
The facts: at this time, Gimp 2.10 XCF files can be only loaded by Gimp (to be written by another application is not the subject here)
Except Gimp 2.10, none application is able to display Gimp 2.10 XCF multi-layers thumbnails.
Digikam, Gthumb, Nautilus, Thunar, Pcmanfm, Gwenview, Geeqie, etc, none of them can display XCF thumbnails
It has changed since 2.9 version. Before, other graphic applications could display <= 2.8 Gimp version XCF thumbnails.
Why this changing behaviour? It's a big problem for users and others apps developers.
This issue doesn't come from others softwares developers incompetence.
It comes from the fact that Gimp 2.10 code cannot be used inside other graphic softwares.
I imagine they could, in an other world, do a kind of "reverse engineering" but that would be a strange thing in a Free Software / Open Source world.
Personally, I use Gimp and Digikam. As i explain above, since 2.9 Gimp version, I can't see XCF thumbnails in Digikam.
I have to convert Gimp 2.10 XCF files to proprietary Adobe format, .psd, to see them. It may sound strange in the Linux world but, sadly, it's the reality....
Gimp is an extraordinary, irreplaceable software but, I regret to announce here that Gimp 2.10 XCF format looks like a closed format. On paper, the code is open but, in facts, it is unusable in other projects, just like a closed code.
All open source file formats offer a library to handle the data inside. Why not Gimp ?

Git commit a94d1a85fe205de4d9ca6bbdd3e48f423d3e4164 by Boudewijn Rempt.
Committed on 26/09/2019 at 07:39.
Pushed by rempt into branch 'master'.

Fix clearing the recent files from the welcome screen

M  +1    -1    libs/ui/KisMainWindow.cpp
M  +7    -6    libs/ui/KisMainWindow.h
M  +1    -6    libs/ui/KisWelcomePageWidget.cpp
M  +0    -2    libs/ui/KisWelcomePageWidget.h

https://invent.kde.org/kde/krita/commit/a94d1a85fe205de4d9ca6bbdd3e48f423d3e4164

Oops, sorry for the noise.

Seems resolved in https://invent.kde.org/frameworks/kimageformats/-/commit/c60e77c048d32ccf743cec695743b77b2b25dc87 ?

Yes, looks like it. Let's close this ticket.