Bug 360821 - Gimp 2.10 xcf files cannot be loaded
Summary: Gimp 2.10 xcf files cannot be loaded
Status: RESOLVED FIXED
Alias: None
Product: frameworks-kimageformats
Classification: Frameworks and Libraries
Component: general (show other bugs)
Version: 5.46.0
Platform: Arch Linux Linux
: NOR wishlist
Target Milestone: ---
Assignee: Alex Merry
URL:
Keywords:
: 379550 (view as bug list)
Depends on:
Blocks:
 
Reported: 2016-03-21 16:28 UTC by Alex6
Modified: 2020-10-25 17:18 UTC (History)
14 users (show)

See Also:
Latest Commit:
Version Fixed In:
Sentry Crash Report:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Alex6 2016-03-21 16:28:08 UTC
Gwenview unable to load and show xcf files from Gimp 2.9 version.
( no problem with xcf from Gimp 2.8 )

Reproducible: Always
Comment 1 Alex6 2016-03-21 16:30:01 UTC
Gwenview version 15.12.3 using KDE Fw 5.19.0
Comment 2 Christoph Feck 2016-03-21 18:21:36 UTC
If possible, add a link with description of the new file format. I only found references to the old file format.
Comment 3 Rog131 2016-03-21 19:12:30 UTC
Here is a thread of the thumbnailing of the Gimp 2.9 XCF files:  https://mail.gnome.org/archives/gimp-developer-list/2016-January/msg00007.html
Comment 4 maderios 2016-09-11 15:51:25 UTC
I can reproduce bug with:
Fedora 24 system
digikam 5.1 fedora package compiled with kde 5.24.0
installed kde version: 5.25
I use gimp-2.9.4 dev version. I can't open in Digikam 5.1 .xcf files or get  .xcf thumbnails.
DK 5.1 works normally with .xcf created with gimp 2.8.x. 
Digikam doesn't work with .xcf created with gimp 2.9.4 
It seems to be a compatibility problem with xcf version 8 created with gimp 2.9.4
Comment 5 Christoph Feck 2016-09-12 00:45:37 UTC
Marcel, see comment #2. Without information about the new file format, there is nothing we can do.
Comment 6 maderios 2016-09-12 10:38:25 UTC
I think it's useful
- Add LZMA2 compressed file support (.xcf.xz/.xcfxz)
- Internal tile compression for XCF is now zlib (instead of RLE) 
http://wiki.gimp.org/wiki/Release:2.10_changelog
Comment 7 maderios 2016-09-13 16:15:06 UTC
More information about xcf here:
https://github.com/GNOME/gimp/blob/master/devel-docs/xcf.txt
Mainly:
PROP_COMPRESSION (essential)
  uint32  17       Type identification
  uint32  1        One byte of payload
  byte    comp     Compression indicator; one of
                     0: No compression
                     1: RLE encoding
                     2: (Never used, but reserved for zlib compression)
3: (Never used, but reserved for some fractal compression)
Comment 8 Christoph Feck 2016-09-13 18:45:35 UTC
The document referenced in comment #7 describes the old (2.8) format, but does not have any information about the 2.9 additions/changes.
Comment 9 Christoph Feck 2017-05-05 17:59:33 UTC
*** Bug 379550 has been marked as a duplicate of this bug. ***
Comment 10 Janet 2017-05-05 22:49:41 UTC
(In reply to Christoph Feck from comment #2)
> If possible, add a link with description of the new file format. I only
> found references to the old file format.

Might help?
https://bugzilla.gnome.org/show_bug.cgi?id=782244
Comment 11 maderios 2018-04-29 09:31:36 UTC
kimageformats 5.45.0-1 (kf5) is installed on my (Arch linux) system but .xcf thumbnails created with gimp-2.10 or 2.9 are not displayed. Thumbs created with all gimp 2.8 versions are displayed. Problem is present from gimp-2.9, when .xcf format changed. Dolphin-18.0.4 can display new .xcf format created with gimp-2.10 
https://wiki.gimp.org/wiki/Release:2.10_changelog
Substantial rewrite of layer modes and how they are stored in XCF
Add LZMA2 compressed file support (.xcf.xz/.xcfxz)
Internal tile compression for XCF is now zlib (instead of RLE)
Comment 12 maderios 2018-04-29 10:13:09 UTC
Precision, it seems to be a specific kimageformats bug: Dolphin-18.0.4, Nautilus-3.28 and Thunar-1.6.15 can display new .xcf format created with gimp-2.10
Comment 13 maderios 2018-05-14 14:01:30 UTC
I installed today new framework 5.46 + kimageformats-5.46 then I compiled again Digikam-5.9. It doesn't display .xcf thumbnails created with Gimp-2.10.
All others formats (ora, jpeg, png, tiff, etc) are displayed in DK.
Comment 14 Christoph Feck 2018-05-14 14:04:14 UTC
Please provide the information requested in comment #2.
Comment 15 Jehan 2018-05-18 02:57:00 UTC
(In reply to Christoph Feck from comment #14)
> Please provide the information requested in comment #2.

Hello!
I have started updating the documentation of the XCF format.
See the last commits to see the major changes in the format since GIMP 2.8:
https://git.gnome.org/browse/gimp/log/devel-docs/xcf.txt

I think the only remaining feature that still have to be added (will do tomorrow) is about masks on group layers. And there may be an information about endianness but this is mostly a bug which touched development format of XCF and which should not be a problem if you only target stable XCF versions.
Comment 16 maderios 2018-05-18 08:50:24 UTC
Good news. Thanks, Jehan :)
Comment 17 Christoph Feck 2018-05-18 11:42:45 UTC
Thanks for the update; changing status.

Regarding layer blending modes, do you have a link to the code to understand the actual formulas?
Comment 18 Jehan 2018-05-18 16:44:58 UTC
(In reply to Christoph Feck from comment #17)
> Thanks for the update; changing status.
> 
> Regarding layer blending modes, do you have a link to the code to understand
> the actual formulas?

In this file:
https://git.gnome.org/browse/gimp/tree/app/operations/layer-modes/gimp-layer-modes.c
You'll see the list of layer modes, and in particular notice the op_name parameter, which is the string you want to look for the implementation.

For instance, the "Normal" layer mode is the GEGL operation "gimp:normal" whose implementation is: https://git.gnome.org/browse/gimp/tree/app/operations/layer-modes/gimpoperationnormal.c
Comment 19 Jehan 2018-05-18 23:13:12 UTC
So I spent a good part of my day updating the docs, and I think it's not too bad now. After reviewing myself the changes, I think that the main points you'll want to check are:

- of course the new layer modes algorithms, as you note, but also the new composite mode, composite space and blend mode properties.
- the fact that pointers can now be on 8 bytes (XCF 11 and over).
- zlib compression
- and color precision per component.
Comment 20 Halla Rempt 2018-05-20 20:59:28 UTC
Ha! There's https://git.gnome.org/browse/gimp/tree/devel-docs/xcf.txt -- and it's been updated just now. See https://www.gimp.org/news/2018/05/20/gimp-2-10-2-released/
Comment 21 Bernd Lachner 2018-08-04 07:48:19 UTC
Is there no way to reuse the XCF load/save code from the gimp source for the xcf kimage format plugin? Is it really necessary to implement it from scratch? I think support for the new xcf format in the xcf kimage format now is even more important as GIMP 2.10 is released. For example, digikam now can't show many xcf files saved with GIMP 2.10.
Comment 22 Halla Rempt 2018-08-04 08:23:04 UTC
No, that is not possible. Gimp's xcf code loads directly into gimp's internal data structures, and saves directly from them. Re-using that code basically means embedding gimp. So it would be better to update the xcftools codebase, but afaik, nobody is working on that yet.
Comment 23 Holger 2018-10-07 20:08:42 UTC
If there is a major overhaul of the xcf reader due, please also consider bug 126072 to incorporate gzip and bzip2 compression for image collections viewed in gwenview / digikam. Thanx!
Comment 24 Halla Rempt 2018-10-07 20:15:09 UTC
That should be completely independent -- and the patch looks like it could have been committed before in any case.
Comment 25 Christoph Feck 2019-01-05 14:22:02 UTC
*** Bug 360806 has been marked as a duplicate of this bug. ***
Comment 26 Halla Rempt 2019-01-27 11:30:44 UTC
Albert asked me on IRC last night whether I thought that this might be a good gsoc project, but I was asleep (01:54:53)... The answer is, yes, could very well be. Take the xcftools library, modernize it and update it to xcf for gimp 2.10, then update this kimageio plugin and Krita's xcf plugin.
Comment 27 caulier.gilles 2019-03-30 14:51:32 UTC
Hi all in this room.

I just recompiled digiKam 6.1.0 pre-release AppImage with KImageFormat 5.46:

https://files.kde.org/digikam/

...and the problem still reproducible with XCF images generated with Gimp 2.10 (zlib compression).

This is the console trace when digiKam try to load XCF through Qt Image Loader delegate to KImageFormat :

Digikam::MetaEngine::Private::printExiv2ExceptionError: Cannot load metadata using Exiv2   (Error # 11 :  /home/gilles/Pictures/Ubaye 2018/09-07-2018/DSC05816.xcf: The file contains data of an unknown image type
Digikam::DImg::load: "/home/gilles/Pictures/Ubaye 2018/09-07-2018/DSC05816.xcf"  : QIMAGE file identified
Digikam::QImageLoader::load: Can not load " "/home/gilles/Pictures/Ubaye 2018/09-07-2018/DSC05816.xcf" " using DImg::QImageLoader!
Digikam::QImageLoader::load: Error message from loader: "Unable to read image data"

The message from the last one come from KImageFormat XCF image loader.

I tested with gimp 2.8 and KImageFormat 5.46 has no problem to generate XCF thumbnails.

Gilles Caulier
Comment 28 caulier.gilles 2019-03-30 15:14:20 UTC
Fro info, ImageMagick is able to display Gimp 2.10 XCF files :

[gilles@localhost 09-07-2018]$ display --version
Version: ImageMagick 7.0.8-35 Q16 x86_64 2019-03-26 https://imagemagick.org
Copyright: © 1999-2019 ImageMagick Studio LLC
License: https://imagemagick.org/script/license.php
Features: Cipher DPC HDRI Modules OpenCL OpenMP 
Delegates (built-in): bzlib cairo fftw fontconfig freetype gvc heic jbig jng jp2 jpeg lcms lqr ltdl lzma openexr pangocairo png rsvg tiff webp wmf x xml zlib
[gilles@localhost 09-07-2018]$ 


Gilles Caulier
Comment 29 maderios 2019-04-01 15:17:54 UTC
If I understand:
- For three years, Kimageformats can't diplay xcf files (except old gimp files)
- ImageMagick can display all xcf files (I tested it)
Replacing Kimageformats by ImageMagick could be a good way to solve issue.
https://github.com/ImageMagick/ImageMagick
https://en.wikipedia.org/wiki/ImageMagick
ImageMagick is already required by 91 Archlinux packages:
https://www.archlinux.org/packages/extra/x86_64/imagemagick/
Comment 30 Albert Astals Cid 2019-04-01 17:13:53 UTC
You can't replace kimageformats with imagemagick, that simply makes no sense
Comment 31 caulier.gilles 2019-04-01 18:12:27 UTC
To replace whole image format supported by KImageFormat certainly no, but to call ImageMagick C++ API to get the big advantage to render plenty of image formats, yes...

https://www.imagemagick.org/script/formats.php#supported

Gilles Caulier
Comment 32 Holger 2019-04-01 19:33:12 UTC
@Gilles

I love that suggestion, as it will also solve the long-standing SVG issue: bug 336436

Hope I'm not the april fool now!
Comment 33 Halla Rempt 2019-04-02 06:12:08 UTC
That used to be the way Krita imported/exported images. It was not a huge success, and we moved away to using libpng etc. directly.
Comment 34 caulier.gilles 2019-04-02 07:50:05 UTC
ImageMagick C++ API have been a problem ? Which one exactly ? The IM C++ exception no compatible with Qt ?

Gilles Caulier
Comment 35 Halla Rempt 2019-04-02 07:58:59 UTC
The ImageMagick API's are very unstable: there is no guarantee of source compatibility between versions. That's why GraphicsMagick was created back in the days, but that fell behind. It was a huge maintenance burden and mess. It would be much better to just check what ImageMagick has done in their XCF filter and try to port it to kimageformats. It cannot be that hard...
Comment 36 caulier.gilles 2019-04-02 08:03:07 UTC
Ah yes, i hear about the IM vs GM story.

IM XCF codec is here :

https://github.com/ImageMagick/ImageMagick/blob/master/coders/xcf.c

Gilles Caulier
Comment 37 maderios 2019-04-02 20:35:53 UTC
(In reply to Boudewijn Rempt from comment #35)
> The ImageMagick API's are very unstable: there is no guarantee of source
> compatibility between versions. 
In this case, why 91 softwares (see Arch IM page) depend on ImageMagick?
Comment 38 Halla Rempt 2019-04-02 20:53:39 UTC
Either they don't care, or they have the manpower to follow the api changes, or whatever. I'm not here to argue with you: I give you the benefit of my experience with ImageMagick, but if you want to come up with a patch that rewrites kimageformats to use ImageMagick, well, we'll see.
Comment 39 Christoph Feck 2019-04-02 20:57:13 UTC
Packages depend on the ImageMagick binary, not the libraries.
Comment 40 maderios 2019-04-03 09:42:31 UTC
@Christoph Feck 
Libmagick is required by 16 Arch package, whose ImageMagick, Transcode, Cuneiform, Sk1, Converseen... 
https://www.archlinux.org/packages/extra/x86_64/libmagick/
Comment 41 Holger 2019-04-03 16:44:40 UTC
(In reply to Boudewijn Rempt from comment #33)
> That used to be the way Krita imported/exported images. It was not a huge
> success, and we moved away to using libpng etc. directly.

But libpng is png only, right? OTH Krita has xcf support ... so how does Krita do it? Are they also using KImageFormats? Are they using selfwritten code? Is Krita already compatible with 2.9 and 2.10 of Gimp?

Looking at the impressive long list of supported fileformats, I'd say it could still be worthwhile to follow ImageMagick despite API changes ... I mean, how often do they overthrow it? Maybe they also learned from last time, that their changes were met with mixed reactions?
Comment 42 Halla Rempt 2019-04-03 16:54:55 UTC
No, kimageformats (and qimagio) is only suitable as a fallback for formats we haven't implemented yet, it's not possible to offer image format specific options for export, or multiple layers of different kinds for import.

For xcf we use xcftools, the first "library" created for loading xcf files (not saving), which stopped being maintainted donkeys years ago.

In fact, the gimp people a) are of the opinion that nothing bug gimp should load or save xcf since it's their internal format and b) don't want to add a rendered image somewhere in the xcf file like Photoshop or Krita does, because that would take extra space and c) don't want to document their format other than in the source code, because of a).

Porting kimageformats to ImageMagick would be more work than just fixing the xcf filter, I'm pretty sure of that.
Comment 43 Christoph Feck 2019-09-25 20:37:56 UTC
*** Bug 412339 has been marked as a duplicate of this bug. ***
Comment 44 maderios 2019-09-25 20:58:50 UTC
Hi
I posted here a bug report to Gimp Team about Gimp 2.10 XCF support
https://gitlab.gnome.org/GNOME/gimp/issues/3990

This is a copy:

GIMP version: 2.10.12
Operating System: Arch Linux and all others
Is the issue reproducible? : Always
Hi
The facts: at this time, Gimp 2.10 XCF files can be only loaded by Gimp (to be written by another application is not the subject here)
Except Gimp 2.10, none application is able to display Gimp 2.10 XCF multi-layers thumbnails.
Digikam, Gthumb, Nautilus, Thunar, Pcmanfm, Gwenview, Geeqie, etc, none of them can display XCF thumbnails
It has changed since 2.9 version. Before, other graphic applications could display <= 2.8 Gimp version XCF thumbnails.
Why this changing behaviour? It's a big problem for users and others apps developers.
This issue doesn't come from others softwares developers incompetence.
It comes from the fact that Gimp 2.10 code cannot be used inside other graphic softwares.
I imagine they could, in an other world, do a kind of "reverse engineering" but that would be a strange thing in a Free Software / Open Source world.
Personally, I use Gimp and Digikam. As i explain above, since 2.9 Gimp version, I can't see XCF thumbnails in Digikam.
I have to convert Gimp 2.10 XCF files to proprietary Adobe format, .psd, to see them. It may sound strange in the Linux world but, sadly, it's the reality....
Gimp is an extraordinary, irreplaceable software but, I regret to announce here that Gimp 2.10 XCF format looks like a closed format. On paper, the code is open but, in facts, it is unusable in other projects, just like a closed code.
All open source file formats offer a library to handle the data inside. Why not Gimp ?
Comment 45 Halla Rempt 2019-09-26 07:40:42 UTC
Git commit a94d1a85fe205de4d9ca6bbdd3e48f423d3e4164 by Boudewijn Rempt.
Committed on 26/09/2019 at 07:39.
Pushed by rempt into branch 'master'.

Fix clearing the recent files from the welcome screen

M  +1    -1    libs/ui/KisMainWindow.cpp
M  +7    -6    libs/ui/KisMainWindow.h
M  +1    -6    libs/ui/KisWelcomePageWidget.cpp
M  +0    -2    libs/ui/KisWelcomePageWidget.h

https://invent.kde.org/kde/krita/commit/a94d1a85fe205de4d9ca6bbdd3e48f423d3e4164
Comment 46 Halla Rempt 2019-09-26 07:45:38 UTC
Oops, sorry for the noise.
Comment 48 Halla Rempt 2020-10-25 17:18:54 UTC
Yes, looks like it. Let's close this ticket.