Bug 421584

Summary: File Saving is getting extremely slow
Product: [Applications] krita Reporter: Manga Tengu <mangatengu>
Component: File formatsAssignee: Halla Rempt <halla>
Status: RESOLVED FIXED    
Severity: normal CC: ghevan, halla, lynx.mw+kde
Priority: NOR    
Version: 4.3.0-beta1   
Target Milestone: ---   
Platform: macOS (DMG)   
OS: macOS   
Latest Commit: Version Fixed In:
Sentry Crash Report:

Description Manga Tengu 2020-05-15 21:36:45 UTC
Saving a 65 mb file takes up 40 seconds in average.
Recently Krita has gotten really slower in file saving, but with 4.3.0-beta1 
I'm dreading the file saving moment.
Comment 1 Manga Tengu 2020-05-15 21:37:47 UTC
forgot to say I have a 2018 macbook pro 15` with a good cpu.
Comment 2 Manga Tengu 2020-05-15 22:23:42 UTC
The saved file is 3508x6905px and has 16 layers
Comment 3 vanyossi 2020-05-16 05:12:38 UTC
How is the performance of that file on Linux/Windows? Or if you can, send me a link to the file so I can test if the speed is really abnormal on macos compared to other platforms. (you can send the link to my mail if the file cannotbe shared in public)
Comment 4 Manga Tengu 2020-05-16 09:10:38 UTC
Sadly I don't have a linux with a comparable CPU.
I wish I could mail the original it but it's too big. So I made a few slimming operations and tested the performance along the way which led me to interesting infos:

I flattened the image -> still as slow
I made the image 2/3 of the size, it got from 45 to 35 seconds
I filled the layer with solid white -> saves in 1 or 2 seconds.

This is proportional to the size of the file btw. So My guess is the more the final image is complex and fragmented, the slower it gets. The more solid color zones there are, the faster it is...

I'm mailing you the flattened reduced file so it's a tad lower than 20 mb and fits gmail. (still the image is slow and complex -> 35 sec per save). Fill it with white, it will save fast.
Comment 5 Bug Janitor Service 2020-05-17 04:33:16 UTC
Thanks for your comment!

Automatically switching the status of this bug to REPORTED so that the KDE team
knows that the bug is ready to get confirmed.

In the future you may also do this yourself when providing needed information.
Comment 6 Halla Rempt 2020-05-17 09:42:17 UTC
Did you switch on "Trim files before saving" in the the file handling tab of the settings dialog?
Comment 7 Manga Tengu 2020-05-17 10:35:21 UTC
No I didn't, but I also tried manually trimming the file before saving as It was bleeding a lot -> no noticeable change.
I can send you the sample file if your email can handle a 20mb file attachement.
Comment 8 Halla Rempt 2020-05-17 10:46:22 UTC
That shouldn't be a problem, but a google drive or dropbox link or weshare would be fine as well.
Comment 9 Manga Tengu 2020-05-17 11:18:22 UTC
Here you go https://drive.google.com/open?id=18jiDSzE27-DYcMwfmTf2rksXzZOT2zHb
Comment 10 Halla Rempt 2020-05-17 11:35:57 UTC
Hm, that's a pretty simple image... And it does take quite a bit of time saving, though not 40 seconds. Let's see what's going on here...
Comment 11 Manga Tengu 2020-05-17 11:52:24 UTC
yes originally it was bigger and had lots of layers but it was too big to share via email or here. (didn't think of drive).
So I simplified it and realized it was still very slow.
43 seconds seconds to be exact on my configuration: 2018 macbook pro 15` with a good cpu.
Comment 12 Lynx3d 2020-05-18 03:50:09 UTC
Okay I dug a bit into this, for me it was mergedimage.png that ate the most amount of time compressing, and the reason is apparently that since this commit 4d609a56ef572b22ec9a712985e1927c7a0143d0 the compression level is now Z_BEST_COMPRESSION (level 9), because KoQuaZipStore::setCompressionEnabled(true) set this, while initially it is initialized with Z_DEFAULT_COMPRESSION (-1, which apparently is 6 internally for zlib).

Using Z_DEFAULT_COMPRESSION gets the save time and file size pretty much identically to the old code used in krita 4.2.9 on the file I tested.

The file size difference was pretty much negligible btw, ~200kB for a 23MB file for taking 3-4 times longer to save.
Comment 13 Manga Tengu 2020-05-18 07:29:08 UTC
This is very interesting indeed.
Leads me to think the following:

In my comics workflow I have LOTS of files and I do save a lot. Since the compression difference is not significant, can we configure krita through ui settings to compression to 0 ?
Can we even save files without mergedimage.png embeded files ? Then being able to command line krita to generate those mergedimage.png on demand would be very welcome. Would that affect thumbnails btw ?
Comment 14 Halla Rempt 2020-05-18 08:48:13 UTC
The strange thing is, that _is_ a setting, "compress kra files more", so I'll have to check why this is happening, because it's off by default.
Comment 15 Manga Tengu 2020-05-18 08:55:17 UTC
Yes but what I understood is default level is 6, and "compress more" is 9. But as an extrem saver I'd like to know if we could set the level with a slider rahter than a checkbox so we can chose from 0 to 9 ?
Comment 16 Halla Rempt 2020-05-18 08:58:21 UTC
That's for formats like PNG and JPG, not KRA, and apparently it's the compression on the KRA file (which is a zip file) which is the problem, but that should be off by default. I'm looking into it now.
Comment 17 Halla Rempt 2020-05-18 10:15:17 UTC
Git commit f271ef9ac18d79013c99a8780691840d09d8dc4d by Boudewijn Rempt.
Committed on 18/05/2020 at 10:14.
Pushed by rempt into branch 'krita/4.3'.

Do not compress the mergedimage.png file

M  +2    -0    plugins/impex/libkra/kis_kra_saver.cpp

https://invent.kde.org/graphics/krita/commit/f271ef9ac18d79013c99a8780691840d09d8dc4d
Comment 18 Halla Rempt 2020-05-18 10:15:24 UTC
Git commit 183da0759c1ddfd30335480375bb7c0425ba08b9 by Boudewijn Rempt.
Committed on 18/05/2020 at 10:15.
Pushed by rempt into branch 'master'.

Do not compress the mergedimage.png file
(cherry picked from commit f271ef9ac18d79013c99a8780691840d09d8dc4d)

M  +2    -0    plugins/impex/libkra/kis_kra_saver.cpp

https://invent.kde.org/graphics/krita/commit/183da0759c1ddfd30335480375bb7c0425ba08b9
Comment 19 Halla Rempt 2020-05-18 12:35:15 UTC
Git commit 0458ece78da69d290ab0ebc5443919012d858a6d by Boudewijn Rempt.
Committed on 18/05/2020 at 12:34.
Pushed by rempt into branch 'krita/4.3'.

Set the default compression for png files in stores to 3

M  +1    -1    libs/ui/kis_png_converter.cpp

https://invent.kde.org/graphics/krita/commit/0458ece78da69d290ab0ebc5443919012d858a6d
Comment 20 Halla Rempt 2020-05-18 12:35:27 UTC
Git commit 0ac3c5b05ef9075ea11759f7e6f24f7ad76f5b69 by Boudewijn Rempt.
Committed on 18/05/2020 at 12:35.
Pushed by rempt into branch 'master'.

Set the default compression for png files in stores to 3
(cherry picked from commit 0458ece78da69d290ab0ebc5443919012d858a6d)

M  +1    -1    libs/ui/kis_png_converter.cpp

https://invent.kde.org/graphics/krita/commit/0ac3c5b05ef9075ea11759f7e6f24f7ad76f5b69