Bug 424797 - Alpha calculations on alpha masked layers with opacity is 'wrong'?
Summary: Alpha calculations on alpha masked layers with opacity is 'wrong'?
Status: RESOLVED FIXED
Alias: None
Product: krita
Classification: Applications
Component: Layer Stack (show other bugs)
Version: nightly build (please specify the git hash!)
Platform: Microsoft Windows Microsoft Windows
: NOR normal
Target Milestone: ---
Assignee: Dmitry Kazakov
URL:
Keywords: triaged
Depends on:
Blocks:
 
Reported: 2020-07-29 13:00 UTC by Ralek Kolemios
Modified: 2020-08-12 20:44 UTC (History)
3 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments
Example of the bug (273.11 KB, application/x-krita)
2020-07-29 13:00 UTC, Ralek Kolemios
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Ralek Kolemios 2020-07-29 13:00:44 UTC
Created attachment 130486 [details]
Example of the bug

This one's going to be a hard one.

Krita seems to miscalculate how semi-opaque pixels merge with other semi-opaque pixels when combined as an alpha mask. This can create a 'halo' effect around lineart or completely mess up gradients with alpha masks applied to them.

To better explain the issue, I've included a Krita file with 3 examples. Take a look at the red square to see what I'm talking about.

The top group [group 1] is the setup when I first noticed this behavior. Something was wrong, off. Even if the calculations on this layer are somehow mathematically 'correct', this definitely is not what artists want since it can make lineart look like it has a white 'gap' between the lineart and the color behind it. For example: https://i.imgur.com/96114DI.png

There's a fix for it, technically speaking. 
In group 2, It's the exact same setup, but the 50% opacity layer is set to the 'addition' layer blend mode. This should theoretically change nothing, but it fixes the alpha issue entirely.

The final group [group 3] is the reference, this is how the line should look if there was no alpha mask entirely and the line was just straight up colored what you wanted it to be. Notice that it more closely matches group 2 than group 1, which is affected by the bug.


Krita
  Version: 4.3.1-alpha (git 08ecea2)

OS Information
  Kernel Version: 10.0.18363
  Pretty Productname: Windows 10 (10.0)
Comment 1 Tiar 2020-07-29 15:33:17 UTC
Situation one, mode: Normal
- if you have black dot of color with 50% transparency, and the layer opacity is set to 100%,
- and a new layer with opaque dot with a color, and the layer opacity is set to 50%,
then the result will be: 2/3 of the result color, opacity: 50%.

mode: Addition:
the result will be: 1/2 of the result color, opacity: 50%.

For example in case of color (200, 100, 20)
case Normal: -> result (133, 67, 13)
case Addition: -> result (100, 50, 10)

However I cannot say which one is more correct mathematically.
Comment 2 Dmitry Kazakov 2020-07-30 10:03:44 UTC
This bug is related to bug 424525 in some way. I will investigate further.
Comment 3 Bug Janitor Service 2020-07-30 11:57:29 UTC
A possibly relevant merge request was started @ https://invent.kde.org/graphics/krita/-/merge_requests/452
Comment 4 Dmitry Kazakov 2020-07-30 12:03:28 UTC
Hi, Aaron!

Could you please test this package:

https://yadi.sk/d/LwK1uXxECb5k4w

This package should fix the halo effect. But most importantly I need to know if it causes some other unexpected regressions.

There is a detailed explanation of what testing is needed in this merge request:
https://invent.kde.org/graphics/krita/-/merge_requests/452
Comment 5 Ralek Kolemios 2020-07-30 15:30:54 UTC
Hey Dmitri,
Unfortunately I'm not primarily a painter so normal use for me and previous artworks likely won't produce any undesired effects. The one use case in which this bug was bothering me (coloring lineart non-destructively by overlaying the color on top of the lineart) works great in this build.

That said I do understand that this is a core change to how the program processes the layer stack and will inherently change the output of some older files for others. While I'm unsure of how I can rigorously test the change myself, I'll be using this build for the next week or two just to see if anything very obvious sticks out to me during my usual workflow.

As for the questions posed in the merge request, I'll give my personal opinion on. Keep in mind I'm likely not the kind of artist who would be most affected by this change.
> The patch changes the behavior of "Normal" blending mode in "Inherit Alpha" mode. Is this change acceptable?
This behavior is acceptable as far as I can see.
> May it cause harm to any existing files that painters own?
While it may change how previous artwork is rendered, I have a hard time believing the changes will be considered a regression by artists.
> Should we implement any backward compatibility mode? (It can only technically be a global switch with Krita restart)
I personally don't think backwards compatibility is necessary unless this change gets pushed and artists begin to ask for it. Which is something I don't see happening, in my opinion.
> The patch may theoretically change how brushes in BuildUp mode (or ColorSmudge engine) work in "Locked Alpha" mode. Can you reproduce any changes in existing presets?
I don't notice any changes to build-up mode or color smudge mode on Inherit Alpha layers, but painterly-style is not my area of expertise. Color Smudge engine does not work on 'Alpha-locked' layers or inside specific lasso selections, but that's a different bug entirely.
Comment 6 Dmitry Kazakov 2020-07-31 07:18:05 UTC
Hi, Ralek!

Thank you very much for your feedback! I'll wait a week for the results of your testing! :)

I've also started a thread on Krita Artists, so more people could test that:

https://krita-artists.org/t/painters-we-need-help-in-testing-halos-around-strokes-in-inherit-alpha-layers-with-transparency/10318/6
Comment 7 Ralek Kolemios 2020-08-04 21:36:17 UTC
I've been using the build for a little while now, and haven't noticed any major regressions in any of my brushes or layer stack. However most of my layer setups and brushes are not paint/opacity-oriented and I'm probably not the best example case for finding niche bugs in this update.
Comment 8 Dmitry Kazakov 2020-08-12 19:38:41 UTC
Okay, thanks for your feedback! :)
Comment 9 Dmitry Kazakov 2020-08-12 20:14:14 UTC
Git commit 4e1cb547991ea020568df088827c9b8e1afa68c0 by Dmitry Kazakov.
Committed on 12/08/2020 at 19:40.
Pushed by dkazakov into branch 'krita/4.3'.

Fix halos around strokes in "Inherit Alpha" layers with transparency

When a layer is applied in inherit alpha mode, then the blending should
happen only via source alpha value. Destination alpha should not be mixed
into it, because it will create halos around the shapes. Conceptually,
it works as if the backdrop alpha value were zero.

M  +4    -1    libs/pigment/compositeops/KoCompositeOpAlphaBase.h
M  +1    -1    libs/pigment/compositeops/KoOptimizedCompositeOpOver128.h
M  +1    -1    libs/pigment/compositeops/KoOptimizedCompositeOpOver32.h

https://invent.kde.org/graphics/krita/commit/4e1cb547991ea020568df088827c9b8e1afa68c0
Comment 10 Dmitry Kazakov 2020-08-12 20:44:16 UTC
Git commit a5965ae2f73d75ae98ac54c97dfa614d0b349bec by Dmitry Kazakov.
Committed on 12/08/2020 at 20:15.
Pushed by dkazakov into branch 'master'.

Fix halos around strokes in "Inherit Alpha" layers with transparency

When a layer is applied in inherit alpha mode, then the blending should
happen only via source alpha value. Destination alpha should not be mixed
into it, because it will create halos around the shapes. Conceptually,
it works as if the backdrop alpha value were zero.

M  +4    -1    libs/pigment/compositeops/KoCompositeOpAlphaBase.h
M  +1    -1    libs/pigment/compositeops/KoOptimizedCompositeOpOver128.h
M  +1    -1    libs/pigment/compositeops/KoOptimizedCompositeOpOver32.h

https://invent.kde.org/graphics/krita/commit/a5965ae2f73d75ae98ac54c97dfa614d0b349bec