Bug 359160

Summary: 15.13.0 git master - Composite in tracks not working right when three (or more?) tracks are used. (Video example included)
Product: [Applications] kdenlive Reporter: Unknown <null>
Component: Effects & TransitionsAssignee: Vincent PINON <vpinon>
Status: RESOLVED FIXED    
Severity: normal CC: jb, null, wegwerf-1-2-3
Priority: NOR    
Version: unspecified   
Target Milestone: ---   
Platform: Compiled Sources   
OS: Linux   
URL: https://youtu.be/yHfh1ul9b_o
Latest Commit: Version Fixed In:

Description Unknown 2016-02-09 00:05:07 UTC
See video for example with details included in it, in the URL field.

Tracks aren't compositing tracks properly when there are clips on both tracks above track Video 3.

Reproducible: Always




Tested with Kdenlive 15.13.0 - built from source on 2016.02.08 - on Fedora Linux x64, GNOME 3.18.
Comment 1 Wegwerf 2016-02-09 16:58:55 UTC
I'm under the impression that this is a general problem with the transparent track mechanism. You could verify this by switching all tracks to opaque and then adding classic composite transitions to tracks 1 and 2. This should show the same effect as you've shown in your video. I think that this comes due to the fact that track 1 is composited with 2 ... and that's already wrong. It should be composited to track 3 instead.

The existing transparent track mechanism could be fixed: for each transparent track, find the nearest track below that's opaque. Only this will set up the composition pipeline as intented. The current implementation isn't exactly correct for three and more transparent tracks stacked onto each other. Maybe the algorithm I outlined above can help?
Comment 2 Unknown 2016-02-10 07:36:44 UTC
Ideally, in a perfect video editing world, the timeline tracks would behave like so:

VIDEO TRACK 3 --------------------------[|||||||||||||||]-----
VIDEO TRACK 2 -------[|||||||]-------------------------------------
VIDEO TRACK 1 ----[|||||||||||||||||||||||||]-----------

([||||||||||||||||||] = a clip in the timeline)

If a video clip is on track 1, and track 2 has a clip above it with an alpha channel, then the transparency portion of that clip would show the clip below it on Track 1. Similarly, if there was a video clip on track 3, two tracks above the video clip on track 1, the alpha transparency from the video clip on track 3 would show the video clip on track 1. This would happen automatically.

In other words, If there is a video clip in the timeline with an alpha channel, the transparency part would show the clip beneath it, regardless of how many tracks below the clip in question is on.

Hope that makes sense. :/
Comment 3 Wegwerf 2016-02-10 11:03:00 UTC
Jesse, it does make sense.

However, the catch is in the handling of the alpha layer: ideally, this should be a MIN(a,b) handling when compositing one clip with alpha layer onto another clip also with alpha layer. Unfortunately, MLT doesn't support MIN, MAX, etc. but instead only AND, OR, XOR. Now boolean operators only work for alpha when there are exactly two alpha levels in use, namely 0 and 255. For gradual alpha, as it is often used to achieve smooth blending, AND is not substitute for MIN. The only way around this is to *not cascade* composition of multiple tracks, one after another, but instead to all them composite only with the final, bottommost layer. Only then alpha AND handling doesn't cause any problems. MLT should support at least the MIN operator for alpha layer compositing...
Comment 4 Unknown 2016-02-10 18:20:06 UTC
Wegwerf, appreciate the explanation. So, is there anything that can be done to remedy this? Many users will expect the function of tracks in the timeline to be as I'd depicted above. I think it's GREAT that the dev's are implementing this function, and when the time's right, maybe they can take a look at this and see if it can be improved?
Comment 5 Wegwerf 2016-02-10 18:25:44 UTC
Maybe discussing with Dan Dannedy of MLT fame first regarding handling of alpha layers and the binary AND versus MIN operators?
Comment 6 Unknown 2016-02-10 19:45:42 UTC
Will do, I'll see if I can contact him and correspond about the issue.
Comment 7 Unknown 2016-02-16 20:20:34 UTC
Well, I downloaded Shotcut and tested the same track tests as the project I was working on in Kdenlive (I know that Dan D. heads up Shotcut and MLT, so I'm assuming the projects are pretty tightly intertwined, and that Shotcut should reflect what MLT can do).

I tried to perform the track composite functions, and they worked EXACTLY as I thought they should, without issue. I've included a video to see what I'm talking about: https://youtu.be/KjoPkPMmYbA

I had three tracks set up, a video clip on bottom, a black .png image in middle, and a logo .png image with alpha channel on top. I was able to remove the middle black image, and the video showed the logo on the top track, and the video on the bottom without issue. I was able to manipulate the opacity and the cropping of both the middle and top tracks without them affecting any of the other tracks at all.

I'm going to assume that this means that MLT has the capability to do what I'm suspecting it can. I'm going to suggest, like in the OP, that there's still a bug in the composite function of the tracks themselves in Kdenlive. I expect the compositing in all Kdenlive tracks - even the ones added - to behave like they did in Shotcut; I shouldn't need an affine or composite transition anymore to make these work properly (though, even when they're implemented, I still can't achieve my goal with the track layers).

Can any of the dev's confirm this issue?
Comment 8 Jean-Baptiste Mardelle 2016-02-25 07:28:15 UTC
Thanks Jesse for your investigation. I confirm and should be easy to behave like Shotcut. Will work on it tonight.
Comment 9 Unknown 2016-02-25 17:23:06 UTC
Gratitude is all mine, JB. Thanks for considering this suggestion! I'm confident that it'll make things much more user-friendly for a lot of editors out there. :)

I look forward to making a video with a list of all the big improvements to Kdenlive since the 15.12 release (and there have been many). We'll take the majority marketshare of free-software video editors, yet!
Comment 10 Jean-Baptiste Mardelle 2016-02-26 17:49:43 UTC
Git commit ffae410dab837906843a77743eea1db4d7228bf1 by Jean-Baptiste Mardelle.
Committed on 26/02/2016 at 17:49.
Pushed by mardelle into branch 'master'.

Fix composite in multitrack

M  +7    -18   src/doc/kdenlivedoc.cpp
M  +2    -44   src/timeline/customtrackview.cpp
M  +1    -3    src/timeline/customtrackview.h
M  +20   -0    src/timeline/timeline.cpp
M  +6    -2    src/timeline/timeline.h
M  +42   -1    src/timeline/transitionhandler.cpp
M  +2    -0    src/timeline/transitionhandler.h

http://commits.kde.org/kdenlive/ffae410dab837906843a77743eea1db4d7228bf1
Comment 11 Unknown 2016-02-26 18:08:24 UTC
Works way, WAY easier!!! No more need for the Affine transition or Composite transition to apply alpha channels. Seriously, this will be a huge benefit to the ease-of-use for users.

Thanks JB, you rock!

Marking this bug as resolved.
Comment 12 Wegwerf 2016-02-26 18:18:34 UTC
Just tested it using an SVG inkscape clip on top video track, a Kdenlive title clip on the middle track, and a video clip on the bottom video clip. It now works correctly in principle.

However I notice that there is no proper anti-aliasing so that the text in the SVG and title clips look very ragged and torn-out. To cross-check I've added manual affine transitions that will result in proper smooth anti-aliasing the SVG and title text onto the video clip.
Comment 13 Wegwerf 2016-02-26 18:21:37 UTC
Uhoh, please reopen this bug; while it works in principle, the results are not smooth.
Comment 14 Wegwerf 2016-02-26 18:39:04 UTC
Never mind, this seems to be an issue in my installation -- I could not reproduce the problem. So please leave this bug closed-fixed. If I happen to see any problems I can create a new bug that would be better anyway.

Jean-Baptiste, I join Jesse in his praise: this surely is an important Kdenlive improvement that will help a lot of newcomers get around successfully in Kdenlive.

From a short glimpse at the code changes I grasped that my understanding seems to be at least partly correct in that the "transition composition pipeline" wasn't correctly set up: composition needs to happen to the "lowest" track and not unconditionally with the next lower track. Did I understand this correct?
Comment 15 Wegwerf 2016-02-26 19:12:23 UTC
Can now reproduce render quality bug with transparent tracks when semi-transparent text is used: https://bugs.kde.org/show_bug.cgi?id=359836