See video for example and narrated details. When I have multiple video clips in the timeline on different video tracks, and they're on top of each other, the playback speed & frame rate slows WAY down. Reproducible: Always Steps to Reproduce: 1. Put multiple clips in timeline. Stack them on top of each other so they're overlaying each other in timeline. 2. Playback project. 3. Actual Results: When the timeline position cursor crosses over the area where there are multiple video clips on top of each other, playback speed slows down significantly. Regardless of if there are any compositing effects on any of the clips or not. Expected Results: Playback speed should be a LOT smoother, even realtime, especialy on a machine like mine (AMD FX(tm)-8150 Eight-Core Processor, 8GB DDR3 RAM, Nvidia Geforce GTX 660 graphics). Bug reported while using Kdenlive 16.07.70 git master build via kdenlive-master ppa. Ubuntu 16.04 x64, Unity 7.4.0 desktop environment. Kdenlive is using CPU for operations, but Ubuntu is using the latest Nvidia proprietary graphics driver for video processing -- if that makes a difference.
Hi Jesse, I also noticed this on my projects. Seems to be related to the track composite transitions. Can you disable the track compositing and see if it fixes the issue ? Not exactly sure if some changes to track compositing made it worse or if the problem is here since we added track compositing...
Hey sir, yes I can confirm that disabling the composite tracks and putting a Composite transition between the video clips in separate tracks restores the playback speed to real time. It does fix the issue. Of course, we know that doing so removes the benefits of the composite tracks: not able to put the video clips in any place in the timeline and have them composite properly, not needed to put a composite transition in between video clips on separate tracks, etc. Any way to keep the framerate with the track compositing?
UPDATE: Interestingly enough, when you have a title clip on a track with composite enabled, and have a video clip in the base track below it (no composite function) and you put a dissolve transition between them, the playback while passing over the dissolve transition is flawless. However, once the playback reaches the end of the dissolve and plays both clips full composited over each other, the playback speed reduces. If it goes to a dissolve transition at the end of a clip, then the playback through the dissolve transition works great; it's very smooth.
I'm changing the status of this bug from normal to major, simply because many users (first time or otherwise) who experience this bug will probably be perplexed and more than likely frustrated with Kdenlive's performance, when many of us more experienced users know there's a workaround. Kdenlive's performance should be optimum right from the get go, yeah? Will help many users greatly and build better rapport.
A little more testing revealed this: working with projects and/or video clips that were 1920x1024 or 1920x1080 resulted in the bug in my OP. However, working with a project and/or video clip that was 720x320 resulted in no playback issues when placing clips over another in the timeline, or even using transitional effects -- they played at 24fps no problem. So, perhaps the issue is in the way MLT processes 1920x1080 or higher resolution clips? :/ I dunno, just spit-balling, here.
(In reply to Jean-Baptiste Mardelle from comment #1) > Hi Jesse, > > I also noticed this on my projects. Seems to be related to the track > composite transitions. Can you disable the track compositing and see if it > fixes the issue ? Not exactly sure if some changes to track compositing > made it worse or if the problem is here since we added track compositing... I can confirm that the performance is good when using a composite transition instead of track compositing. The framerate definitely drops when using track compositing.
I don't believe that Shotcut has this issue with playback and track compositing. Being that I think the track compositing used in Kdenlive was based off of Shotcut's model (correct me if I'm wrong), maybe looking at how Shotcut handles it might give a clue to a solution?
A little bit more testing (if it helps) shows that there's much more CPU load when playing clips overlaying each other with track compositing, than playing a clip that doesn't have another clip over or under it. In other words, during playback over a section in the timeline where there's more than one clip present on multiple tracks, my system monitor shows a heftier CPU load. Playing a single clip in a single track results in almost no CPU load at all. Interestingly enough, when I disable track compositing and put an Affine or composite transition in between two clips on the timeline, the CPU STILL revs up. Once the timeline position cursor passes the Affine/composite transition, things start to smooth out to a perfect 24fps, and the CPU goes back to minimal. So, with the current build from the kdenlive-master ppa (May 17, 2016 build) using track compositing OR Affine/composite transitions with track compositing disabled produces the same result: considerable drop in playback speed/framerate, and the CPU revs up, signifying a greater work load (one of my 8 CPU cores goes to 100% during this time). Hope that helps. I feel like this is a pretty significant bug. The last couple of weeks has shown some EPIC feature additions to the Kdenlive master build, but if there's still an underlying issue of playback speed during basic editing functions, I feel like progress is being somewhat crippled. Not a complaint at all, just trying to offer some helpful advice to keep the solid foundation solid, you know?
I asked Dan Dennedy about this and he suggests that we need to dig deeper into the way Kdenlive uses MLT. He created a short MLT XML test that composes seven clips onto each other using composite transitions and only sees the expected load. In contrast, I often see a nonlinear breakdown of performance around six or seven composited clips: rendering drops to one frame every five seconds or so...
I am finally close to fixing this issue. Problem is that the track compositing in Kdenlive (and shotcut) uses the frei0r.cairoblend transition. There is one transition between each track, but because of its current design, the transition is performed even if the track had no transparency. So if you put 3 clips on 3 different tracks one above the other, the transition requests a frame for each clip and performs a compositing operation between each track, resulting in the huge framedrop you noticed. I have now written a new transition for MLT (qtblend) that checks if the top frame has transparency and if it does not, it stops here, so there is no performance loss. I am doing the final testing and hope to merge that in MLT and integrate in Kdenlive next week for the 16.08 release.
That's such good news. You're the man, JB! Happy to test and confirm that everything works over here once it hits the git-master ppa. I think that's great that you're looking to have this hopefully ready before the final 16.08 release. With such great improvements and stability and features, each Kdenlive release becomes better than the last, and more relevant to us video editors and producers. :) Great work, sir!
Testing this the past 48 hours with the latest git master build on some projects. It's coming along quite nicely. I'm not able to put clips on top of each other, or overlapping each other, but spaced out between 1, 2, or three video tracks without any loss in framerate playback. Great work sir! One issue, so far. .png files don't work at all -- the transparency doesn't reveal the clips underneath the image, whether it's right over the clip in the timeline, or whether its a few tracks above it.
Also, are the composite icons supposed to be gone from the timeline track headers, now? Because that's what I'm experiencing with the latest git master build.
No, looks like your project has no composite transition. Did you update mlt to latest git? melt -query transitions | grep qt Should give you qtblend as output. Can you check?
My mlt packages are updated from ppa:kdenlive/kdenlive-master. I've seen a few updates over the last couple of days. This is the current package version installed on my PC: 6.3.0+git201607182118~ubuntu16.04.1 The output from the command you have me (melt -query transitions | grep qt) shows this: jesse@Legacy ~ $ melt -query transitions | grep qt No LADSPA plugins were found! Check your LADSPA_PATH environment variable. jesse@Legacy ~ $
Just checked and there seems to be a problem witou our MLT package. The qtblend files are there but it is not compiled... Going to check that with Vincent and keep you informed.
Ok, found the problem, updated MLT package should be available in the next hour
Rock n' roll. I'll test it once the update is available and confirm the fix. :)
Updated. Preliminary testing shows that (1) Composite icons are back, (2) clips playback smoothly when overlapping other clips in timeline, (3) transparency works with .png files, and (4) title clips are working as well in conjunction with all of the other clips mentioned. These all seem to be working in various combinations with different tracks. Looks like you nailed it!!! Killer, sir! This is a huge leap in workflow and performance for Kdenlive. Can't thank you enough for your time in improving this. :) I'm going to send this bug to the mailing list to have others confirm that they can confirm the fix as well (they may have video files that I don't have, different framerate, etc.)
I now remembered that I had a related bug report about inferior compositing quality still open. So I upgraded my self-compiled MLT to recent git master that comes with the new qtblend transition. Upgraded my existing regression project manually by editing its XML to use qtblend in place of frei0r.cairoblend ... and could not see any noticeable differences between manual affine compositing and automatic qtblend compositing. Didn't check performance though. But looks good to me too. I would like to propose to close this bug. If there are problems appearing later it might be better to track them separately anyway.
Glad to hear it, Wegwerf. Sounds like a plan. Additional testing since my last post has shown great results. Happy to mark this bug as resolved/fixed.
I hate to do this, but I was just working with Kdenlive shortly after closing this bug, and the problem returned: having clips overlapping one another in the timeline is significantly reducing playback speed.
Further investigation shows that this is only apparent on one of my other workstations, with the only real difference between it and the others are that it has two Nvidia GTX660's in it, where the others have two Nvidia GTX970ti's and 980ti's. The machine with the 660's in it is also having the problem of layouts not properly loading at start-up, where the others don't. They're both running the Nvidia 361.42 proprietary driver (as it's needed to utilize the GPU for programs like Blender). Could the graphics card be the issue? Is it possible there's some incompatibility between Kdenlive and older graphics cards?
A few hours ago, I just fixed a huge memleak in the qtblend transition. That might be the cause for your slowdown. I have pushed the fix to MLT git master but for some reason the build fails on our PPA so it has not been updated, I am trying to fix this... will let you know when the fix is in PPA
No sweat. I'll be standing by. :)
Update should be available from PPA now, please test.
Hmmm, nothing in the update manager, yet. I checked the website, and there's a green gear instead of a check mark near the MLT packages that were built 17 minutes ago. Hovering over it shows the tooltip "All packages were successfully built but have not yet been published." I'm also attaching a screenshot of the page. Should I download all of the packages manually and install? Will that mess up anything once the ppa publishes any further updates?
Created attachment 100219 [details] Screenshot of current ppa:kdenlive/kdenlive-master packages on website
Oh, yes sorry I forgot to check the upload status... update should then be available whenever launchpad decides it.. probably something between 30 minutes and 2 hours...
Sounds good. I'll keep checking and report after testing the updates.
Been testing the updated packages. 90% working, but I came across a few issues. I made an in-depth video so you could see them: https://youtu.be/KYySd7hJtKQ. Sorry if I'm talking too fast. Let me know so I can slow it down in future videos. :)
Jesse, thanks for the detailed video. On your remote machine where the slowdown occurs with 2 non transparent clips, can you save the project and open the xml file to make sure it uses the qtblend transition and not the frei0r.cairoblend? (just look for the word "qtblend" in the file, if it is there it means it is used). Otherwise, the slowdown experienced when compositing 2 titles over a video clip is what I expected and I see the same behavior on my machine. The situation will slightly improve when I commit an optimization of MLT's titler module (currently compositing titles is slightly less efficient than compositing png's, my optimization should make titles similar to png's performance). Another possibility is to use the composite transition instead of qtblend. MLT's composite transition is much faster and supports overlaying 3 title clips over a video while keeping a 25fps playback rate. But the drawback of composite is that it has luma bleed (it is fast because it does some approximations), so for example you can see a small blue border around green texts... but maybe we could use composite for timeline and switch to qtblend only for rendering...
You bet. I'm still working on the other stuff, but on the project with the two non-transparent clips, I found this in the .kdenlive saved file: <transition id="transition4"> <property name="a_track">3</property> <property name="b_track">4</property> <property name="compositing">0</property> <property name="distort">0</property> <property name="mlt_service">qtblend</property> <property name="internal_added">237</property> </transition> <transition id="transition5"> <property name="a_track">0</property> <property name="b_track">5</property> <property name="mlt_service">mix</property> <property name="always_active">1</property> <property name="combine">1</property> <property name="internal_added">237</property> </transition> <transition id="transition6"> <property name="a_track">3</property> <property name="b_track">5</property> <property name="compositing">0</property> <property name="distort">0</property> <property name="mlt_service">qtblend</property> <property name="internal_added">237</property> </transition> </tractor> I'm assuming it means that it's using the qtblend, yeah? Slow playback is still occurring, today: 24fps goes down to 9fps using the same files used in the video.
Seeing also a slow-down with a test project of mine, compositing a single title clip on top of a mp4 clip. Interestingly, the fps doesn't immediately drops down as with Affine, but it takes several seconds into the title clip before the frame rate drops from 25fps to 19fps. Strange...
Strange: "melt -query transitions" shows "qtblend" transition. But I don't see it in Kdenlive's transition list pane???
I can definitely empathize with the technical struggles behind trying to make multiple tracks with clips over them playing back smoothly. I once looked at the code out of curiosity (with no programming background), and I near threw up from the confusion. I paid a therapist for a week to counsel me through the trauma. :) Whatever path you think will get the job done best, I'm all for it. From the end user's perspective, all I can say is this: the expected use and experience from the timeline is to use any number of clips (video, .png, title) in any number of tracks - overlapping each other or not - properly showing transparency on each one, and experiencing smooth playback (if their machine/CPU/GPU with movit can handle it). Having effects/filters/etc. on clips is another matter entirely, and you've given a GREAT remedy with the Preview Render feature -- truly a fantastic solution to the framerate drop when playing back clips with FX. I've used it on multiple projects, already, and it's saved me a lot of time. I know using the composite and Affine transitions over clips may seem like the best course because it's already established, but I've never really been for the Affine or composite transitions since my first encounter with Kdenlive (in the form of yellow bars) for two reasons: 1. Because it really makes the term "transition" confusing. Transitions should be moving FROM one clip TO another; one ending, and the other beginning. At least, that's how the term is used in the Industry. Using a "transition" simply to overlay one clip over another doesn't, to the non-uber-techie end user, make sense. And 2. It's making the user take an extra step to do something that they're expecting the editor to do automatically. Other editors you don't have to do anything to make sure that the transparency in any kind of clip shows normally. It's one of the reasons I was - and am - for composite tracks in the first place: it makes the timeline behave like one would expect. If having the "composite transition" built into the timeline would allow for smooth playback, using multiple clips (3 or more) over each other with proper transparency, would show in the monitor exactly how the rendered video would look, and would allow for real-time playback, then I think that would be a great decision. :) Alright, I won't beat a dead horse about it. I'll keep running some more tests from the git master version. Let's keep working to find an elite solution, you know?
I am also seeing the slowdown on my laptop. I am wondering if this might be related to the Qt version. The big speed gain I experienced was on my PC with Qt 5.7, so maybe some optimizations in Qt make the difference. Jesse, can you check the Qt versions on your 2 machines to see if there is a difference there? qmake -v will output the Qt version...
Strange, it didn't look like qmake was installed. I installed the package "qt5-qmake" (using ubuntu 16.04 package base on Linux Mint), but the command still didn't work. I installed "qt4-qmake", and the command worked, then. Here's the output: QMake version 2.01a Using Qt version 4.8.7 in /usr/lib/x86_64-linux-gnu If I'm reading that right, is my Qt version WAY behind? Has Qt made that much progress since the Ubuntu 16.04 release?
The post above was in reference to my machine with two Nvidia 660's in it (it's the only way I can really distinguish my work PC's apart, by the GPU). My work laptop which has two 970ti's in it has the same result: QMake version 2.01a Using Qt version 4.8.7 in /usr/lib/x86_64-linux-gnu
Looking up the version tab in the latest git master build of Kdenlive that I have, it shows this version of Qt: KDE Frameworks 5.18.0 Qt 5.5.1 (built against 5.5.1) The xcb windowing system
Ok, thanks for the infos. I did some more tests, and finally found out the issue, unrelated to Qt. The qtblend transition (like affine and frei0r.cairoblend) requests rgba frames to process. Most videos being in yuv format, this means a conversion has to be done (probably processed internally by ffmpeg), then the image has to be converted again in yuv format for display. Composite transition is a yuv transition which is why it is so much faster. This does not really explain why the conversion seems to run fast enough for the first 5 seconds on some systems and then the fps drops, but still this means that qtblend cannot be easily fixed. So for the 16.08 release, Vincent and me were thinking to do this: Introduce a config option to decide compositing mode (could be a combobox in timeline toolbar): * disabled * preview (using composite transition fast but with small luma bleed) * Hq (slower qtblend but best quality). We are also planning to remove the individual tracks "enable/disable" composite since they produce unexpected results, for example disabling a middle track compositing will make the upper clip disappear). Would that be ok for you?
Jean-Baptiste, Vincent, removing the composite controls is a great idea! I always wondered what the expected behavior should be when there are opaque tracks in between in the track stack. This simplifies the user interface. Do we need some backward compatibility with existing projects? I for one don't need it, as I always prefer explicit transitions for video compositing. Any chance of qtblend wipe support? Or would this result in just bloat and no performance increase over cairoblend?
Sounds like a good plan. :) We can always test in the fueld, and if there needs to be changes, we can re-work it as we go along. Thanks for looking into this, everyone.
Now with most recent git master MLT+Kdenlive I can play a single title over an MP4 on transparent tracks in full 25fps. However, using an explicit "Composite and Transform" (aka qtblend) transition shows the odd behavior of 25fps at the first few seconds, then drops to 18fps.
After some more testing, I decided it was easier to post a video with my recent thoughts -- it'd be a lot to write out, and videos make it really easy to see what's going on in the program. Terribly sorry, the mic volume was much lower than I realized; I hope turning up the volume will allow you to hear it. See video: https://youtu.be/JDxJhLbCjGc
Thanks Jesse for the feedback. For sure we all want to be able to composite 6 tracks in high quality without any drop in framerate. But for Kdenlive's 16.08 release I will not do any further changes, it's too late in the release cycle. But for sure there is definitely room for improvement.
No doubt! Definitely getting better and better with each release. :) Thanks for the dev' support and open collaboration JB.
Just to update on my latest testing. With "High Quality" compositing enabled, even having a single video clip in a single video track with nothing above or below it still produces a major drop in speed/framerate. Changing it to "Preview" brings it back to 24fps, solid. This was after testing the latest git master build from ppa:kdenlive/kdenlive-master, today.
Here's an updated issue that arose when testing with a .png file: it doesn't show the entire monitor when "Preview" mode is enabled. https://youtu.be/D6Z909fB3xo
I'd like to come to a group consensus on what it would look like to have this issue resolved, and maybe use this as a roadmap towards fixing this bug: (1) Have multiple clips in timeline stacked on top of each other and still play through timeline in real time without drop in framerate (this is, of course, in reference to a simple edit where the clips have no effects/transitions on them). (2) Remove the new track compositing dropdown box (as it would be unnecessary at that point, and I think its existence is going to be confusing to most users as it is. There's no visual different in quality between the three in the monitors.) Can we agree that this would be an appropriate resolution/goal to fix this bug?
Dont do #2, as this messes up some existing projects. It's necessary in certain projects to switch off any implicit/internally-added transitions. Also, I regularly switch off timeline compositing as I need close, explicit control over all compositing in the timeline. Also, there are rendering/performance differences.
@Wegwerf, definitely, I could see a small improvement in framerate playback when switching to Preview vs High Quality. But I didn't see any differences, visually, in playback. Maybe I need to If that's the case that #2 needs to stay, maybe we need to find a way to make the user more easily educated about its purpose & function, you know? The thumbnail images next to each option in the dropdown box suggests, to me, that it adjusts the quality of what's being seen on the monitor to improve real-time playback in the timeline vs. seeing what the final quality render output will look like. I realize that the tooltip text says "Track Compositing"... but I don't know if that gives it away on first read.
Affected by this too. Version 17.04.3.
It seems that this report is related to very old unmaintained version. A lot changed since then, especially the timeline got a complete rewrite and it is likely that this has been fixed. Please test with the latest version (https://kdenlive.org/en/download/)
Dear Bug Submitter, This bug has been in NEEDSINFO status with no change for at least 15 days. Please provide the requested information as soon as possible and set the bug status as REPORTED. Due to regular bug tracker maintenance, if the bug is still in NEEDSINFO status with no change in 30 days the bug will be closed as RESOLVED > WORKSFORME due to lack of needed information. For more information about our bug triaging procedures please read the wiki located here: https://community.kde.org/Guidelines_and_HOWTOs/Bug_triaging If you have already provided the requested information, please mark the bug as REPORTED so that the KDE team knows that the bug is ready to be confirmed. Thank you for helping us make KDE software even better for everyone!
This bug has been in NEEDSINFO status with no change for at least 30 days. The bug is now closed as RESOLVED > WORKSFORME due to lack of needed information. For more information about our bug triaging procedures please read the wiki located here: https://community.kde.org/Guidelines_and_HOWTOs/Bug_triaging Thank you for helping us make KDE software even better for everyone!