Bug 384764 - Proxy clips on time line play slower than original clips
Summary: Proxy clips on time line play slower than original clips
Status: RESOLVED FIXED
Alias: None
Product: kdenlive
Classification: Applications
Component: Video Display & Export (show other bugs)
Version: 19.12.1
Platform: Debian unstable Linux
: NOR normal
Target Milestone: ---
Assignee: Jean-Baptiste Mardelle
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-09-16 10:40 UTC by mikko.rapeli
Modified: 2021-03-27 15:22 UTC (History)
6 users (show)

See Also:
Latest Commit:
Version Fixed In:
fritzibaby: Brainstorm+


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description mikko.rapeli 2017-09-16 10:40:52 UTC
Proxy clips are still working in clip preview but seem to be performance wise broken on time line.

My proxy clips for gopro footage plays fast on a slow machine in clip preview and in&out points can be adjusted easily.

Copying the clip to timeline works but playback of the timeline slows down for these clips to 1 fps. If I deselect proxy clip feature for the clips, they play 10 fps on my slow i686 machine.

Something is broken on the timeline with proxy clips.
Comment 1 mikko.rapeli 2017-09-16 14:46:34 UTC
Now I could try to git bisect this issue and possibly work around by using an older version of kdenlive, BUT the project file from v17.08.1 does not work with 17.04.3.

I'm sorry, I will give up with kdnelive now.
Comment 2 Jean-Baptiste Mardelle 2017-09-19 06:46:25 UTC
Can you paste the proxy profile you were using ? The only change I can think of affecting proxies is a change in the size parameter allowing to make sure the proxy height is a multiple of 2. Also doing:
ffmpeg -i myproxy.mp4

on one of your proxies might give some clues.
Comment 3 mikko.rapeli 2017-09-19 19:10:31 UTC
(In reply to Jean-Baptiste Mardelle from comment #2)
> Can you paste the proxy profile you were using ? The only change I can think
> of affecting proxies is a change in the size parameter allowing to make sure
> the proxy height is a multiple of 2. Also doing:
> ffmpeg -i myproxy.mp4
> 
> on one of your proxies might give some clues.

It is just kdenlive. Playback on timeline is 10 times slower than in clip preview.

I switched to a bit beefier 64bit Core2 Duo machine with 8 gigs of RAM but it is not helping. I get maybe 5fps when previewing the 640x360 proxy clip on timeline. In clip preview it plays better but nowhere as good as with mplayer or melt from command line where I can not see any frame drops or complains.

I created a test project archive and uploaded to:

https://mcfrisk.kapsi.fi/temp/proxy_test01.tar.gz

Older versions of kdenlive were ok to use with this kind usage. Time line playback went slow only after multiple effect were applied to the clips, which I often did as the last step, also because they quite often crashed kdenlive and corrupted the timeline.

I did go through a large set of clips and split and set in/out points to them and now can't easily switch to an older kdenlive version. The project files should really be more backwards and forwards compatible! I should start archiving the full kdenlive debian packages with each of my edits...
Comment 4 mikko.rapeli 2017-09-19 19:17:16 UTC
On this machine with:

01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] RV635/M86 [Mobility Radeon HD 3650]

clip and timeline playback using libmovit7 is even slower.
Comment 5 mikko.rapeli 2017-09-19 19:44:08 UTC
One thing that I have changed lately is the project profile from 720p30 to 1080p25. That seems to have a major effect on playback performance even when proxy clips are 640x360p50. Hmm why are they p50? Shouldn't be needed really...

I was able to go back to 17.04.3-1 version but could not see any improvement there. 16.12.3-1 binaries from snapshot.debian.org no longer work on Debian unstable due to Qt dependencies.

What could work for me is to use really low project profile 640x360p15 even, but in the past changing profile was a destructive operation which screwed timeline completely when going back to 1080p25 for final rendering.
Comment 6 mikko.rapeli 2017-09-19 21:13:53 UTC
Playing with the test project. By only changing project profile from 1080p25 to 720p25 the proxy clip playback on timeline changes from roughly 7fps to 15 fps. Would be nice to have an fps counter on the preview screen btw.

That is to me odd. There should not be reasons for the preview rendering pipeline to change when project profile is changed, especially when rendering proxy clips.

For the final render yes, but not for time line preview.

Feels like that during proxy clip playback, the frames are first rendered from 640x360 to project resolution 1080p or 720p, and then scaled to the monitor preview window size. To me this intermediate step is not necessary.

The preview rendering should happen from 640x360 directly to the monitor preview window size for each frame.
Comment 7 mikko.rapeli 2017-09-21 20:37:36 UTC
Running with my old pet peeve where I remove the caching and frame dropping from kdenlive rendering, since on these old machines I get better performance that way:

diff --git a/src/renderer.cpp b/src/renderer.cpp
index e839d10b0..5b87caec9 100644
--- a/src/renderer.cpp
+++ b/src/renderer.cpp
@@ -754,9 +754,12 @@ void Render::switchPlay(bool play, double speed)
             m_mltProducer->seek(0);
         }
         if (m_mltConsumer->get_int("real_time") != m_qmlView->realTime()) {
-            m_mltConsumer->set("real_time", m_qmlView->realTime());
-            m_mltConsumer->set("buffer", 25);
-            m_mltConsumer->set("prefill", 1);
+            //m_mltConsumer->set("real_time", m_qmlView->realTime());
+            m_mltConsumer->set("buffer", 0);
+            m_mltConsumer->set("prefill", 0);
+            // try to avoid fps drop in preview:
+            m_mltConsumer->set("drop_max", 1);
+            m_mltConsumer->set("real_time", 0);
             // Changes to real_time require a consumer restart if running.
             if (!m_mltConsumer->is_stopped()) {
                 m_mltConsumer->stop();
@@ -771,7 +774,7 @@ void Render::switchPlay(bool play, double speed)
         }
         m_mltProducer->set_speed(speed);
     } else {
-        m_mltConsumer->set("real_time", -1);
+        m_mltConsumer->set("real_time", 0);
         m_mltConsumer->set("buffer", 0);
         m_mltConsumer->set("prefill", 0);
         m_mltProducer->set_speed(0.0);
@@ -794,9 +797,12 @@ void Render::play(double speed)
         resetZoneMode();
     }
     if (speed != 0 && m_mltConsumer->get_int("real_time") != m_qmlView->realTime()) {
-        m_mltConsumer->set("real_time", m_qmlView->realTime());
-        m_mltConsumer->set("buffer", 25);
-        m_mltConsumer->set("prefill", 1);
+        //m_mltConsumer->set("real_time", m_qmlView->realTime());
+        m_mltConsumer->set("buffer", 0);
+        m_mltConsumer->set("prefill", 0);
+        // try to avoid fps drop in preview:
+        m_mltConsumer->set("drop_max", 0);
+        m_mltConsumer->set("real_time", 0);
         // Changes to real_time require a consumer restart if running.
         if (!m_mltConsumer->is_stopped()) {
             m_mltConsumer->stop();
@@ -926,7 +932,7 @@ void Render::setDropFrames(bool drop)
             dropFrames = -dropFrames;
         }
         //m_mltConsumer->stop();
-        m_mltConsumer->set("real_time", dropFrames);
+        //m_mltConsumer->set("real_time", dropFrames);
         if (m_mltConsumer->start() == -1) {
             qCWarning(KDENLIVE_LOG) << "ERROR, Cannot start monitor";
         }


This is slightly better at around 10 fps in 1080p25 mode when previewing proxy clips, but there is a quite easily triggered crash where stack trace actually shows that the frame was rendered in 1080 format from proxy clip which is 640x360:

Thread 12 "FrameRenderer" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fe3be453700 (LWP 18154)]

#0  0x00007fe41cba1ee3 in ?? () from /usr/lib/x86_64-linux-gnu/libQt5Gui.so.5
No symbol table info available.
#1  0x00007fe41cbbf99e in ?? () from /usr/lib/x86_64-linux-gnu/libQt5Gui.so.5
No symbol table info available.
#2  0x00007fe41cbf9c26 in ?? () from /usr/lib/x86_64-linux-gnu/libQt5Gui.so.5
No symbol table info available.
#3  0x00007fe41cc004bf in QRasterPaintEngine::drawImage(QPointF const&, QImage const&) () from /usr/lib/x86_64-linux-gnu/libQt5Gui.so.5
No symbol table info available.
#4  0x00007fe41cc1d8af in QPainter::drawImage(QPointF const&, QImage const&) ()
   from /usr/lib/x86_64-linux-gnu/libQt5Gui.so.5
No symbol table info available.
#5  0x00007fe3d2f8a979 in QPainter::drawImage (flags=..., sh=-1, sw=-1, sy=0,
    sx=0, image=..., y=0, x=0, this=0x7fe3be452428)
    at /usr/include/x86_64-linux-gnu/qt5/QtGui/qpainter.h:858
        sy = 0
        sw = -1
        image = @0x7fe3be452460: <incomplete type>
        y = 0
        this = 0x7fe3be452428
        sx = 0
        sh = -1
        x = 0
#6  get_image (a_frame=a_frame@entry=0x7fe3a8014170,
    image=image@entry=0x7fe3be452650, format=format@entry=0x7fe3be45269c,
    width=width@entry=0x7fe3be452694, height=height@entry=0x7fe3be452698,
    writable=writable@entry=0) at transition_qtblend.cpp:190
        error = <optimized out>
        b_frame = <optimized out>
        b_properties = <optimized out>
        properties = 0x7fe3a8014170
        transition_properties = <optimized out>
        b_image = 0x7fe3907f9020 "13N\377\061\063N\377\061\063N\377\061\063N\377\061\063N\377\061\063N\377\061\063N\377\061\063N\377\061\063N\377\061\063N\377\061\063N\377\061\063N\377\061\063N\377\061\063N\377\061\063N\377\061\063N\377\061\063N\377\061\063N\377\061\063N\377\061\063N\377\061\063N\377\061\063N\377\061\063N\377\061\063N\377\063\066P\377\063\066P\377\063\066P\377\063\066P\377\063\066P\377\063\066P\377\063\066P\377\063\066P\377\063\066P\377\063\066P\377\063\066P\377\063\066P\377\063\066P\377\063\066P\377\063\066P\377\063\066P\377\063\066P\377\063\066P\377\063\066P\377\063\066P\377\063\066P\377\063\066Q\377\063\066Q\377\063\066Q\377\065\070T\377\065\070T\377"...
        hasAlpha = <optimized out>
        opacity = 1
        b_width = 1920
        b_height = 1080
        transform = {affine = {_m11 = 1, _m12 = 0, _m21 = 0, _m22 = 1,
            _dx = 0, _dy = 0}, m_13 = 0, m_23 = 0, m_33 = 1, m_type = 0,
          m_dirty = 0, d = 0x0}
        rect = {x = <optimized out>, y = <optimized out>, w = <optimized out>,
          h = -1, o = <optimized out>}
        length = <optimized out>
        position = <optimized out>
        profile = <optimized out>
        normalised_width = 1920
        normalised_height = 1080
        output_ar = <optimized out>
        interps = <optimized out>
        image_size = 0
        a_image = 0x7fe3867fd020 ""
        hqPainting = false
        bottomImg = <incomplete type>
        topImg = <incomplete type>
        painter = {static staticMetaObject = {d = {superdata = 0x0,
              stringdata = 0x7fe41cd6eca0, data = 0x7fe41cd6ebe0,
              static_metacall = 0x0, relatedMetaObjects = 0x0,
              extradata = 0x0}}, d_ptr = {d = 0x7fe3b000dde0}}
#7  0x00007fe426311c87 in mlt_frame_get_image (self=self@entry=0x7fe3a8014170,
    buffer=buffer@entry=0x7fe3be452650, format=format@entry=0x7fe3be45269c,
    width=width@entry=0x7fe3be452694, height=height@entry=0x7fe3be452698,
    writable=writable@entry=0) at mlt_frame.c:585
        properties = 0x7fe3a8014170
        get_image = 0x7fe3d2f8a5c0 <get_image(mlt_frame, uint8_t**, mlt_image_format*, int*, int*, int)>
        requested_format = <optimized out>
        error = 0
#8  0x00007fe426327cc9 in producer_get_image (self=self@entry=0x7fe3a8149cf0,
    buffer=buffer@entry=0x7fe3be452650, format=format@entry=0x7fe3be45269c,
    width=width@entry=0x7fe3be452694, height=height@entry=0x7fe3be452698,
    writable=writable@entry=0) at mlt_tractor.c:370
        data = 0x0
        size = 0
        properties = 0x7fe3a8149cf0
        frame = 0x7fe3a8014170
        frame_properties = 0x7fe3a8014170
#9  0x00007fe426311c87 in mlt_frame_get_image (self=0x7fe3a8149cf0,
    buffer=buffer@entry=0x7fe3be452650, format=format@entry=0x7fe3be45269c,
    width=width@entry=0x7fe3be452694, height=height@entry=0x7fe3be452698,
    writable=writable@entry=0) at mlt_frame.c:585
        properties = 0x7fe3a8149cf0
        get_image = 0x7fe426327b60 <producer_get_image>
        requested_format = <optimized out>
        error = 0
#10 0x00007fe4260f774b in Mlt::Frame::get_image (
    this=this@entry=0x7fe3be452700, format=@0x7fe3be45269c: mlt_image_rgb24a,
    format@entry=@0x7fe3be45269c: mlt_image_yuv420p, w=@0x7fe3be452694: 1920,
    w@entry=@0x7fe3be452694: 0, h=@0x7fe3be452698: 1080,
    h@entry=@0x7fe3be452698: 0, writable=writable@entry=0) at MltFrame.cpp:83
        image = 0x7fe3b000d7e0 "13N\377\061\063N\377\061\063N\377\061\063N\377\061\063N\377\061\063N\377\061\063N\377\061\063N\377\061\063N\377\061\063N\377\061\063N\377\061\063N\377\061\063N\377\061\063N\377\061\063N\377\061\063N\377\061\063N\377\061\063N\377\061\063N\377\061\063N\377\061\063N\377\061\063N\377\061\063N\377\061\063N\377\063\066P\377\063\066P\377\063\066P\377\063\066P\377\063\066P\377\063\066P\377\063\066P\377\063\066P\377\063\066P\377\063\066P\377\063\066P\377\063\066P\377\063\066P\377\063\066P\377\063\066P\377\063\066P\377\063\066P\377\063\066P\377\063\066P\377\063\066P\377\063\066P\377\063\066Q\377\063\066Q\377\063\066Q\377\065\070T\377\065\070T\377"...
#11 0x000055599248f095 in FrameRenderer::showFrame (
    this=this@entry=0x555995539e60, frame=...)
    at ./src/monitor/glwidget.cpp:1323
        width = 1920
        height = 1080
        format = mlt_image_rgb24a
#12 0x00005559925fe53c in FrameRenderer::qt_static_metacall (
    _o=0x555995539e60, _id=<optimized out>, _a=<optimized out>,
    _c=<optimized out>)
    at ./obj-x86_64-linux-gnu/src/kdenlive_autogen/T5U2TTTZLX/moc_glwidget.cpp:772
        _t = 0x555995539e60
#13 0x00007fe41c40f3d2 in QObject::event(QEvent*) ()
   from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
No symbol table info available.
#14 0x00007fe41fbcd46c in QApplicationPrivate::notify_helper(QObject*, QEvent*)
    () from /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5
No symbol table info available.
#15 0x00007fe41fbd4d34 in QApplication::notify(QObject*, QEvent*) ()
   from /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5
No symbol table info available.
#16 0x00007fe41c3dfd68 in QCoreApplication::notifyInternal2(QObject*, QEvent*)
    () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
No symbol table info available.
#17 0x00007fe41c3e24dd in QCoreApplicationPrivate::sendPostedEvents(QObject*, int, QThreadData*) () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
No symbol table info available.
#18 0x00007fe41c438dd3 in ?? () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
No symbol table info available.
#19 0x00007fe415594f67 in g_main_context_dispatch ()
   from /lib/x86_64-linux-gnu/libglib-2.0.so.0
No symbol table info available.
#20 0x00007fe4155951a0 in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
No symbol table info available.
#21 0x00007fe41559522c in g_main_context_iteration ()
   from /lib/x86_64-linux-gnu/libglib-2.0.so.0
No symbol table info available.
#22 0x00007fe41c4383ff in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
No symbol table info available.
#23 0x00007fe41c3dddba in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
No symbol table info available.
#24 0x00007fe41c1fd3ca in QThread::exec() ()
   from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
No symbol table info available.
#25 0x00007fe41c20229d in ?? () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
No symbol table info available.
#26 0x00007fe4187e1494 in start_thread (arg=0x7fe3be453700)
    at pthread_create.c:333
        __res = <optimized out>
        pd = 0x7fe3be453700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140616126510848, 
                780787169714677039, 0, 140733228324927, 0, 140617891983424, 
                -787402878124382929, -784231928223193809}, 
              mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, 
            data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
        pagesize_m1 = <optimized out>
        sp = <optimized out>
        freesize = <optimized out>
        __PRETTY_FUNCTION__ = "start_thread"
#27 0x00007fe41b607abf in clone ()
    at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97

So, would be really nice that for preview, the rendering resolution would be the render window size and not the full project profile which then needs to be downscaled to the small preview screen.

The scaling for proxy clips is now done twice: once by mlt to render 640x to 1080x and then by the viewer to downscale 1080x to what ever 300x is the size of preview.
Comment 8 mikko.rapeli 2017-09-21 20:40:25 UTC
And that's why proxy clips are slower to play than originals, though the different code does impact too. h264 original is tougher to decode than mpeg2video proxy.
Comment 9 José JORGE 2018-04-04 18:59:29 UTC
(In reply to mikko.rapeli from comment #8)
> And that's why proxy clips are slower to play than originals, though the
> different code does impact too. h264 original is tougher to decode than
> mpeg2video proxy.

I've just upgraded my Kdenlive from 16.x to 17.12.2 and I can see the same bug : the proxy clip is used when playing the single clip, but not used anymore when the clip is in the timeline.
Comment 10 emohr 2018-12-29 12:47:13 UTC
Is this something similar to Bug 385842 ? 
And do we have a profile issue as well? 
Both issues I can reproduce on the Refactoring version.
Comment 11 krewetki 2019-05-11 21:22:18 UTC
Hi,

I'm not sure what I'm doing with settings/status of this bug, so please forgive mistakes.

I'm changing status to CONFIRMED because it happens to me: Manjaro Stable, Kdenlive 19.04.0, MLT 6.14.0-5.
Comment 12 CzAndrew 2020-01-24 23:50:15 UTC
I confirm also that proxy clips doesn't work in the newest kdenlive (19.12.1; win10).
The proxy clip play smooth in the Clip monitor, but when I add it to Project monitor, the playback speed is reduced to ~2 fps. I see that the proxy clip is being played both in the project and clip monitor (image resolution reduced), but the project monitor plays it very choppy.
I tried to convert the proxy clips with different codecs, results are the same.
Does anyone know a workaround for this?
Comment 13 emohr 2020-01-25 13:35:58 UTC
In the meantime MLT and Kdenlive have implemented timeline playing with scaled clips. This is speeding up the timeline. 

Get the nightly build:
Windows: https://binary-factory.kde.org/job/Kdenlive_Nightly_mingw64/lastSuccessfulBuild/artifact/

Linux: https://binary-factory.kde.org/job/Kdenlive_Nightly_Appimage_Build/lastSuccessfulBuild/artifact/ 


Goto Monitor -> Preview resolution -> choose either 720p or 540p. Does this help?
Comment 14 CzAndrew 2020-01-26 12:20:18 UTC
I downloaded the newest nightly, and started playing with the settings of the proxy clips. In the nightly the proxy clips are working well, and since then the proxy clips are working in my 19.12.1 also. I can' recreate the problem, later I'll try it on another computer with a clean kdenlive.
There were some settings which slowed down the playback on the timeline (I mean that the playback wasn't perfectly somooth):
- rendering the proxy clips to x264 was better: renders faster, smaller proxy clip filesize and smoother playback on the timeline
- reducing the playback resolution of the 540p proxy clips to 540p was helped
- if the aspect ratio of the project and the source video clip wasn't the same, then there were more framedrops in the playback (but I guess it is nothing to do with proxy clips)

I'll try the proxy clips on another machine, and see if I can recreate the problem, and find the solution.
Comment 15 farid 2021-03-27 15:22:49 UTC
I would say this is fixed but if you still encounter issues please reopen. :)

(In reply to CzAndrew from comment #14)
> I downloaded the newest nightly, and started playing with the settings of
> the proxy clips. In the nightly the proxy clips are working well, and since
> then the proxy clips are working in my 19.12.1 also. I can' recreate the
> problem, later I'll try it on another computer with a clean kdenlive.
> There were some settings which slowed down the playback on the timeline (I
> mean that the playback wasn't perfectly somooth):
> - rendering the proxy clips to x264 was better: renders faster, smaller
> proxy clip filesize and smoother playback on the timeline

although x264 produces smaller clips it needs more processing power. I'd recommend mjpeg or if you have space maybe dnxhd or prores...

> - reducing the playback resolution of the 540p proxy clips to 540p was helped
> - if the aspect ratio of the project and the source video clip wasn't the
> same, then there were more framedrops in the playback (but I guess it is
> nothing to do with proxy clips)

definitely the smaller the proxies the better the playback.
> 
> I'll try the proxy clips on another machine, and see if I can recreate the
> problem, and find the solution.

please let us know and reopen if you encounter issues.