Bug 354835 - Git Master 2015-11-04 - Would like to have more than 3 concurrent processing threads.
Summary: Git Master 2015-11-04 - Would like to have more than 3 concurrent processing ...
Status: RESOLVED FIXED
Alias: None
Product: kdenlive
Classification: Applications
Component: User Interface (show other bugs)
Version: unspecified
Platform: Arch Linux Linux
: NOR normal
Target Milestone: ---
Assignee: Jean-Baptiste Mardelle
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-11-04 13:55 UTC by Evert Vorster
Modified: 2016-02-03 11:40 UTC (History)
3 users (show)

See Also:
Latest Commit:
Version Fixed In:
Sentry Crash Report:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Evert Vorster 2015-11-04 13:55:50 UTC
Hi there. 

I have to apply defish and rotate and so on on quite a bit of footage. 
The CPU I have has 8 cores, and it's fairly common. I recently worked on a machine that has 64 cores. 

So, the limitation of only 3 concurrent threads is a bit sucky. 
I have tried just typing in 8 in the box, but it simply does not take any integer higher than 3. 

Would this be an easy fix?

Reproducible: Always

Steps to Reproduce:
1. Select Configure - Environment - Processing threads
2. Try to increase the amount of threads to match the actual amount of cores in the machine

Actual Results:  
The apparent maximum number of concurrent threads is 3. 

Expected Results:  
At least as many concurrent threads as is installed on the computer.
Comment 1 Peter 2015-11-05 14:58:56 UTC
I produce scripts with the generate scripts thing and just put a larger number in the script (change real_time=-3 to real_time=-8 or whatever it's called), this works fine and does make the render go faster, so it would be great to be able to push it higher in kdenlive itself.
Comment 2 Peter 2015-11-05 17:57:59 UTC
Do bear in mind though, that the mlt processing threads is not that same as encoding threads in general. If you're encoding an h264 movie with one mlt thread, it'll still use all your cores to encode the movie.

See this for more -> http://www.mltframework.org/bin/view/MLT/Questions#Does_MLT_take_advantage_of_multi
Comment 3 Jean-Baptiste Mardelle 2015-11-06 16:35:29 UTC
Git commit 579bee161e588c259731a0564ea4fd5d7b0e22e8 by Jean-Baptiste Mardelle.
Committed on 06/11/2015 at 16:34.
Pushed by mardelle into branch 'master'.

Allow more than 3 concurrent threads, however not sure about stability, feedback welcome

M  +2    -1    src/dialogs/kdenlivesettingsdialog.cpp
M  +3    -0    src/mainwindow.cpp
M  +8    -1    src/monitor/glwidget.cpp
M  +1    -0    src/monitor/glwidget.h
M  +3    -3    src/renderer.cpp

http://commits.kde.org/kdenlive/579bee161e588c259731a0564ea4fd5d7b0e22e8
Comment 4 Wegwerf 2015-11-07 18:27:07 UTC
I had disappointing effects with MLT threads > 1 within Kdenlive: playhead position did not work anymore.
Comment 5 Wegwerf 2015-11-07 18:29:02 UTC
See also https://bugs.kde.org/show_bug.cgi?id=352489
Comment 6 Evert Vorster 2015-11-16 08:58:21 UTC
Sadly, having multiple processing threads also messes up video encoding. ( I was planning on starting up kdenlive in one core mode, then switching to multiple processing threads for rendering, and then switching back to single core because of the UI instabilities)

I get various irregularities, like the entire effects stack not being applied, a corrupted first frame in some tests.... In short, the processing threads seem to divvy up the effects stack between themselves, and some effects take longer to apply than others, hence messing up the order in which they are applied, if at all. 

For rendering purposes, it would be smarter to assign a CPU to a frame, and that CPU renders the whole stack on to the frame, and the next CPU tackles the next frame, and so on for the number specified of threads, and just wait until the frame is done before passing it to the encoder. 

As for the UI, I have noticed some instablities there, too. 
In summary, the way this feature is implemented needs to be re-thunk. 
There needs to be processing threads on render, and one for the UI. 
There is already a processing threads on the encoder, and that works beautifuly. 

So, melt also mis-handles multiple processing threads, as far as processing goes.
Comment 7 qubodup 2015-12-01 20:00:55 UTC
On v15.11.80-53-g0918054, with am amd phenom x4 9550 quadcore CPU I am able to select up to 4 threads for rendering.

I would like to suggest that the limit is removed or set to twice the amount of available CPU cores.

This is based on my experience (because unfortunately http://stackoverflow.com/questions/481970/how-many-threads-is-too-many doesn't have a mathematical function as an answer):

In my experience (in previous versions and when using script export and editing the script), I can use 7 threads with slight speed bumps when trying to use the computer for other things from time to time. 8 threads make it more noticeable but still ok and I tend to use 8 for rendering while I'm away from the computer.