Bug 368962 - Git master:2016-09-17:Changing project FPS corrupts clip segments on timeline
Summary: Git master:2016-09-17:Changing project FPS corrupts clip segments on timeline
Status: RESOLVED FIXED
Alias: None
Product: kdenlive
Classification: Applications
Component: User Interface (show other bugs)
Version: unspecified
Platform: Compiled Sources Linux
: NOR normal
Target Milestone: ---
Assignee: Jean-Baptiste Mardelle
URL:
Keywords:
: 350948 389859 (view as bug list)
Depends on:
Blocks:
 
Reported: 2016-09-17 16:04 UTC by Evert Vorster
Modified: 2019-07-30 05:13 UTC (History)
5 users (show)

See Also:
Latest Commit:
Version Fixed In:
evorster: Brainstorm+
evorster: low_hanging+
fritzibaby: timeline_corruption+


Attachments
Screen capture showing the issue (1.90 MB, video/quicktime)
2016-09-17 16:07 UTC, Evert Vorster
Details
Screencapture highlighting one subset of the issue (794.68 KB, video/quicktime)
2016-11-28 13:10 UTC, Evert Vorster
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Evert Vorster 2016-09-17 16:04:15 UTC
This is totally related to bug 357780 - In fact, it is just a subset of the issue. I saw someone get a bug fixed on project FPS change, and thought I would start small. 

Sooo, what's needed to see the issue quite clearly is a special clip.
Start a 25fps project, and make a generator that counts the frames. Use about 3000 frames to make the issue painfully obvious. 

Render this generator to get a clip at 25fps. 

Import that rendered clip into the project, and select a section from the clip, say from frame 1400 to frame 2000. 

Drag this segment into the timeline. Dragging your cursor over the timeline shows the frames 1400 - 2000 in the clip. 

Now, change the project framerate to 50fps. 

When dragging your cursor over the segment on the timeline, frames 700 - 1000 are now displayed, instead of frames 1400 - 2000, with each frame duplicated. 

Conceptually, if you select a segment from a clip, you are interested in something that happens in that range of frames. When changing the project's framerate, the range of frames should be preserved. 

The simplest formula for this is from Orig_Start/Orig_Fps*Dest_fps, rounded up to the nearest integer, until Orig_Stop/Orig_Fps*Dest_Fps, also rounded up to the nearest integer. 

There might be a frame or two extra or taken away from the clip on the timeline, however this is much better than displaying the completely wrong segment of the source video. 

Until this is fixed, changing the framerate on a project completely corrupts all the clips on the timeline. 

Reproducible: Always
Comment 1 Evert Vorster 2016-09-17 16:07:23 UTC
Created attachment 101147 [details]
Screen capture showing the issue

Ignore the first bit of the capture, I changed the project's framerate back to 25fps, and it seemed to have confused the display a little until I re-read the source clip, and put it on the timeline again. 

-Evert-
Comment 2 Evert Vorster 2016-09-17 16:17:04 UTC
Oh, I might add that changing the framerate of a project re-reads the source clips, but at the new framerate, so the segment on the timeline needs to have the frame numbers updated to keep the duration and part of the video displayed correct. 

Kind regards,
-Evert-
Comment 3 farid 2016-10-05 03:27:04 UTC
I cannot currently test this but I hope that it could be addressed. Anywhere I see the word corruption I get shivers down my spine.
Comment 4 Evert Vorster 2016-10-05 04:49:26 UTC
Hi Farid.

It's fairly easy to test... 

Take a clip that has some action in it... say a dog catching a ball. Import it into the clip monitor, select the bit where the action happens, in this case, the catching of the ball. 

Drag that to the timeline. You should then clearly see the dog catching the ball. 

Now, go into the project settings and double or halve the framerate. 

When you play the timeline again, the dog is now doing something else, as the frame range has shifted. 

I have a screen capture attached to this ticket showing the issue.
Comment 5 Evert Vorster 2016-11-28 13:10:43 UTC
Created attachment 102497 [details]
Screencapture highlighting one subset of the issue

Hi there. This screencapture was on the latest kdenlive, clearly showing the issue. 
To reproduce:
1. Create 10 second counter, and render out to a clip.
2. Load rendered clip in 25fps project.
3. Select Second 8 in clip preview, and drag to timeline. 
4. change project fps to 50fps

Clip on timeline now shows the second half of second 4, not the whole of second 8 like before 

This is because the number of frames from the source clip is updated when the project's frames per second changes, but the frame numbers on the timeline is not updated to match. 

In this case, frames 200 - 225 were selected on the source clip in the 25fps project. 

When changing the project's framerate, the clip on the timeline should now be displaying frames 400 - 500 to display the proper part of the clip as selected. 


When changing the framerate down from a high framerate project, total corruption of the clips on the timeline may result. This is because the source clip at the new framerate may not have the frames as were specified in the higher framerate project. 
In this case, if I started off with a 50fps project, and loaded that second 8 of the source clip, I would be looking at frames 400 - 500. If I then drop the project framerate, the clip is completely corrupted, because it only goes to frame 300...

I really hope that this bug can be sorted out, it makes changing framerates on existing projects completely unusable.
Comment 6 Evert Vorster 2016-11-28 13:13:48 UTC
Can we please consider this bug for the bug hunting day?
-Evert-
Comment 7 Evert Vorster 2018-03-26 08:26:50 UTC
*** Bug 350948 has been marked as a duplicate of this bug. ***
Comment 8 Evert Vorster 2018-03-26 08:56:21 UTC
This bug is still continuing to bite people, and is also present in the timeline refactoring branch. 

I assigned an even older recently re-opened bug from 2015 as a duplicate of this one as it got re-opened with a report that more closely resembles this bug from 2016.

I can't code, but I have been studying this bug for a while and can aid any programmer that is willing to try with testing all the corner cases of a fix.

In order to sort this bug, the following needs to happen:

When selecting a change in project framerate, each source clip has to be re-read to get the new frame numbers correct. This already happens, and is good. 

However, the next thing that needs to happen is that every referenced frame on all the clips needs to be updated.
This does not happen. 

In short, say a clip has an in point of 100 and out of 200, and the user now doubles the frame rate, then the in point needs to be 200 and the out point needs to be 400, to reflect the new frame rate. 

Every frame that is referenced on the timeline also needs to be updated to reflect the new frame number at the changed frame rate.
This also does not happen.

Since the frame numbers are integer, there is some loss of fidelity when lowering the frame rate, this is unavoidable, and expected. 

There is also the case of frames being adjacent on the timeline that needs to stay adjacent,for instance, when a clip ends on one frame, and the next clip starts on the next frame.

Conceivably, with integer math the two frames can end up with the same number when dropping the frame rate, or be separated when the frame rate increases. This would seriously corrupt the project.
We might just have to check for adjacent frames, and keep then adjacent by simply assigning the starting frame of the new clip to be adjacent to the old rather than doing the frame rate change math for that frame. This "feels" a little hackish, however since videos are forced to be integer frames there is no elegant way to do this. 



The other option, of course, is to not allow a project's frame rate to be changed at all. When a user starts a new project, they have to input the frame rate and not have the option to change it. They can render at a different frame rate, of course, but that would be the same as using ffmpeg to resample the output file.
Comment 9 farid 2018-03-26 12:49:45 UTC
(In reply to Evert Vorster from comment #8)
> This bug is still continuing to bite people, and is also present in the
> timeline refactoring branch. 
> 
> I assigned an even older recently re-opened bug from 2015 as a duplicate of
> this one as it got re-opened with a report that more closely resembles this
> bug from 2016.
> 
> I can't code, but I have been studying this bug for a while and can aid any
> programmer that is willing to try with testing all the corner cases of a fix.
> 
> In order to sort this bug, the following needs to happen:
> 
> When selecting a change in project framerate, each source clip has to be
> re-read to get the new frame numbers correct. This already happens, and is
> good. 
> 
> However, the next thing that needs to happen is that every referenced frame
> on all the clips needs to be updated.
> This does not happen. 
> 
> In short, say a clip has an in point of 100 and out of 200, and the user now
> doubles the frame rate, then the in point needs to be 200 and the out point
> needs to be 400, to reflect the new frame rate. 
> 
> Every frame that is referenced on the timeline also needs to be updated to
> reflect the new frame number at the changed frame rate.
> This also does not happen.
> 
> Since the frame numbers are integer, there is some loss of fidelity when
> lowering the frame rate, this is unavoidable, and expected. 
> 
> There is also the case of frames being adjacent on the timeline that needs
> to stay adjacent,for instance, when a clip ends on one frame, and the next
> clip starts on the next frame.
> 
> Conceivably, with integer math the two frames can end up with the same
> number when dropping the frame rate, or be separated when the frame rate
> increases. This would seriously corrupt the project.
> We might just have to check for adjacent frames, and keep then adjacent by
> simply assigning the starting frame of the new clip to be adjacent to the
> old rather than doing the frame rate change math for that frame. This
> "feels" a little hackish, however since videos are forced to be integer
> frames there is no elegant way to do this. 
> 
> 
> 
> The other option, of course, is to not allow a project's frame rate to be
> changed at all. When a user starts a new project, they have to input the
> frame rate and not have the option to change it. They can render at a
> different frame rate, of course, but that would be the same as using ffmpeg
> to resample the output file.

Evert, thanks for your input! You commented that this is happening on the refactoring branch, is that happening in the latest beta release? We are handling all the refactoring problems in this thread:

https://phabricator.kde.org/T7082 

Would you mind commenting on this issue there and a dev will definitely engage you on it.

Cheers
Comment 10 Evert Vorster 2018-10-09 04:28:18 UTC
In short, this bug makes the fps change feature almost unusable. 

The solution is already described, I just don't know C and so can't fix it myself. 

Kind regards,
-Evert-
Comment 11 Bart 2018-10-28 13:41:42 UTC
It would be great to add a warning, while this issue hasn't been fixed, like "changing the frame rate may mess up your existing editing on this project, are you sure you wish to continue?".
I just lost a few hours of work because I started a project with a profile at 25fps, edited a big part of it, then switched to a new profile at 30fps, continued editing, only to notice at the end that all editing done before the profile switch was messed up.
Comment 12 Evert Vorster 2018-10-29 13:20:54 UTC
Hi there, Bart. 

There used to be a warning going with the frame rate changes. 

It is important to understand that there will always be _some_ artifacts associated with changing framerates as each frame represents a discrete point in time that do not necessarily coincide between various framerates, the methods described in this report will keep them to a minimum. 

Have a look at Slowmovideo project to see what is needed for perfect frame rate changes. 

Kind regards,
-Evert-
Comment 13 emohr 2018-11-10 13:56:10 UTC
*** Bug 389859 has been marked as a duplicate of this bug. ***
Comment 14 Evert Vorster 2019-07-30 05:13:12 UTC
Fixed! Thanks, JBM...