Bug 353066 - WISHLIST - Add keyframeable speed effect with intact audio (like iPhone slow-motion)
Summary: WISHLIST - Add keyframeable speed effect with intact audio (like iPhone slow-...
Status: RESOLVED FIXED
Alias: None
Product: kdenlive
Classification: Applications
Component: Effects & Transitions (show other bugs)
Version: unspecified
Platform: Compiled Sources Linux
: NOR wishlist
Target Milestone: ---
Assignee: Vincent PINON
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-09-23 00:20 UTC by Paul Konecny
Modified: 2021-12-06 21:12 UTC (History)
5 users (show)

See Also:
Latest Commit:
Version Fixed In:
fritzibaby: Backport+


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Paul Konecny 2015-09-23 00:20:53 UTC
Hi guys, 
it would be great to have an easy way to handle high framerate footage easily and correctly. 
What I want to accomplish:
1. Record in higher framerate than the timeline (i.e.: 60fps or 120fps for a production timeline of 30fps)
2. Add the clip to the 30fps timeline.
3. This clip should behave like a normal clip. With normal Audio and normal playback speed.
4. Add keyframeable speed effect to smoothly decelerate the playback to the desired value. 
5. Adjust audio accordingly so it sounds like "slowing-down". If I don't want any sound on that clip I can still throw a "mute" effect at it. 

What it does right now:
Ad 2:
Right now if I add a 50fps gopro clip to a 25fps timeline, the video plays at 25 fps as expected but there is no sound. Reducing the speed to 50% gives me a smooth slow motion as expected but also seems to eliminate even the audio thumbnails. As a start, the speed effect should not eliminate the audio on a respective clip.
Ad 3:
I believe this can be accomplished by reducing the sound speed while increasing the pitch at the same time. This should be the default behaviour of the timeline when a non-native fps clip gets added.
Ad 4: The speed effect is not keyframeable. The new speed effect should be
1. Keyframeable
2. Adjust speed and pitch of the audio accordingly (Slower: reduce speed - increase pitch; Faster: Increase Speed - reduce pitch)
3. Have an option to leave the pitch untouched so you can get higher and lower pitched voices accordingly.
Ad 5: Audio is not present when adding a clip to a non-native speed timeline and using the speed effect on any clip disables the audio on it.

I believe this to be a pretty cool feature and I hope we will see it someday in kdenlive!
Cheers!
Comment 1 Wegwerf 2016-08-05 18:01:59 UTC
Paul, audio support has finally landed in stable 16.04.2 if I remember correctly, or at least in current beta.

Keyframing the speed effect is not really possible at this time, the reason is how the clip length is to be calculated? This would need integration over the whole clip from the beginning in order to find out where a particular frame for a given position in the timeline is. Do you have any idea as to how to solve this?

Paul, would you consider the fixed audio support to be sufficient, and in that case close this bug as resolved fixed? Unfortunatately, I don't see a any clear way in how the keyframing actually could work.
Comment 2 Paul Konecny 2016-10-06 08:34:12 UTC
Sorry I got back to you so late. As I said I was burried in my thesis. 

I think that the speed effect as it is now is a great improvement and I consider the most part of my initial bug report fixed. 
The only thing that still bugs me it the keyframeability of the effect. 
I have an idea how to deal with it but I don't know whether I should open a new whishlist or post it here. 
(The idea would be to keep the clip at a constant length and stretch or compress the part with different speed)
See the gui on this page. 
http://www.cultofmac.com/408250/how-to-speed-up-slo-mo-videos-on-your-iphone/

Cheers!
Comment 3 Jens 2021-05-12 08:44:57 UTC
I was just about to report quite the same wishlist item.

Use case:
I have 5 videos of people singing the same song. I want to mix these to create a chorus. But these people are just *so* slightly

a) out of time (sometimes faster, sometimes slower, one makes dramatic pauses where they don't belong),
b) out of pitch, actually singing in different keys, and
c) generally not singing at the same speed.


Right now, I have to cut the tracks into multiple pieces and play these at different pitch-compensated speeds. This is a lot of manual work, especially because Kdenlive (current, 21.07.20, nightly Flatbak build) does not like changing speeds in the same track, and sometimes loses video. Different bug.) Also the sound sometimes stutters at the cut positions.

I would LOVE to create a keyframeable "speed curve" over each track, which is optionally pitch compensated, so that I can slow down the track in some parts, and speed it up in some others - and even stop it completely (still frame) when one singer misses a pause and continues singing.

This should happen independantly of what original FPS the tracks were recorded at. "100% speed" is always set to the FPS which the track originally had.

Do you think this is feasible? I'm willing to discuss and help brainstorm this, since I think it's a major feature.

Thanks!
Comment 4 emohr 2021-05-12 15:44:09 UTC
We will implement keyframeable speed with MLT-7. This is on the to do list.
Comment 5 Jens 2021-05-13 15:29:40 UTC
That's great to hear.

If I may add one more wish into the same topic:
Please also add a quantize option.

Imagine somebody playing an instrument or singing, but sometimes too fast, sometimes too slow. I can see the beats in the audio timeline and mark them and I want Kdenlive to equalize the time between these markers (eg. as "500ms" or "80 bpm").

This would be doable with a keyframable speed setting, but very cumbersome.
A quantizer effect for audio would be awesome. Even more awesome if you could automatically snap events (like clip transitions or slideshow images) to these markers.

Does this sound feasible in the Kdenlive infrastructure?
Comment 6 deresiant 2021-12-06 03:09:21 UTC
Is thus just for keyframinf speed or will it be smooth transitioning between speeds when keyframing
Comment 7 emohr 2021-12-06 15:07:10 UTC
Time-remap is implemented since version 21.08.0 with smooth transitions between keyframes. 
Details see here: https://docs.kdenlive.org/en/effects_and_compositions/effects.html#time-remapping-speed-ramps
Comment 8 Jens 2021-12-06 21:12:10 UTC
That's perfect, thank you! :-)