Bug 371062

Summary:	audio on some clips does not match playhead position - only plays later audio, never earlier
Product:	[Applications] kdenlive	Reporter:	S. Christian Collins <s_chriscollins>
Component:	Rendering & Export	Assignee:	Jean-Baptiste Mardelle <jb>
Status:	RESOLVED FIXED
Severity:	normal	CC:	aslanex, contatoedsonxavier, fritzibaby, sbc_hates_spam, snd.noise
Priority:	HI	Flags:	fritzibaby: Brainstorm+ fritzibaby: timeline_corruption+
Version:	Appimage - Refactoring
Target Milestone:	---
Platform:	Neon
OS:	Linux
Latest Commit:	https://invent.kde.org/multimedia/kdenlive/commit/faf778167d723de182bbbef79450d1f017820dc6	Version Fixed In:
Sentry Crash Report:
Attachments:	video showing the bug Short video segment to show sync problem with video and sound desk audio audio-video sync test

Description S. Christian Collins 2016-10-18 07:05:30 UTC

My screen capture software + codec combination creates video files where the video feed appears to start a bit later than the audio. If I drag one of these video clips to the timeline in Kdenlive and render the project, the resulting video file has no synchronization issues.

However, if I try to play the clip in Kdenlive, either through the clip monitor or on the timeline, the audio gets out of sync in a very strange way. Whenever I stop the project and restart playback, the audio jumps forward beyond the current playhead position. If I stop playback and then position the cursor at an earlier point, the audio playback will still resume from a later point. In fact, the audio is never able to go back to an earlier point of playback. If you keep playing, starting, stopping, rewinding, etc., eventually, the duration of the clip's audio will have been played, and any further attempts to play the clip will result in nothing but silence.

The previous version of Kdenlive that I was using (don't remember--it was pre KF5) did not have this problem, although I had to trim the beginning of the clip a few seconds once imported onto the timeline if I wanted the audio and video to be in sync. I edited many screencasts in that version of Kdenlive without issues.

Reproducible: Always

Steps to Reproduce:
1. Download the following video and import it into Kdenlive's project bin: https://dl.dropboxusercontent.com/u/8126161/audio-video-sync-test.avi
2. Play the clip either by A) playing it in the clip monitor, or B) dragging it onto a track and playing the project.
3. Occasionally stop playback and reposition the playhead (both earlier and later)

Actual Results:
The audio only plays forward, never returning to an earlier point in the audio stream.

Expected Results:
The audio should be synced with the video.

Kdenlive 16.08.2 w/ MLT 6.0.0
OS: KDE Neon 5.8.1 64-bit (Plasma Desktop 5.8.1, KDE Frameworks 5.27.0, Qt 5.7.0)

Comment 1 farid 2017-03-05 19:02:16 UTC

I have tried with MLT 6.4.1 and Kdenlive 16.12.2 and can't reproduce, could you upgrade both and test?

Comment 2 S. Christian Collins 2017-03-09 15:35:48 UTC

Created attachment 104472 [details]
video showing the bug

I can repro this 100% using Kdenlive 16.12.2 with MLT 6.5.0 (appimage). Video attached.

Comment 3 aslanex 2017-04-19 15:24:59 UTC

I am affected with this bug too, trying on 16.12.3. my opus music is out of sync when playing both in clip monitor and timeline, a working solution is to remux the file with .mkv


Technical details about the .opus file from avprobe:
Input #0, matroska,webm, from '...':
  Metadata:
    encoder         : google
  Duration: 00:03:02.00, start: 0.007000, bitrate: 129 kb/s
    Stream #0:0(eng): Audio: opus, 48000 Hz, stereo, fltp (default)

Avprobe of the same file remuxed with .mkv:
Input #0, matroska,webm, from '...':
  Metadata:
    ENCODER         : Lavf56.40.101
  Duration: 00:03:02.11, start: 0.000000, bitrate: 1764 kb/s
    Stream #0:0(eng): Audio: flac, 48000 Hz, stereo, s32 (24 bit) (default)
    Metadata:
      LANGUAGE        : eng
      ENCODER         : Lavc56.60.100 flac
      DURATION        : 00:03:02.113000000


A similar problem appears even when using a .mkv video rendered by kdenlive, it's avprobe:
Input #0, matroska,webm, from '...':
  Metadata:
    ENCODER         : Lavf56.40.101
  Duration: 00:00:11.15, start: 0.007000, bitrate: 4660 kb/s
    Stream #0:0: Video: h264 (High), yuv420p(tv, bt709/unknown/bt709), 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 1k tbn, 50 tbc (default)
    Metadata:
      DURATION        : 00:00:11,127000000
    Stream #0:1: Audio: opus, 48000 Hz, stereo, fltp (default)
    Metadata:
      DURATION        : 00:00:11,148000000

Remuxing, however, does not help in this case.

Comment 4 aslanex 2017-04-19 15:30:48 UTC

Ok, sorry, the second avprobe is wrong, that was produced by a Lossless Matroska transcode. This one is from Remux with MKV, still plays correctly:

Input #0, matroska,webm, from '...':
  Metadata:
    ENCODER         : Lavf56.40.101
  Duration: 00:03:02.02, start: 0.007000, bitrate: 129 kb/s
    Stream #0:0(eng): Audio: opus, 48000 Hz, stereo, fltp (default)
    Metadata:
      LANGUAGE        : eng
      DURATION        : 00:03:02.021000000

Comment 5 Samuel 2017-04-20 06:56:25 UTC

I have Kdenlive 16.12.3 on Kubuntu 17.04 with KDE plasma 5.9.4 and Frameworks 5.31.  The problem described below also occurred on Kubuntu 16.10 with the same version of Kdenlive.

When I load MTS camera files from the same camera, the audio timeline is delayed by about 80ms from the image of the speaker.  Using the camera audio with the image there is no problem with this waveform delay.

However, I need to load an audio file from the sound desk and create an mp3 audio file which gives better quality than that recorded by the camera.  The problem is that when I use the audio timeline waveform to sync with the camera waveform the sound is out of sync with the image and I have to advance it by at least 80ms.

This problem occurs randomly on MTS files.  I was originally thinking that it was a problem associated with copying files, but because I could reproduce this problem I now think it is with Kdenlive.

Comment 6 Samuel 2017-04-20 07:27:19 UTC

Created attachment 105108 [details]
Short video segment to show sync problem with video and sound desk audio

The audio timeline waveform from the mp3 sound desk file was visually aligned with the camera video timeline.

Comment 7 S. Christian Collins 2017-04-20 15:13:03 UTC

Samuel, your problem sounds like a different bug than this one. This bug isn't just about out-of-sync audio/video, but rather the case where it is impossible to play earlier bits of audio than whatever audio has been played, even when playing an earlier section of video. If this does not fit the description of what you are experiencing, then you should file your issue as a separate bug report.

Comment 8 emohr 2018-12-26 19:09:29 UTC

Hi Christian. The test file audio-video-sync-test.avi isn’t available anymore. 

As we have still audio sync issues it would be nice to test it. Could you upload the file again.

Can you test with the actual Refactoring Nigtly build: https://binary-factory.kde.org/job/Kdenlive_Nightly_Appimage_Build/lastSuccessfulBuild/artifact/

Comment 9 S. Christian Collins 2018-12-30 15:59:09 UTC

Created attachment 117188 [details]
audio-video sync test

Here is the "audio-video-sync-test.avi" referenced in the original bug report.

Comment 10 S. Christian Collins 2018-12-30 16:05:27 UTC

I am able to repro this bug in the latest Kdenlive nightly (kdenlive-19.03.70-b7d4afd-x86_64.appimage) on KDE neon 18.04 with Plasma 5.14.4.

Comment 11 emohr 2019-01-21 08:30:14 UTC

Task in GitLab: https://invent.kde.org/kde/kdenlive/issues/12

need some testing.

Comment 12 farid 2019-08-30 12:58:11 UTC

Please see here and test:

https://github.com/mltframework/mlt/issues/231#issuecomment-526588347

Comment 13 S. Christian Collins 2019-09-26 16:04:07 UTC

Adjusting the audio sync does not fix the bug as I reported it. My bug report isn't about a consistent amount of de-synchronization, but rather that when you play, e.g., at timestamp 2:00 in the attached video file and then move the playback cursor to 1:00, the audio continues to play after 2:00 while the video is at 1:00. Eventually, no audio will play back at all on that clip until restarting Kdenlive.

Comment 14 farid 2020-01-21 18:07:50 UTC

Hi again, are you still experiencing this? I cannot reproduce...

Comment 15 S. Christian Collins 2020-01-22 06:25:59 UTC

(In reply to farid from comment #14)
> Hi again, are you still experiencing this? I cannot reproduce...

I just tested with "kdenlive-20.03.70-84910e0-x86_64.appimage" playing the "audio-video-sync-test.avi" file in the clip monitor, and I can reproduce the bug exactly as I described in the original report.

Comment 16 farid 2021-03-26 12:15:17 UTC

Probably similar to this: BUG431936

Comment 17 Jean-Baptiste Mardelle 2021-04-08 20:38:59 UTC

Thanks for your detailed report. In fact, MLT cannot work reliably with this  file type. I am not exactly sure why, but it cannot be reliably seeked, and Shotcut, the other reference MLT based editor proposes to transcode the file for editing. I will have to implement something simlar

Comment 18 Jean-Baptiste Mardelle 2021-04-16 13:03:51 UTC

Git commit faf778167d723de182bbbef79450d1f017820dc6 by Jean-Baptiste Mardelle.
Committed on 16/04/2021 at 13:02.
Pushed by mardelle into branch 'master'.

Implement check and transcoding for non seekable files.

M  +7    -0    data/kdenlivetranscodingrc
M  +22   -0    src/bin/bin.cpp
M  +6    -0    src/bin/bin.h
M  +4    -0    src/bin/projectclip.cpp
M  +6    -0    src/jobs/loadjob.cpp
M  +17   -3    src/jobs/transcodeclipjob.cpp
M  +2    -1    src/jobs/transcodeclipjob.h
M  +1    -1    src/mainwindow.cpp
M  +1    -0    src/project/CMakeLists.txt
A  +67   -0    src/project/transcodeseek.cpp     [License: GPL]
A  +47   -0    src/project/transcodeseek.h     [License: GPL]
A  +101  -0    src/ui/transcodeseekable_ui.ui

https://invent.kde.org/multimedia/kdenlive/commit/faf778167d723de182bbbef79450d1f017820dc6