Bug 472983 - Audio Scratching when scrubbing through Animation Timeline with MP3 files
Summary: Audio Scratching when scrubbing through Animation Timeline with MP3 files
Status: CONFIRMED
Alias: None
Product: krita
Classification: Applications
Component: Animation (show other bugs)
Version: unspecified
Platform: Microsoft Windows Microsoft Windows
: NOR minor
Target Milestone: ---
Assignee: Krita Bugs
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2023-08-04 02:29 UTC by Clyde
Modified: 2024-08-01 12:12 UTC (History)
2 users (show)

See Also:
Latest Commit:
Version Fixed In:
Sentry Crash Report:


Attachments
Here you can hear the scratch when scrubbing, Does happen in other Animation Timelines. (819.11 KB, video/mp4)
2023-08-04 02:32 UTC, Clyde
Details
attachment-2665791-0.html (2.34 KB, text/html)
2024-01-26 05:27 UTC, Clyde
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Clyde 2023-08-04 02:29:07 UTC
SUMMARY
***
NOTE: If you are reporting a crash, please try to attach a backtrace with debug symbols.
See https://community.kde.org/Guidelines_and_HOWTOs/Debugging/How_to_create_useful_crash_reports
***


STEPS TO REPRODUCE
1. Open Animation Timeline in the Docker
2. Import Sound To Animation Timeline
3. Scrub Audio Back and Forth and you'll hear the squeak like noise that plays. 

OBSERVED RESULT
The Animation Scrubbing Works Now Thankfully, But It makes a slight noticeable squeak like noise when scrubbing through frames. Sounds like a DJ scratch noise

EXPECTED RESULT
Should be able to scrub with no such noise happening.

SOFTWARE/OS VERSIONS
Windows: Version: 5.2.0-beta1 / Build ABI: x86_64-little_endian-llp64
  Build CPU: x86_64
  CPU: x86_64
  Kernel Type: winnt
  Kernel Version: 10.0.22621
  Pretty Productname: Windows 10 Version 2009
  Product Type: windows
  Product Version: 10
macOS: 
Linux/KDE Plasma: 
(available in About System)
KDE Plasma Version: 
KDE Frameworks Version: 
Qt Version: 

ADDITIONAL INFORMATION
Comment 1 Clyde 2023-08-04 02:32:14 UTC
Created attachment 160722 [details]
Here you can hear the scratch when scrubbing, Does happen in other Animation Timelines.
Comment 2 Emmet O'Neill 2024-01-26 02:00:27 UTC
Hey Clyde, thanks for the report. 
I can confirm this behavior. :)

I think this happens mainly because of sudden jumps ("discontinuities") in the audio waveform when we jump between different frames.
Something like what's being described here: https://dsp.stackexchange.com/questions/22860/stitching-together-audio-buffers-with-potential-discontinuity-at-the-boundary

I have a few different ideas for how we *might* be able to mitigate these annoying pops and scratches (like starting audio chunks only at zero crossings or applying a simple fade in/out amplitude envelope), but it's a relatively low priority task for now.
Comment 3 Clyde 2024-01-26 05:27:10 UTC
Created attachment 165230 [details]
attachment-2665791-0.html

I actually found a fix awhile back! MP3 Files are the only files to do this.
People will have to switch to a WAV or Ogg Audio format for it to go away.

On Thu, Jan 25, 2024 at 9:00 PM Emmet O'Neill <bugzilla_noreply@kde.org>
wrote:

> https://bugs.kde.org/show_bug.cgi?id=472983
>
> Emmet O'Neill <emmetoneill.pdx@gmail.com> changed:
>
>            What    |Removed                     |Added
>
> ----------------------------------------------------------------------------
>                  CC|                            |emmetoneill.pdx@gmail.com
>      Ever confirmed|0                           |1
>              Status|REPORTED                    |CONFIRMED
>
> --- Comment #2 from Emmet O'Neill <emmetoneill.pdx@gmail.com> ---
> Hey Clyde, thanks for the report.
> I can confirm this behavior. :)
>
> I think this happens mainly because of sudden jumps ("discontinuities") in
> the
> audio waveform when we jump between different frames.
> Something like what's being described here:
>
> https://dsp.stackexchange.com/questions/22860/stitching-together-audio-buffers-with-potential-discontinuity-at-the-boundary
>
> I have a few different ideas for how we *might* be able to mitigate these
> annoying pops and scratches (like starting audio chunks only at zero
> crossings
> or applying a simple fade in/out amplitude envelope), but it's a
> relatively low
> priority task for now.
>
> --
> You are receiving this mail because:
> You reported the bug.
Comment 4 Dmitry Kazakov 2024-08-01 11:55:09 UTC
Just a few notes about the bug:

It seems like the bug happen in the process of decoding MP3 file with avformat library. We might need to update avformat, which might resolve the issue. If it doesn't, the following steps could get us to the solution:

1) Try to dump per-frame audio data in MLT library (in consumer_sdl2_audio.c:consumer_play_audio). 
2) Dump it for two cases: 
    1) normal playback
    2) playback in scrubbing
3) If the buffer data is different for the same frame in two modes, then it confirms that the issue in avformat and/or in a way how MLT uses it to decode the partial MP3 data.
Comment 5 Dmitry Kazakov 2024-08-01 12:12:51 UTC
I can confirm that the problem in a way how MP3 file is decoded. Every time an MP3 frame is decoded, it is decoded differently (i.e. with different PCM data), even when played in a stream.

It doesn't happen with WAV or OGG files. The frame is decoded with exactly the same data, whatever play mode is used.