Bug 487031

Summary: Unable to play h264 videos on apple silicon mac
Product: [Applications] digikam Reporter: brancaleone <cedric.devillers>
Component: Preview-VideoAssignee: Digikam Developers <digikam-bugs-null>
Status: RESOLVED FIXED    
Severity: normal CC: caulier.gilles, metzpinguin
Priority: NOR    
Version: 8.3.0   
Target Milestone: ---   
Platform: macOS (DMG)   
OS: macOS   
Latest Commit: Version Fixed In: 8.5.0
Sentry Crash Report:
Attachments: sample file from ffmpeg bug report

Description brancaleone 2024-05-14 20:28:18 UTC
SUMMARY

When trying to display h264 videos from my digital camera (lumix gx9) nothing is shown in preview. Metadata are correctly reported. Same video where displayed correctly on previous versions.

When enabling logging this message is displayed continuously on the console : 

FFmpeg: [h264 @ 0x7f8f05827000] hardware accelerator failed to decode picture

using null output device, none available
using null output device, none available

I've seen that a similar bug has been reported on ffmpeg track some time ago : https://trac.ffmpeg.org/ticket/9713.
The patch fixing this is included on ffmpeg branches 6.0 and up.
I've noticed that the ffmpeg executable embeded in the macos package is version 4.4.4.

I've tried the 8.4.0 preview version and the bug is the same (but audio is ok on that version).

Would it be possible to upgrade ffmpeg to avoid this bug on apple silicon macs ?

STEPS TO REPRODUCE
1. Try to preview a mp4 video using h264 codec from a panasonic camera (4k30p 100Mb/s)

OBSERVED RESULT
No video is show (also no sound but it's a different bug fixed in 8.4.0 already)


SOFTWARE/OS VERSIONS
macOS: Sonoma 14.4 on M2 max.
Comment 1 caulier.gilles 2024-05-14 20:43:42 UTC
This problem must be fixed with the current 8.4.0 pre-release PKG available here :

https://files.kde.org/digikam/

Gilles Caulier
Comment 2 brancaleone 2024-05-14 20:52:11 UTC
No this is not fixed. The audio is working with that package but still no image is displayed.

Same error in the logs : 
FFmpeg: [h264 @ 0x7fcd7c053400] hardware accelerator failed to decode picture
Comment 3 caulier.gilles 2024-05-15 02:12:35 UTC
In help / Components Info dialog did you see the h264 codec in ffmpeg features ?
Comment 4 caulier.gilles 2024-05-15 03:06:21 UTC
Here on my MacBook Pro M1, an H264 video file play perfectly :
https://i.imgur.com/TVksqA5.png

Please share a video file sample to try to reproduce the problem.

Gilles Caulier
Comment 5 Maik Qualmann 2024-05-15 06:15:17 UTC
@Gilles, according to the FFmpeg bug report from Description, the problem does not occur with every H264 video on an M1 Mac. There is also a 1 second sample video in the bug report.

Maik
Comment 6 brancaleone 2024-05-15 21:17:23 UTC
Created attachment 169515 [details]
sample file from ffmpeg bug report

Indeed it does not happen to every h264 video.

I have checked h264 codec feature in ffmpeg and it is listed same as in your screenshot.

I have also tested different files:
* video converted to h264 using handbrake : plays correctly.
* video recorded with codec h264 directly on the camera in fullhd settings : plays correctly
* video recorded with codec h264 directly on the camera in 4K settings : doesn't play and error about hardware decoding in the logs.

I should add that the same files that now cause issues (4K h264 from the camera) where fine in previous versions of digikam (i may have missed a few updates but quite sure 8.0.0 was ok). Have been using the same camera settings for a few years now without issues in digikam.

I cannot had my sample here (4K gets too large for file size limit); but here is a temporary link : https://www.swisstransfer.com/d/dc09bb95-1ad9-443d-bf2f-670a99683267

I can also confirm that the sample from ffmpeg bug report does indeed not play in digikam and show the same error in logs
Comment 7 caulier.gilles 2024-06-15 07:53:12 UTC
Git commit 57536d9f6188f0370d120dd84605a41549651a7b by Gilles Caulier.
Committed on 15/06/2024 at 07:45.
Pushed by cgilles into branch 'master'.

Update media player with git/master branch from GitHub
Related: bug 484479, bug 485468

M  +2    -2    core/libs/video/QtAVPlayer/QtAVPlayer_manifest.txt
M  +3    -0    core/libs/video/QtAVPlayer/qavhwdevice_d3d11.cpp
M  +1    -1    core/libs/video/QtAVPlayer/qavhwdevice_vaapi_drm_egl.cpp
M  +1    -0    core/libs/video/QtAVPlayer/qavsubtitleframe.cpp
M  +3    -1    core/libs/video/QtAVPlayer/qtavplayerglobal.h

https://invent.kde.org/graphics/digikam/-/commit/57536d9f6188f0370d120dd84605a41549651a7b
Comment 8 caulier.gilles 2024-09-15 04:56:24 UTC
Maik,

I add the sample video in a dedicated sub-album outside digiKam.
When i access to this album in digiKam, it crash when rendering thumbnail.

Thread 40 Crashed:: Thread (pooled)
0   libavformat.58.dylib          	       0x103c784fc 0x103b18000 + 1443068
1   libdigikamcore.8.5.0.dylib    	       0x10425aefc Digikam::VideoThumbnailer::generateThumbnail(QString const&, Digikam::VideoThumbWriter&, QImage&) + 428
2   libdigikamcore.8.5.0.dylib    	       0x10425b92c Digikam::VideoThumbnailer::generateThumbnail(QString const&, QImage&) + 68
3   libdigikamcore.8.5.0.dylib    	       0x1041fcbb0 Digikam::ThumbnailCreator::createThumbnail(Digikam::ThumbnailInfo const&, QRect const&) const + 4016
4   libdigikamcore.8.5.0.dylib    	       0x1041f5a50 Digikam::ThumbnailCreator::load(Digikam::ThumbnailIdentifier const&, QRect const&, bool, bool) const + 1036
5   libdigikamcore.8.5.0.dylib    	       0x1041f612c Digikam::ThumbnailCreator::pregenerate(Digikam::ThumbnailIdentifier const&) const + 88
6   libdigikamcore.8.5.0.dylib    	       0x104211bb4 0x103e04000 + 4250548
7   libdigikamcore.8.5.0.dylib    	       0x104213330 Digikam::LoadSaveThread::run() + 400
8   libdigikamcore.8.5.0.dylib    	       0x104282908 0x103e04000 + 4712712
9   QtCore                        	       0x103847324 0x1036e4000 + 1454884
10  QtCore                        	       0x10384112c 0x1036e4000 + 1429804
11  libsystem_pthread.dylib       	       0x18e53df94 _pthread_start + 136
12  libsystem_pthread.dylib       	       0x18e538d34 thread_start + 8

digiKam is compiled with ffmpeg 4 under Macports. Qt 6.7.2 depend only of this older version. I open a ticket to move to ffmpeg 6.
Note that ffmpeg has x264 video codec of course. 

Gilles
Comment 9 Maik Qualmann 2024-09-15 09:43:24 UTC
Git commit 9a74e01d09819894241f10816ec8af14d8b479c1 by Maik Qualmann.
Committed on 15/09/2024 at 09:42.
Pushed by mqualmann into branch 'master'.

always assign a video backend to environment variable
And make the video backend platform
dependent selectable in the setup.
Related: bug 492922

M  +9    -5    core/app/main/main.cpp
M  +9    -5    core/showfoto/main/main.cpp
M  +12   -2    core/utilities/setup/misc/systemsettings.cpp
M  +6    -1    core/utilities/setup/misc/systemsettings.h
M  +42   -5    core/utilities/setup/misc/systemsettingswidget.cpp

https://invent.kde.org/graphics/digikam/-/commit/9a74e01d09819894241f10816ec8af14d8b479c1
Comment 10 caulier.gilles 2024-09-15 11:17:24 UTC
Maik,

It crash as before :

Thread 46 Crashed:: Thread (pooled)
0   libavformat.58.dylib          	       0x105d744fc 0x105c14000 + 1443068
1   libdigikamcore.8.5.0.dylib    	       0x106356734 Digikam::VideoThumbnailer::generateThumbnail(QString const&, Digikam::VideoThumbWriter&, QImage&) + 428 (videothumbnailer.cpp:169)
2   libdigikamcore.8.5.0.dylib    	       0x106357164 Digikam::VideoThumbnailer::generateThumbnail(QString const&, QImage&) + 68 (videothumbnailer.cpp:215)
3   libdigikamcore.8.5.0.dylib    	       0x1062f83e8 Digikam::ThumbnailCreator::createThumbnail(Digikam::ThumbnailInfo const&, QRect const&) const + 4016 (thumbnailcreator_engine.cpp:203)
4   libdigikamcore.8.5.0.dylib    	       0x1062f1288 Digikam::ThumbnailCreator::load(Digikam::ThumbnailIdentifier const&, QRect const&, bool, bool) const + 1036 (thumbnailcreator.cpp:275)
5   libdigikamcore.8.5.0.dylib    	       0x1062f1964 Digikam::ThumbnailCreator::pregenerate(Digikam::ThumbnailIdentifier const&) const + 88 (thumbnailcreator.cpp:180)
6   libdigikamcore.8.5.0.dylib    	       0x10630d3ec Digikam::ThumbnailLoadingTask::execute() + 188 (thumbnailtask.cpp:71)
7   libdigikamcore.8.5.0.dylib    	       0x10630eb68 Digikam::LoadSaveThread::run() + 400 (loadsavethread.cpp:118)
8   libdigikamcore.8.5.0.dylib    	       0x10637e140 Digikam::DynamicThread::Private::run() + 96 (dynamicthread.cpp:173)
9   QtCore                        	       0x105943324 0x1057e0000 + 1454884
10  QtCore                        	       0x10593d12c 0x1057e0000 + 1429804
11  libsystem_pthread.dylib       	       0x18e53df94 _pthread_start + 136
12  libsystem_pthread.dylib       	       0x18e538d34 thread_start + 8

Look like the crash is in libav so it's a ffmpeg 4 problem.

Gilles
Comment 11 Maik Qualmann 2024-09-15 12:02:34 UTC
Yes, it's about the QMultimedia Player. It is now possible to select "Darwin" as video backend (restart digiKam) to compare whether H264 videos can be played better with the macOs backend or with FFmpeg.

Unfortunately, this does not include the video thumbnailer.

Maik
Comment 12 caulier.gilles 2024-09-15 12:06:36 UTC
Maik,

I cannot test the playback with this file as digiKam crash while thumb generation...

Gilles
Comment 13 caulier.gilles 2024-09-15 17:01:38 UTC
Maik,

I prepared a new compilation of qt6-qtmultimedia with ffmpeg6. I patched the ports configuration files for that. Until now all compile fine and very quickly. QtWebEngine which depends of QtMultimedia have been recompiled in 30 mns. Silicon architecture is very impressive, without to hear the CPU Fan. I don't know how Apple deal with the chips temperature. By comparison, the Intel Macbook pro take age and very noisy.

So i will be able to check if the libav crash is fixed and if the mp4 playback work as expected.

Gilles
Comment 14 caulier.gilles 2024-09-16 04:15:36 UTC
Hi Maik,

With the ffmpeg6, video thumbnail work fine :

https://i.imgur.com/NnX77vb.png

But playback with ffmpeg backend do not work :

https://i.imgur.com/y4ZW4xX.png

No more chance with the system codec :

https://i.imgur.com/A6vUX3B.png

VLC on the same system play properly :

https://i.imgur.com/e83LbLF.png

Gilles
Comment 15 caulier.gilles 2024-09-16 04:26:32 UTC
Maik,

My h264 video taken with an iphone play well with both digiKam backends :

https://i.imgur.com/Qta3H1Z.png

Gilles
Comment 16 caulier.gilles 2024-09-16 04:33:17 UTC
digiKam terminal trace when tring to play the defect h264 with ffmpeg :

digikam.general: Stacked View Mode :  6
digikam.general: Database Info populated for QUrl("file:///System/Volumes/Data/Users/gilles/Pictures/BUG 487031/m1_not_decodable_with_acceleration.mp4")
digikam.general: Play video with QtMultimedia started: QUrl("file:///System/Volumes/Data/Users/gilles/Pictures/BUG 487031/m1_not_decodable_with_acceleration.mp4")
digikam.general: Stacked View Mode :  6
digikam.general: Database Info populated for QUrl("file:///System/Volumes/Data/Users/gilles/Pictures/BUG 487031/m1_not_decodable_with_acceleration.mp4")
digikam.general: Play video with QtMultimedia started: QUrl("file:///System/Volumes/Data/Users/gilles/Pictures/BUG 487031/m1_not_decodable_with_acceleration.mp4")

with darwing codec :

digikam.general: Stacked View Mode :  6
digikam.general: Database Info populated for QUrl("file:///System/Volumes/Data/Users/gilles/Pictures/BUG 487031/m1_not_decodable_with_acceleration.mp4")
digikam.general: Play video with QtMultimedia started: QUrl("file:///System/Volumes/Data/Users/gilles/Pictures/BUG 487031/m1_not_decodable_with_acceleration.mp4")
digikam.general: QtMultimedia Error:  "Failed to load media"
digikam.general: Stacked View Mode :  6

gilles
Comment 17 caulier.gilles 2024-09-16 04:38:20 UTC
Correction: with ffmpeg codec, the player try to start but do not play without error. 

https://i.imgur.com/6QrjYTC.png

My previous screenshot with ffmpeg was wrong, as you need to restart digiKam when codec is changed in setup/misc.

This explain the terminal difference between both codecs.

Gilles
Comment 18 Maik Qualmann 2024-09-16 06:10:26 UTC
The video is playing under Linux, which FFmpeg version are you using exactly? It would be best to take a screenshot of the component window. Make QtMultimedia more talkative with this debug rule (the digiKam internal one must be deactivated in the setup).

export QT_LOGGING_RULES=qt.multimedia.ffmpeg*=true

Maik
Comment 19 caulier.gilles 2024-09-16 06:46:56 UTC
It's ffmpeg 6.1.1 : https://ports.macports.org/port/ffmpeg6/
Comment 20 caulier.gilles 2024-09-16 07:36:26 UTC
Maik,

It's clear. QtMultimedia do not compile fine with ffmpeg :

No QtMultimedia backends found. Only QMediaDevices, QAudioDevice, QSoundEffect, QAudioSink, and QAudioSource are available.
Failed to create QVideoSink "Not available"
Failed to initialize QMediaPlayer "Not available"

Gilles
Comment 21 caulier.gilles 2024-09-16 07:39:23 UTC
Maik,

If i switch to Darwing codec the message do not appears on the terminal and video play back work fine, with my own h264 video files.

Gilles
Comment 22 caulier.gilles 2024-09-16 12:19:22 UTC
Git commit 90ee89c42deed9faa320ecd08839150e4496187c by Gilles Caulier.
Committed on 16/09/2024 at 12:17.
Pushed by cgilles into branch 'master'.

Move ffmpeg6 for libexec to macports prefix, else qt6multimedia do not handle it properly.

M  +0    -2    bootstrap.macports
M  +6    -4    project/bundles/macports/01-build-macports.sh
M  +1    -1    project/bundles/macports/04-build-installer.sh
M  +1    -1    project/bundles/macports/tools/console.sh

https://invent.kde.org/graphics/digikam/-/commit/90ee89c42deed9faa320ecd08839150e4496187c
Comment 23 caulier.gilles 2024-09-16 13:49:39 UTC
Great.

With ffmpeg 6 + Qt6Multimedia properly assembled, it's works on my Macbook M1...

https://i.imgur.com/1O5LCa6.png

PKG 8.5.0 pre-release from today : digiKam-8.5.0-20240916T131337-Qt6-MacOS-arm64.pkg

https://files.kde.org/digikam/

I can close this file now.

Gilles Caulier
Comment 24 brancaleone 2024-09-16 20:26:41 UTC
I can confirm that it is working on my files too after switching decoding backend to Darwin. Maybe it can be make default for apple silicon ?