Bug 404898 - Portrait video gets a preview that's in landscape orientation
Summary: Portrait video gets a preview that's in landscape orientation
Status: CONFIRMED
Alias: None
Product: kdemultimedia
Classification: Miscellaneous
Component: ffmpegthumbs (show other bugs)
Version: 19.08.0
Platform: openSUSE Linux
: NOR normal
Target Milestone: ---
Assignee: Unassigned bugs mailing-list
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-02-28 10:10 UTC by Philippe ROUBACH
Modified: 2022-10-16 13:44 UTC (History)
6 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments
icon view, player view (391.94 KB, image/png)
2019-02-28 10:10 UTC, Philippe ROUBACH
Details
exif data of the video (443.16 KB, image/png)
2019-03-02 14:30 UTC, Philippe ROUBACH
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Philippe ROUBACH 2019-02-28 10:10:44 UTC
Created attachment 118430 [details]
icon view, player view

SUMMARY

portrait video icon is a landscape one, video player view is a landscape one

STEPS TO REPRODUCE
1. open dolphin in a folder containing portrait videos
2. ask for icon view for folder
3. play video with video player in data panel

OBSERVED RESULT

portrait video icon is a landscape one

player view is a landscape one

EXPECTED RESULT

portrait video icon is a portrait one

player view is a portrait one

SOFTWARE/OS VERSIONS
Windows: 
MacOS: 
Linux/KDE Plasma: 
(available in About System)
KDE Plasma Version: 5.15.2
KDE Frameworks Version: 5.55.0
Qt Version: 5.15.1

ADDITIONAL INFORMATION

ffmpegthumbs is used to create video icons

it seems that exif data "top,right" is not applied by ffmpegthumb and video player

with digikam 6.0.0 there is no problem with icon view but also there is a problem with its video player

see bug #404896
in this bug report you can see the exif data thus the position
Comment 1 Nate Graham 2019-03-02 13:56:18 UTC
Any chance you can attach the video file itself?
Comment 2 Philippe ROUBACH 2019-03-02 14:28:48 UTC
(In reply to Nate Graham from comment #1)
> Any chance you can attach the video file itself?

for an attachment file zize limit is 4000 KB !

here is a link

https://photos.app.goo.gl/kj8ENJpYMEe2Rbup7
Comment 3 Philippe ROUBACH 2019-03-02 14:30:06 UTC
Created attachment 118480 [details]
exif data of the video

you can see the exif orientation
Comment 4 Rog131 2019-03-03 10:53:29 UTC
The KDE is using kdemultimedia/ffmpegthumbs to thumbnail the video files
Comment 5 Rog131 2019-03-03 11:15:01 UTC
Maybe this helps to add the rotation metadata to ffmpegthumbs...

By Rotation Metadata in Video Files Created by Mobile Devices: https://blog.addpipe.com/mp4-rotation-metadata-in-mobile-video-files/ :

"The rotation metadata is stored as an integer, it represents degrees, and has 4 possible values: 0, 90, 180 and 270."

The example clip: 20130913T153029.mp4 has 90 degree rotation. The meta/exif data can be read with the ExifTool - https://www.sno.phy.queensu.ca/~phil/exiftool/ :

"$ exiftool 20130913T153029.mp4 | grep Rotation
Rotation                        : 90
"

and MediaInfo - https://mediaarea.net/en/MediaInfo :

"$ mediainfo 20130913T153029.mp4 | grep Rotation
Rotation                                 : 90°"


The KDE ffmpegthumbs is based on the ffmpegthumbnailer: https://github.com/dirkvdb/ffmpegthumbnailer

The FFmpegThumbnailer 2.1.2 changelog: "Take rotation metadata into account when generating thumbnails"
Comment 6 Nate Graham 2019-03-03 22:35:13 UTC
Thanks Rog131! That's very helpful. I'll see if I can put together a patch.
Comment 7 Rog131 2019-03-04 18:49:28 UTC
Meanwhile - alternatives:

Scripting KDE video preview thumbnails with the ffmpegthumbnailer: https://forum.kde.org/viewtopic.php?f=224&t=153847#p409077 . Preview clip: https://www.youtube.com/watch?v=L694nghnPo8

The ffmpegthumbnailer ( https://github.com/dirkvdb/ffmpegthumbnailer )  has the kffmpegthumbnailer ( https://github.com/dirkvdb/ffmpegthumbnailer/tree/master/kffmpegthumbnailer ). It is for the KDE4/Qt4 but it is easy to port ot the KF5/Qt5.

Preview image of the alternatives & ffmpegthumbs: https://imgur.com/VqVJFBi
Comment 8 Philippe ROUBACH 2019-08-16 14:52:52 UTC
problem still there with

opensuse Argon 15.1
kde apps 19.08.0
kde plasma 5.16.4
kde frameworksc 5.61.0
qt 5.13.0
Comment 9 Philippe ROUBACH 2019-08-16 14:54:23 UTC
with digikam 6.0.0+ no problem.
Comment 10 Philippe ROUBACH 2019-08-18 09:51:15 UTC
In digikam 6.0.0+ preview thumbnail of a portrait video is a portrait preview 

In the digikam 6.0.0 realease announcement it is said "In release 5.5.0 we started to use the QtAV framework to play video media in digiKam. We chose QtAV because this framework directly uses ffmpeg codecs which de facto supports all formats very well".

I recommend reading "Video Files Management as Photo" here
https://www.digikam.org/news/2019-02-10-6.0.0_release_announcement/

if it can help.
Comment 11 Philippe ROUBACH 2021-05-21 16:12:06 UTC
kde 21.04.1

problem still there.

In case there is no rotation data perhaps it is useful to use vertical and horizontal dimensions to decide portrait or landscape for the preview.
Comment 12 Lapineige 2021-05-22 10:21:40 UTC
I can confirm it's still there.

> In case there is no rotation data perhaps it is useful to use vertical and horizontal dimensions to decide portrait or landscape for the preview.

I would like to second this, but I wonder if there could be some edge cases where this can't work.

Most videos won't have a square format, but there will still be some video that are encoded in a landscape mode while displaying portrait content (and reverse ?).
Maybe that's not an issue as it is rare and that would already be an improvement ?
Comment 13 Lapineige 2021-05-22 10:22:10 UTC
I forgot: I'm using Kubuntu, Plasma 5.19
Comment 14 Philippe ROUBACH 2021-09-17 11:19:41 UTC
kde gear 21.08.1
plasma 5.22.5
frameworks 5.86.0
qt 5.15.2

Problem still there.

I don't understand a long time ago now digikam team resolved the problem.
Comment 15 Philippe ROUBACH 2022-04-22 09:07:53 UTC
kde gear 22.04.0
plasma 5.24.4
frameworks 5.93.0
qt 5.15.2

Problem still there.
Comment 19 Méven Car 2022-05-18 04:35:22 UTC
I get those warnings when playing the video in dolphin:
```
(dolphin:252856): GStreamer-Video-CRITICAL **: 06:34:43.987: gst_video_center_rect: assertion 'src->h != 0' failed
```

When Phonon-gstreamer backend. 

https://invent.kde.org/multimedia/ffmpegthumbs/-/merge_requests/18 takes care of the thumbnails.
Comment 20 Méven Car 2022-05-19 12:28:47 UTC
(In reply to Méven Car from comment #19)
> I get those warnings when playing the video in dolphin:
> ```
> (dolphin:252856): GStreamer-Video-CRITICAL **: 06:34:43.987:
> gst_video_center_rect: assertion 'src->h != 0' failed
> ```
> 
> When Phonon-gstreamer backend. 
> 
> https://invent.kde.org/multimedia/ffmpegthumbs/-/merge_requests/18 takes
> care of the thumbnails.

This warning is independent from this particular patch, it is an issue in phonon-gstreamer (the thumbnailer does not use gstreamer)
Comment 21 Philippe ROUBACH 2022-07-04 11:30:37 UTC
kde gear 22.04.2
plasma 5.25.2
frameworks 5.95.0
qt 5.15.5

Problem still there.
Comment 22 Méven Car 2022-07-31 17:49:20 UTC
Git commit 0b17ce95d4e4a38c9135ea970129370ddb355cea by Méven Car, on behalf of Martin T. H. Sandsmark.
Committed on 31/07/2022 at 17:47.
Pushed by meven into branch 'master'.

Implement image transforms

Should really just throw out 90% of the old "upstream" code and decode
straight to a QImage. Would improve the performance and significantly
slim down the codebase.

Code stolen from myself:
    https://invent.kde.org/sandsmark/qt-ffmpeg-imageplugin/-/blob/master/ffmpegimageplugin.cpp#L468-493

M  +5    -1    ffmpegthumbnailer/imagewriter.cpp
M  +2    -1    ffmpegthumbnailer/imagewriter.h
M  +40   -1    ffmpegthumbnailer/moviedecoder.cpp
M  +3    -0    ffmpegthumbnailer/moviedecoder.h
M  +1    -1    ffmpegthumbnailer/videothumbnailer.cpp

https://invent.kde.org/multimedia/ffmpegthumbs/commit/0b17ce95d4e4a38c9135ea970129370ddb355cea
Comment 23 Philippe ROUBACH 2022-08-22 07:04:53 UTC
kde gears 22.08.0
plasma 5.25.4
frameworks 5.97.0
qt 5.15.5

Problem still there.
Comment 24 Philippe ROUBACH 2022-10-16 13:44:20 UTC
Operating System: openSUSE Tumbleweed 20221014
KDE Plasma Version: 5.26.0
KDE Frameworks Version: 5.99.0
Qt Version: 5.15.6
KDE gears 22.08.1

problem still there