Bug 385726

A library from Tumblr
https://github.com/tumblr/laphs

"a Live Photo consists of a JPG combined with a MOV file that contains 45 frames playing back at around 15 frames per second." https://www.macrumors.com/2015/09/22/iphone6s-live-photos-file-details/

Currently, how is detected this kind of file in digiKam ? As a video or as an image ?

We need also some samples to test. Please provide files. Thanks in advance...

Gilles Caulier

It's not clear here :

https://stackoverflow.com/a/34335994

one or 2 separate files are provided ?

Gilles Caulier

All link to this file about HEIF support for digiKAm are JSon library.

DK is based on C++ code, not JSon or another language.

Nokia provide an open source lib to handle HEIF images :

https://github.com/nokiatech/heif

Gilles Caulier

Git commit 86a40546894c7b053ab3aad85a11a0c709e60220 by Gilles Caulier.
Committed on 08/04/2019 at 15:18.
Pushed by cgilles into branch 'master'.

Add ImageMagick codecs support as QImage fail-back image loader to be able
to render thumbnails and load image in editor with extra image format as HEIC, FITS, JPEG-XR, XCF.
For the moment, this will support image loading in this formats. Later we can plan to add writting
support if ImageMagick codecs can do it.
Related: bug 360806, bug 393408
FIXED-IN: 6.1.0

M  +1    -0    Mainpage.dox
M  +1    -0    NEWS
M  +8    -1    core/CMakeLists.txt
M  +4    -0    core/app/DigikamCoreTarget.cmake
M  +3    -0    core/app/utils/digikam_config.h.cmake.in
M  +10   -0    core/libs/dialogs/libsinfodlg.cpp
M  +2    -1    core/libs/dimg/loaders/README
M  +58   -2    core/libs/dimg/loaders/qimageloader.cpp
M  +5    -0    core/libs/dimg/loaders/qimageloader.h

https://commits.kde.org/digikam/86a40546894c7b053ab3aad85a11a0c709e60220

Just tried adding a Live Photo to digikam 6.1 on macOS and all it produces is two entries for the image and the video part each. It doesn't show the two files as one entry as you'd expect coming from Apple Photos.

If this is how it's supposed to work, it's a wrong implementation and not doing anything more than just importing the two files as presented by ANY other software. I would not call this Live Photo support then. However I assume this might be an incompatibility with the macOS port yet?

Should I try the GNU/Linux version or is there any other thing to keep in mind?

2 entries for one image and on video ?

I don't understand the dysfunction exactly. Please :

- take a shot of digiKam dysfunction.
- take a shot of iphoto view with files
- share files + XMP sidecars to try to reproduce the problem here.
- take a shot of DK/Setup/Metadata pannel.

Gilles Caulier

Wooops, no not 2 for each, I changed my phrasing mid-sentence and then that happened. Sorry.

It's one entry for the image and video part each. So it shows the technically correct two files as what they are - technically - 2 files. However the whole point of Live Photos isn't that the capture process is bundled, but also viewing the image alongside the video seamlessly.

Default settings basically beyond some preferences during install wizard. SQLite db, locally stored. No sidecar files.

Attached screenshot of digikam and a short video screencap of Apple Photos viewing a Live Photo. As you can see it only displays one entry visually for the IMG_nnn.JPG & IMG_nnn.MOV pair and on mouse hover it will start playing the video bit without sound and if you go into view mode you can play with sound there. (latter not shown)

Created attachment 120774 [details]
Screenshot digikam Live Photo as two separate entries

Created attachment 120775 [details]
Apple Photos Live Photo View

Ok, now i understand.

So typically, it's a items group import in database. I'm sure that it's not implemented in DK.

The JPEG + MOV file are grouped in Iphoto. In DK we can do the same but in the GUI there is nothing to switch from a JPEG to MOV thumbnails view when there are grouped.

This must be a topic well described in a new bugzilla entry as a wish.

Gilles Caulier

I think if we're going to implement a unified view for the JPG (or HEIF) and movie pair we shouldn't use the regular grouping function, because then we couldn't group multiple Live Photos without creating a messy situation when you ungroup them again.

So in a way, we'd probably need a "level zero grouping" so to say. (sorry, can't find a good way to phrase it)

You say we should create a new ticket for that then? Not sure about that, but I could do that. Whilst we're at it we might as well implement other popular Live Photo-like implementations as well and use the same unified view for that. Samsung has Motion Photos, Google has Motion Photos as well on Pixel devices and some Android One phones as well I think (I know the Nokia 3.1 has that too for example) and then Huawei has an implementation very much like Apple's too.

They all work a little different, but technically it's always a pair of image and movie files being fused into one element visually. Samsung for example embeds the video into the image file itself (making the JPG bigger in size).

I know this might sound like a lot at once, but I swear to God, despite me owning a nice and lovely Canon DSLR often times I will at least take a picture a second time with my smartphone just to capture a Live Photo. It really adds that much more to pictures and lets you relieve moments a bit more vividly that didn't seem like a "video moment" but the little bit of motion can often times add a lot.

Google has a similar feature in the form of motion images. Should I file a separate but report? Our can we track it on this report?

Yes please, fill a new file, as this one is already closed.

Thanks in advance

Gilles Caulier

Why is this item flagged as resolved?

Live Photos are still just presented as a separate JPG/HEIC and a .mov.


Here's some further reading if it helps grasp the concept of how it all works how it's designed to be a) detected b) treated/viewed.

1) Very nice conclusive overview post on Reddit, also links to sample images for Apple's version, but also Google's. https://www.reddit.com/r/DataHoarder/comments/cuk06h/help_preserving_iphone_live_photos_how_they_seem/exx2rj6/

2) This is another program written in C++ that to my knowledge correctly implements Live Photos: https://github.com/LycheeOrg/Lychee-Laravel/issues/378

3) Apple's HIG on how Live Photos should be treated: https://developer.apple.com/design/human-interface-guidelines/live-photos/overview/

If desired I can provide a sample for Samsung's implementation, too.

I would love to see support for embedded movies within jpg (be it Apple Live Photo, Google Photo or Samsung Motion Photos).

I see this has been reopened and couldn't see any other tickets for Google/Samsung support. I can supply sample Samsung images from my S10 if needed :-)

https://drive.google.com/drive/folders/1PgxB3JBql7kRzOkEzhBwqKQUXtddcLAj?usp=sharing

The filesize limit doesn't allow me to include the files directly, which is a shame, not a fan of link rot and I gotta admit, this link WILL go down once this issue is resolved, because I don't like lingering public links on my Drive account.

Either way, I included a sample for Huawei's "Moving Picture" implementation. (consumer info page: https://consumer.huawei.com/uk/support/faq/how-to-take-moving-pictures/)

The pic was taken on my MediaPad M5 8.4" tablet. Not amazing quality on that one, but should do the trick of showing the file's logic.

I also included a sample for Samsung's "Motion Photo". (consumer info page: https://www.samsung.com/global/galaxy/what-is/motion-photo/ (it doesn't list S20 as compatible device, but it's been reported that it is indeed still a maintained feature, guess they forgot to update that page so far)

That pic was taken with my Galaxy Note9.

Forgive the dirty keyboard...

*** Bug 419375 has been marked as a duplicate of this bug. ***

@Gilles Caulier

Here's a link to Google Motion Photo.

https://photos.app.goo.gl/g37hyTahF69qqQ2n6

(In reply to Ritesh Raj Sarraf from comment #19)
> @Gilles Caulier
> 
> Here's a link to Google Motion Photo.
> 
> https://photos.app.goo.gl/g37hyTahF69qqQ2n6

In case Google post-processes and messes it up.

https://drive.google.com/file/d/1EFsJBFL1ZTNhIq5J_EqkOuLcDb8jpATo/view?usp=sharing

Thanks Ritesh. I will take a look.

Gilles Caulier

(In reply to caulier.gilles from comment #21)
> Thanks Ritesh. I will take a look.
> 
> Gilles Caulier

Forgive me if I may come across as impatient, I am just very thankful for finally being close to getting the long awaited Aperture replacement... (yep... I held out this long!)

My question is: is this feature still on track for 7.0 final?

Again, thank you so so much for even getting on this, bridging the different kinds of live photo-like implementations no less. I really think it's a very transformative format of our time and I cannot tell you how often I've been enjoying the little bit of extra life Live Photos and similar approaches have brought to me.

yes, HEIF is supported in digiKam 7.0.0 right now, but not yet the video embeded.

The problem is with Exiv2 which refuse to support HEIF due to possible patents.

If HEIF will host video, we will need to be able to extract this data, and Exiv2 sound like the best way to operate.


The problem between HEIF and EXiv2 do not touch only HEIF, but all formats based on ISO base media format:

https://en.wikipedia.org/wiki/ISO/IEC_base_media_file_format

This include also the famous Canon CR3 raw files, and it's very problematic for the future...

https://github.com/Exiv2/exiv2/issues/236

So we need to have a plain support of ISO/IEC 14496-12 format first before to support extra video from this kind of container...

Gilles Caulier

(In reply to caulier.gilles from comment #23)
> yes, HEIF is supported in digiKam 7.0.0 right now, but not yet the video
> embeded.
> 
> The problem is with Exiv2 which refuse to support HEIF due to possible
> patents.
> 
> If HEIF will host video, we will need to be able to extract this data, and
> Exiv2 sound like the best way to operate.
> 
> 
> The problem between HEIF and EXiv2 do not touch only HEIF, but all formats
> based on ISO base media format:
> 
> https://en.wikipedia.org/wiki/ISO/IEC_base_media_file_format
> 
> This include also the famous Canon CR3 raw files, and it's very problematic
> for the future...
> 
> https://github.com/Exiv2/exiv2/issues/236
> 
> So we need to have a plain support of ISO/IEC 14496-12 format first before
> to support extra video from this kind of container...
> 
> Gilles Caulier

Maybe I'm missing something here, but if we rely on a third-party dependency to resolve this and they don't release a solution in code before July (and even that isn't certain) how can you be sure that this feature is on track for 7.0 final scheduled to release on or around July 12?

Do you have an intermediary workaround in mind to use until this gets resolved?

Thanks for the wealth of background info as well! Very interesting to see this unfold and at the same time I cannot stop but cry over software patents being a thing... ESPECIALLY for things like file formats. Real innovation stifling at work...

digiKam already rely on Exiv2 to all file format metadata support. There is no exception for HEIF and CR3. We need to still homogeneous and not to make a big puzzle. I'm fighting against this idea since a while, as digiKam is already a big application with many external dependencies.

There is no work around planed until July release...

If you want to see a better HEIF/CR3 support in the future, go to Exiv2 project from github and vote to implement metadata support for these format.

Gilles Caulier

Thank you so much for your input.

I wasn't trying to advocate for a work-around, I was just trying to make sense of why you're optimistic, since this seems like it could go either way over at that project. At least from what I read regarding the potential legal implications.

I gave the OP post a thumbs up, I guess that's what you meant by voting for it.

Again thank you so much and I'll just adopt your hopefulness and anticipate the release, hopefully with CR3 and HEIC metadata support.

I guess one way or another a way has to be found either way eventually, so if I don't completely misinterpret this, if the feature doesn't make it into 7.0, hope wouldn't be completely lost?

Cheers!

Don't be afraid. HEIF and CR3 support are popular format, and Bugzilla is here to remember the important entry (you can vote for this support here too).

HEIF is Android and Apple used format -> very popular
CR3 is Canon RAW file -> a major actor in photography.

So...

Gilles Caulier

(In reply to caulier.gilles from comment #27)
> Don't be afraid. HEIF and CR3 support are popular format, and Bugzilla is
> here to remember the important entry (you can vote for this support here
> too).
> 
> HEIF is Android and Apple used format -> very popular
> CR3 is Canon RAW file -> a major actor in photography.
> 
> So...
> 
> Gilles Caulier

Well, I'm a CR2 user thanks to my 60D, I know it's pretty much inevitable a solution will be found. HEIF isn't just Apple and Android though, I think in the future demand for it will rise quite a bit more, because it does provide substantial space savings which can be applied elsewhere too.

Oh also I wanted to let you know that I've donated to Digikam a few days ago because seeing the Live Photo format getting picked up alerted me to help out a bit myself. When this issue gets resolved I'll be looking at weeks if not months of carefully moving data to Digikam and trialing productive use with a large library. Finally a home for my photography and videoclips again.

Much love from Germany! :)

Git commit c23ccd2c0e40c624631e748237aacbd26527eda8 by Gilles Caulier.
Committed on 16/03/2021 at 03:36.
Pushed by cgilles into branch 'master'.

Exiv2 0.27.4 will introduce Base Media File format support for CR3, HEIF, HEIC, and AVIF image containers.
Enable Exiv2 compilation option for BMFF supports.
Add new static method to get BMFF support from Exiv2.
Report BMFF support in help/components dialog.
Fix MetaEngine::initializeExiv2() to register std::atexit C++11 method and cleanup automatically XMP SDK allocation at application exit.
Remove obsolete cleanupExiv2() method from MEtaEngine and relevant calls everywhere.

M  +0    -2    core/app/main/main.cpp
M  +3    -2    core/libs/dialogs/libsinfodlg.cpp
M  +20   -8    core/libs/metadataengine/engine/metaengine.cpp
M  +5    -8    core/libs/metadataengine/engine/metaengine.h
M  +6    -0    core/libs/metadataengine/engine/metaengine_p.cpp
M  +1    -0    core/libs/metadataengine/engine/metaengine_p.h
M  +2    -2    core/showfoto/main/main.cpp
M  +0    -4    core/tests/dimg/dimgabstracthistorytest.cpp
M  +0    -2    core/tests/dimg/dimgfilteractiontest.cpp
M  +0    -2    core/tests/dimg/testcolorbalancefilter.cpp
M  +0    -2    core/tests/dimg/testdimgloader.cpp
M  +0    -2    core/tests/dimg/testequalizefilter.cpp
M  +0    -2    core/tests/dplugins/loadandrun_generic.cpp
M  +0    -2    core/tests/fileio/loadsavethreadtest.cpp
M  +0    -2    core/tests/filters/testautocrop.cpp
M  +0    -2    core/tests/filters/testlensfuniface.cpp
M  +0    -2    core/tests/filters/testnrestimate.cpp
M  +0    -2    core/tests/geolocation/editor/test_gpsimageitem.cpp
M  +0    -3    core/tests/geolocation/geoiface/demo/mainwindow.cpp
M  +1    -2    core/tests/metadataengine/abstractunittest.h
M  +0    -1    core/tests/metadataengine/commentreadwritetest.cpp
M  +0    -1    core/tests/metadataengine/ratingreadwritetest.cpp
M  +0    -1    core/tests/metadataengine/tagsreadwritetest.cpp
M  +0    -2    core/tests/video/framesencoder.cpp
M  +1    -9    project/bundles/3rdparty/ext_exiv2/CMakeLists.txt
M  +1    -0    project/bundles/flatpak/org.kde.digikam.json
M  +1    -1    project/scripts/bootstrap-exiv2.linux.sh
M  +1    -0    project/scripts/bootstrap-exiv2.mxe.sh

https://invent.kde.org/graphics/digikam/commit/c23ccd2c0e40c624631e748237aacbd26527eda8

Just checked the release notes for 7.2 and noticed that Apple Live Photo Support has been scheduled for 7.4, which is fantastic!

I have one question though, are competing, similar formats like Samsung's Motion Photos, Huawei's Moving Picture and possibly other contenders that may be relevant in later feedback still on track for release as well?

Personally for my own needs I am using Apple and Samsung's formats, so those would be my main concern, but Huawei is also pretty popularly used. Not sure what other manufacturers put out there, but having Apple, Huawei, Google and Samsung included would cover a large share of the market.

Photo enthusiasts with mobile device choices around their creative needs may even go for some other brands, but I'm pretty sure those need not be the focus for now and could get added once the main share of the market is implemented and has gone through some wider trialing in the field I guess?

Also, if desired I could add a new Samsung Motion Photo sample taken with a newer device (Note20 Ultra) just in case there are alterations to Samsung's implementation. (doubt it, but I want to be as assisting in this implementation as possible and providing samples is something I can do pretty nicely as someone who doesn't develop themselves)

About Apple Live Photo, Samsung's Motion Photos, and Huawei's Moving Picture, i'm not sure where the differences are located.

I'm sure that Apple Live Photo is HEIF based. 

in all cases libheif must support these format to access to image and video data in container. About metadata, libexiv2 will support Base Media Format (HEIF and AVIF containers) with next 0.27.4 release planed for June 2021.

Samsung's Motion Photo format consists of a picture (jpeg) and movie (mp4) merged into a single file. Between them there is a special marker. The script finds the marker and splits it accordingly.

It's perhaps HEIF container, as HEIF con embed JPEG image. This need investiguations

Huawei's Moving Picture file format => i cannot found technical details about this container on the Web. So we need reverse engineering...

(In reply to Ritesh Raj Sarraf from comment #20)
> (In reply to Ritesh Raj Sarraf from comment #19)
> > @Gilles Caulier
> > 
> > Here's a link to Google Motion Photo.
> > 
> > https://photos.app.goo.gl/g37hyTahF69qqQ2n6
> 
> In case Google post-processes and messes it up.
> 
> https://drive.google.com/file/d/1EFsJBFL1ZTNhIq5J_EqkOuLcDb8jpATo/
> view?usp=sharing

Sorry. It seems that the picture that I linked, for whatever reason, is not what I intended to share as a motion photo from an Android phone.

Please see this one instead:
https://photos.app.goo.gl/rcmuoCg6ZifZo4jD8

As of today, I have verified that this indeed is an Android motion photo and is accessible anonymously.

(In reply to caulier.gilles from comment #31)
> About Apple Live Photo, Samsung's Motion Photos, and Huawei's Moving
> Picture, i'm not sure where the differences are located.
> 
> I'm sure that Apple Live Photo is HEIF based. 
> 
> in all cases libheif must support these format to access to image and video
> data in container. About metadata, libexiv2 will support Base Media Format
> (HEIF and AVIF containers) with next 0.27.4 release planed for June 2021.

Modern implementations yes, but before Apple adopted HEIF Live Photos existed as well and there's plenty of JPEG-based Live Photo backlog that would be in I bet many users' libraries. :)

You may find more details about GoogleMotion Photos at https://github.com/photoprism/photoprism/issues/1739 and the links there. Quite easy to implement. Would be happy to see extraction (jpg and mp4) as well as the preview of video in digikam. Tx a lot.

Git commit febc26188fe4e6037e5c8a9b1840f65729972a47 by Gilles Caulier.
Committed on 06/04/2022 at 20:38.
Pushed by cgilles into branch 'master'.

Add ExifTool backend to load metadata for image or video.
Exiv2 is always used in priority. If it fail, ExifTool is used instead.
ExifTool has the capability to create an EXV container for Exiv2. All metadata extracted with ExifTool
are concatened to the EXV container with all possible translations with Exif, Iptc, and Xmp tags.
Of course tags supported by ExifTool but not supported by Exiv2 are lost.
Related: bug 416516, bug 360714, bug 264210, bug 426938, bug 419801, bug 419375, bug 103247

M  +1    -0    core/libs/metadataengine/CMakeLists.txt
M  +7    -0    core/libs/metadataengine/dmetadata/dmetadata.h
A  +94   -0    core/libs/metadataengine/dmetadata/dmetadata_exiftool.cpp     [License: GPL (v2+)]
M  +42   -17   core/libs/metadataengine/dmetadata/dmetadata_fileio.cpp
M  +7    -6    core/libs/metadataengine/engine/metaengine.h
M  +5    -0    core/libs/metadataengine/engine/metaengine_fileio.cpp
M  +4    -2    core/libs/metadataengine/exiftool/exiftoolparser_command.cpp

https://invent.kde.org/graphics/digikam/commit/febc26188fe4e6037e5c8a9b1840f65729972a47

Hi, I concur with previous posters that it'd be an important feature for Digikam to properly support the various motion photo formats. Here are some thought:

I first encountered motion photos by inadvertently enabling this feature on my Samsung phone. (I didn't even notice that some icon was now shown in yellow instead of white.) Since I was using an alternative photo gallery that didn't support motion photos I never even noticed that my "photos" actually included video *and* audio captures of the people around me...
Hence my first request: Digikam should at the minimum show an overlay icon on all motion photos. (Similar to the globe icon put on geotagged images.)

This may sound rather negative, but I now very much enjoy my motion pictures. In particular on nicely implemented photo galleries. (See https://demo.photoprism.app/library/browse where motion photos come to life the moment you're hovering over them with the mouse pointer.)

Here's a list of features that I'd consider rather useful if not essential. (Some should be rather straightforward to implemented, others probably not so much. I'll be patient ;-))

0. Treat motion photos as regular photos that happen to include a bit of extra data. (some photos have geotags, some have a short video clip attached...)
1. Show an overlay icon to indicate that the file actually includes audio and video. (Samsung motion photos are clearly tagged as such in their XMP metadata, so detection should be straightforward at least for this manufacturer.)
2. Possibility to filter by motion photo presence. (I.e. in the "Filter" tab on the right, either like the geotag presence filtering or in the filetype dropdown menu)
3. Possibility to just get rid of the included video. (For quality or privacy reasons.)
4. Possibility to remove the audio track of the included video but keep the video. (For privacy reasons.)
5. Take care that an included video is not discarded by accident when the image is rewritten. (E.g. by changing the metadata tags or modifying the image in the built-in image editor. An export that would lose the video could maybe emit a warning.)
6. Possibility to play back the included video. (Optionally by just hovering over the photo, as in the photoprism demo linked above. But that would probably entail creating and storing a preview video. This could be out of scope for a photo *management* app.)
Maybe: 7. Possibility to extract the video as a separate file.

If you need sample pictures I'll happily provide Samsung sample pictures. (Galaxy S10e and Galaxy S23, with and without "high efficiency" coding)

I can extract the MP4 video from the JPG image from my Galaxy S8 using the following EXifTool command:

exiftool -b -Samsung:EmbeddedVideoFile image.jpg > image.mp4

It would be interesting to see if anything has changed with the current Galaxy model.

Maik

(In reply to Maik Qualmann from comment #39)
> exiftool -b -Samsung:EmbeddedVideoFile image.jpg > image.mp4
> 
> It would be interesting to see if anything has changed with the current
> Galaxy model.

This is working fine for the motion photos stored as jpeg created by the default Samsung camera app on my S23. The video is extracted as .mp4 and playback is fine. If I choose to store pictures in a "high efficiency" format, i.e. as HEIC, exiftool creates a broken 12 byte mp4. No error message, just a silently broken file. But this could be due to the fact that I'm still on Ubuntu 22.04 and waiting for 24.04. (exiftool 12.40).