Bug 449637

Summary: Freeze due to Error in Exiv2::XMLValidator
Product: [Applications] digikam Reporter: Josh <jkotlerj>
Component: Metadata-XmpAssignee: Digikam Developers <digikam-bugs-null>
Status: RESOLVED FIXED    
Severity: crash CC: caulier.gilles, metzpinguin
Priority: NOR    
Version: 7.5.0   
Target Milestone: ---   
Platform: Microsoft Windows   
OS: Microsoft Windows   
Latest Commit: Version Fixed In: 8.2.0
Sentry Crash Report:

Description Josh 2022-02-05 03:33:48 UTC
SUMMARY
***
NOTE: If you are reporting a crash, please try to attach a backtrace with debug symbols.
See https://community.kde.org/Guidelines_and_HOWTOs/Debugging/How_to_create_useful_crash_reports
***
While scrolling through images in Preview mode, clicking on (seemingly) random images causes a freeze ("Not responding") that lasts several seconds. Using DebugView shows that some error occurs in Exiv2:

============
[4280] digikam.metaengine: Exiv2 ( 3 ) :  XMP Toolkit error 201: Error in XMLValidator
[4280] 
[4280] digikam.metaengine: Exiv2 ( 2 ) :  Failed to decode XMP metadata.
[4280] 
[4280] digikam.metaengine: index     :  0
[4280] digikam.metaengine: properties:  2
[4280] digikam.metaengine: Exiv2 ( 3 ) :  XMP Toolkit error 201: Error in XMLValidator
[4280] 
[4280] digikam.metaengine: Exiv2 ( 2 ) :  Failed to decode XMP metadata.
============

However, after a few seconds digiKam responds again and displays the image just fine. Afterwards, clicking on the same image does not seem to cause similar issues. 

In the attachments I've added an image which caused the same behavior. As noted above, though, the issue doesn't seem to happen when clicking this image again later.

STEPS TO REPRODUCE
1. Select preview mode
2. Begin scrolling through the images, clicking on different as you go (doesn't have to be in order).

OBSERVED RESULT
digiKam hangs on certain images, displays a "Not responding" status. Comes back after a few seconds, continues to work fine until the same behavior repeats.

EXPECTED RESULT
digiKam should not hang on any of the photos as they are all valid, straight-out-of-camera Fujifilm raw files.

SOFTWARE/OS VERSIONS
Windows: Windows 11 Home, Version 21H2, Build 22000.469

ADDITIONAL INFORMATION
Comment 1 Maik Qualmann 2022-02-05 07:29:17 UTC
The XMP-XML structure in these images is incorrect. The Adobe XMP-SDK reports an error and mostly cannot read the XMP metadata. Possibly the Adobe SDK that Exiv2 uses is too old. We already know the problem (including freezing) from other Windows users. Which Fujifilm camera are the images from exactly to reproduce the problem? 

Maik
Comment 2 Josh 2022-02-05 15:40:47 UTC
(In reply to Maik Qualmann from comment #1)
> The XMP-XML structure in these images is incorrect. The Adobe XMP-SDK
> reports an error and mostly cannot read the XMP metadata. Possibly the Adobe
> SDK that Exiv2 uses is too old. We already know the problem (including
> freezing) from other Windows users. Which Fujifilm camera are the images
> from exactly to reproduce the problem? 
> 
> Maik

All of the photos I tried were taken with Fujifilm X-S10, firmware ver. 2.0

It seems like my sample image didn't attach. Please, let me know if there's a standard way to attach large files, as it seems that the RAW file is too big to attach directly here. Also, I'd be happy to provide anything else that can be useful.
Comment 3 caulier.gilles 2022-02-05 15:46:04 UTC
Bugzilla attachment sizes is limited.

Use cloud link instead, as gdrive, dropbox, etc...

Gilles Caulier
Comment 4 Josh 2022-02-05 16:05:09 UTC
Here are two files which have caused the issue: https://www.dropbox.com/sh/0dpx0p0032k9290/AADS-gx2IynOc6OS8ly2uKjva?dl=0
Comment 5 caulier.gilles 2022-02-05 17:25:48 UTC
Hi,

The digiKam 7.6.0 pre-release includes a rolling release of Exiv2 from github. Code is updated each day while compiling the Windows installer.

https://i.imgur.com/8icqXqH.png

Please try with this new version to see if problem is fixed in Exiv2.

Installer is located here : https://files.kde.org/digikam/

Thanks in advance

Gilles Caulier
Comment 6 Josh 2022-02-05 17:42:41 UTC
(In reply to caulier.gilles from comment #5)
> Hi,
> 
> The digiKam 7.6.0 pre-release includes a rolling release of Exiv2 from
> github. Code is updated each day while compiling the Windows installer.
> 
> https://i.imgur.com/8icqXqH.png
> 
> Please try with this new version to see if problem is fixed in Exiv2.
> 
> Installer is located here : https://files.kde.org/digikam/
> 
> Thanks in advance
> 
> Gilles Caulier

Hello, thank you for the suggestion.

I've replaced my instance of digiKam with today's build of 7.6.0, and confirmed that Exiv2 snapshot from today (Feb. 5 2022) is being used. The first image I opened has caused a freeze. After further investigation, it seems as though every image I open causes the same error to show up in DebugView. However, freezes only occur occasionally and I was not able to identify any difference in the logs for files that cause freezes vs. those that do not.
Comment 7 Maik Qualmann 2022-02-05 20:41:58 UTC
I can also reproduce the problem with samples from the web using the Exiv2 CLI tool. The XML seems fine:

<x:xmpmeta xmlns:x="adobe:ns:meta/">
    <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
        <rdf:Description rdf:about="" xmlns:xmp="http://ns.adobe.com/xap/1.0/">
            <xmp:Rating>0</xmp:Rating>
        </rdf:Description>
    </rdf:RDF>
</x:xmpmeta>

A test with ExifTool and active validation shows a warning in the Fujifilm RAW file:

exiftool -xmp:all -validate -warning -error DSCF2022.RAF
Rating                          : 0
Validate                        : 1 Warning
Warning                         : Value for FujiIFD tag 0xf010 VignettingParams overlaps tag 0xf00f ChromaticAberrationParams

I have tested all my Fujifilm sample images, which are from different cameras, all are valid. 
My guess is that Exiv2 doesn't find the XML block start correctly. The problem would have to be reported to Exiv2 along with the test image.

Maik
Comment 8 Maik Qualmann 2022-04-24 15:03:19 UTC
Git commit 802d92a3ec46e2ca1b60aa0e2e9bdb8516d4f041 by Maik Qualmann.
Committed on 24/04/2022 at 14:58.
Pushed by mqualmann into branch 'master'.

add read Exiv2 warnings and errors
to then read the metadata with ExifTool
Related: bug 452567, bug 450117, bug 432265, bug 446363

M  +6    -0    core/libs/metadataengine/engine/metaengine_fileio.cpp
M  +8    -0    core/libs/metadataengine/engine/metaengine_p.cpp
M  +1    -0    core/libs/metadataengine/engine/metaengine_p.h

https://invent.kde.org/graphics/digikam/commit/802d92a3ec46e2ca1b60aa0e2e9bdb8516d4f041
Comment 9 caulier.gilles 2023-05-02 06:18:52 UTC
@Josh

 digiKam 8.0.0 is out. This entry still valid with this release ?

Best regards

Gilles Caulier
Comment 10 caulier.gilles 2023-05-12 14:50:16 UTC
Maik,

This file can be closed now ? You patch prevent a digiKam crash ?

Gilles
Comment 11 caulier.gilles 2023-05-16 03:34:44 UTC
Git commit 3d71acc72593534c2a1398d2945992fbbf45935a by Gilles Caulier.
Committed on 16/05/2023 at 03:00.
Pushed by cgilles into branch 'master'.

Switch bundles from Exiv2 0.27 maintenance branch to new 0.28 release
Related: bug 386291, bug 386291, bug 468830

M  +11   -8    project/bundles/3rdparty/ext_exiv2/CMakeLists.txt

https://invent.kde.org/graphics/digikam/commit/3d71acc72593534c2a1398d2945992fbbf45935a
Comment 12 caulier.gilles 2023-05-20 13:44:05 UTC
@josh

digiKam 8.1.0 pre-release Windows bundle is now ported to Exiv2 0.28 which
come with a huge list of bugfixes :

https://github.com/Exiv2/exiv2/issues/2406#issuecomment-1529139799

Installer file is available here :

https://files.kde.org/digikam/

In case of Exiv2 bug fixed with this version, please give us a feedback.

Thanks in advance

Gilles Caulier
Comment 13 caulier.gilles 2023-10-15 09:54:31 UTC
@josh,

This problem still reproducible with the new digiKam 8.2.0 pre-release Windows
installer available at usual place:

https://files.kde.org/digikam/

This new bundle is based on last Qt framework 5.15.11 and KDE framework 5.110.

Thanks in advance

Gilles Caulier
Comment 14 caulier.gilles 2023-11-19 11:18:49 UTC
This crash is not reproducible with Exiv2 0.28.1 compiled under Windows with VCPKG. digiKam 8.2.0 pre-release installer is here :

https://files.kde.org/digikam/unstable/

Gilles Caulier