Bug 391747 - BQM Tool "Remove Metadata" doesn't remove all metadata from image
Summary: BQM Tool "Remove Metadata" doesn't remove all metadata from image
Status: RESOLVED FIXED
Alias: None
Product: digikam
Classification: Applications
Component: Plugin-Bqm-RemoveMetadata (show other bugs)
Version: 5.8.0
Platform: Fedora RPMs Linux
: NOR normal
Target Milestone: ---
Assignee: Digikam Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-03-12 10:44 UTC by Taras
Modified: 2018-03-24 21:30 UTC (History)
2 users (show)

See Also:
Latest Commit:
Version Fixed In: 6.0.0


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Taras 2018-03-12 10:44:30 UTC
Steps to reproduce:
1. Run BQM queue/workflow with enabled tool "Remove Metadata" (all Exif, Iptc, Xmp checkboxes are checked) on an image
2. See what metadata result image does contain, e.g. running exiftool

Actual result:

Expected result:
In output of exiftool on final image (exiftool img_4181_pub.jpg) see information at least about:
1. Processing software

Processing Software             : digiKam-5.8.0                              
Software                        : digiKam-5.8.0
Creator Tool                    : digiKam-5.8.0
Originating Program             : digiKam

2. Image History (!)

Image History                   : <?xml version="1.0"?>.<history version="1">.    <file type="original">.        <fileParams fileName="img_4181.jpg" filePath="/home/.../" fileHash="e22321c46119d4e5f1df00e8dd216570" fileSize="8373479" creationDate="2018-02-22T19:22:03"/>.    </file>.    <filter filterName="transform:resize" filterDisplayName="Resize" filterVersion="1" filterCategory="reproducible" branch="true">.        <params>.            <param name="height" value="682"/>.            <param name="width" value="1024"/>.        </params>.    </filter>.</history>.

3. Original and final filenames

File Name                       : img_4181_pub.jpg
Document Name                   : img_4181.jpg
Comment 1 caulier.gilles 2018-03-12 10:53:44 UTC
For point 1/

1. Processing software

Processing Software             : digiKam-5.8.0                              
Software                        : digiKam-5.8.0
Creator Tool                    : digiKam-5.8.0
Originating Program             : digiKam

==> Fixed in next 6.0.0

Point 2/ ==> did you put the remorve metadata tool at end of list in your queue ?

Point 3/ ==> both are mostly generated automatically by exiftool at run time. Try to run Exiv2 CLI tool instead.

Gilles Caulier
Comment 2 Taras 2018-03-12 11:24:13 UTC
Gilles,

> 1. Processing software
> ==> Fixed in next 6.0.0
Great!

> Point 2/ ==> did you put the remorve metadata tool at end of list in your queue ?

Yes. I also tried to place it as the first one but the result is the same. It looks like it removes only well-known (or hardcoded) tags.

> Point 3/ ==> both are mostly generated automatically by exiftool at run time. Try to run Exiv2 CLI tool instead.

If so where and how did exiftool get name of original filename?
Comment 3 caulier.gilles 2018-03-12 11:39:28 UTC
>If so where and how did exiftool get name of original filename?

I not sure. Please check with Exiv2 to show XMP or IPTC or Exif metadata with real tags name in the trace list from the console, just to be sure. After all we use Exiv2 in background with DK...

Gilles Caulier
Comment 4 Taras 2018-03-12 16:04:08 UTC
$ exiv2 --version
exiv2 0.26 001a00 (64 bit build)

Exiv2 CLI output with **summary** of the image Exif metadata.

$ exiv2 img_4181_pub.jpg    
File name       : img_4181_pub.jpg
File size       : 381292 Bytes
MIME type       : image/jpeg
Image size      : 1024 x 682
Camera make     : 
Camera model    : 
Image timestamp : 
Image number    : 
Exposure time   : 
Aperture        : 
Exposure bias   : 
Flash           : 
Flash bias      : 
Focal length    : 
Subject distance: 
ISO speed       : 
Exposure mode   : 
Metering mode   : 
Macro mode      : 
Image quality   : 
Exif Resolution : 1024 x 682
White balance   : 
Thumbnail       : None
Copyright       : 
Exif comment    : 


Let's grep for special tags:

$ exiv2 -g Image -pt img_4181_pub.jpg 
Exif.Image.ProcessingSoftware                Ascii      14  digiKam-5.8.0
Exif.Image.ImageWidth                        Long        1  1024
Exif.Image.ImageLength                       Long        1  682
Exif.Image.DocumentName                      Ascii      13  img_4181.jpg
Exif.Image.Software                          Ascii      14  digiKam-5.8.0
Exif.Image.ExifTag                           Long        1  128

$ exiv2 -g History -pa img_4181_pub.jpg 
Xmp.digiKam.ImageHistory                     XmpText   576  <?xml version="1.0"?>
<history version="1">
    <file type="original">
        <fileParams fileName="img_4181.jpg" filePath...
Comment 5 Taras 2018-03-18 19:12:39 UTC
May be it is because ImageHistory stores & saves independently of metadata? I mean that here in removemetadata (https://github.com/KDE/digikam/blob/9330f9c1e25ce754f374abac840bef2521a032ca/utilities/queuemanager/tools/metadata/removemetadata.cpp#L159)


        if (removeExif || removeIptc || removeXmp)
        {
            image().setMetadata(meta.data());
        }

ret = savefromDImg();

digiKam set metadata to empty value(-s) but then savefromDImg object is returned.

In savefromDImg() declaration (https://github.com/KDE/digikam/blob/9330f9c1e25ce754f374abac840bef2521a032ca/utilities/queuemanager/manager/batchtool.cpp#L400) we can see usage of prepareMetadataToSave() method. There is saving of ImageHistory in this last method (https://github.com/KDE/digikam/blob/e114506b7fe6be38727fc33edbba0757f0196989/libs/dimg/dimg.cpp#L3095)   

if (!m_priv->imageHistory.isEmpty())
    {
        DImageHistory forSaving(m_priv->imageHistory);
        forSaving.adjustReferredImages();

        QUrl url         = QUrl::fromLocalFile(intendedDestPath);
        QString filePath = url.adjusted(QUrl::RemoveFilename | QUrl::StripTrailingSlash).toLocalFile() + QLatin1Char('/');
        QString fileName = url.fileName();

        if (!filePath.isEmpty() && !fileName.isEmpty())
        {
            forSaving.purgePathFromReferredImages(filePath, fileName);
        }

        QString imageHistoryXml = forSaving.toXml();
        meta.setImageHistory(imageHistoryXml);
}

So it looks like on the final stage digiKam adds history information as Xmp.digiKam.ImageHistory tag into image with already stripped metadata.
Comment 6 Taras 2018-03-22 14:44:51 UTC
May be it could be fixed by adding code for removing image history into removemetadata() function?
Comment 7 Maik Qualmann 2018-03-22 16:39:24 UTC
It is difficult to prevent the metadata creating not via the DImg save function. However, if you include the metadata removal tool as the only tool in the list, all metadata should be removed. Please test.

Maik
Comment 8 Taras 2018-03-23 10:04:03 UTC
Maik,

> However, if you include the metadata removal tool as the only tool in the list, all metadata should be removed. Please test.

Yes, but it will be too "synthetic scenario". This ticket is about privacy issue. Is someone includes "Remove metadata" tool as last step into BQM workflow then this user hopes to get image with empty metadata in the final. E.g. for publishing somewhere. Nothing like Image History should exists in it.
Comment 9 caulier.gilles 2018-03-23 10:56:50 UTC
Well

The ideal solution is to remove the RemoveMetadata tool and put a generic option in Queue settings to remove metadata, done at end of each file set in Queue.

Gilles Caulier
Comment 10 Maik Qualmann 2018-03-23 11:36:44 UTC
Git commit 59fbe560e917fe1e6466dc52cac3f5528628b00a by Maik Qualmann.
Committed on 23/03/2018 at 11:35.
Pushed by mqualmann into branch 'master'.

save DImg before remove metadata

M  +11   -10   core/utilities/queuemanager/tools/metadata/removemetadata.cpp

https://commits.kde.org/digikam/59fbe560e917fe1e6466dc52cac3f5528628b00a
Comment 11 Maik Qualmann 2018-03-23 11:37:53 UTC
Only the software version still appears. I'll take a look at this evening.

Maik
Comment 12 Maik Qualmann 2018-03-23 21:36:30 UTC
Git commit 9fe4884583d1ce12cf91e5ad5218ffc439eef002 by Maik Qualmann.
Committed on 23/03/2018 at 21:35.
Pushed by mqualmann into branch 'master'.

add a flag to DMetadata::save() to disable saving the program version
FIXED-IN: 6.0.0

M  +2    -1    NEWS
M  +2    -2    core/libs/dmetadata/dmetadata.cpp
M  +1    -1    core/libs/dmetadata/dmetadata.h
M  +2    -2    core/libs/dmetadata/metaengine.cpp
M  +1    -1    core/libs/dmetadata/metaengine.h
M  +8    -15   core/utilities/queuemanager/tools/metadata/removemetadata.cpp

https://commits.kde.org/digikam/9fe4884583d1ce12cf91e5ad5218ffc439eef002
Comment 13 Taras 2018-03-24 21:30:22 UTC
Thanks!