Bug 433822 - DigiKam doesn't read extended (xattr) attributes
Summary: DigiKam doesn't read extended (xattr) attributes
Status: RESOLVED FIXED
Alias: None
Product: digikam
Classification: Applications
Component: Database-Baloo (show other bugs)
Version: 7.2.0
Platform: Neon Linux
: NOR normal
Target Milestone: ---
Assignee: Digikam Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-03-01 21:47 UTC by tagwerk19
Modified: 2021-04-10 13:41 UTC (History)
3 users (show)

See Also:
Latest Commit:
Version Fixed In: 7.2.0


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description tagwerk19 2021-03-01 21:47:12 UTC
SUMMARY:

    Digikam has a new option to read/write information from the
    extended attributes.

    It seems to be able to write attributes such as "rating" and
    "tags" but not read them

STEPS TO REPRODUCE:

    Tell digiKam that it should look at extended attributes, go to the
    "Settings / Configure digiKam / Metadata / Baloo" and set: 

        Store metadata from digiKam in Baloo
        Read metadata from Baloo

    and under "... / Behaviour / Write this information to the Metadata"
    select:

        Image tags
        Rating

    and also under "... / Behaviour / Reading and Writing Metadata",
    select:

        Update file modification timestamp when files are modified
        Rescan file when files are modified 

    Create new test image, add a "three star" rating to a new image
    with Dolphin or Gwenview and "mytag" as a tag. Check this with getfattr:

        getfattr -d testfile.jpg

        # file: testfile.jpg
        user.baloo.rating="6"
        user.xdg.tags="mytag"

    Run digiKam and find the image...

OBSERVED RESULTS

    The image is shown without a rating and no apparent tags.

EXPECTED RESULTS

    As per the "Read metadata from Baloo", it should pick up these values

SOFTWARE/OS VERSIONS

    Neon Unstable

    DigiKam : 7.2.0 (build date 2021-03-01)
    Plasma : 5.21.80
    Frameworks : 5.80.0
    Qt : 5.15.2 

ADDITIONAL INFORMATION

    If the rating and tags are changed within digiKam, they are
    written to the extended attributes
Comment 1 Maik Qualmann 2021-03-02 06:56:46 UTC
Reading the Baloo information works without any problems. Just tested here. However, Baloo has the lowest (last) priority. If digiKam finds a valid rating in the metadata, it will be used, even if it is a zero rating.

Maik
Comment 2 Maik Qualmann 2021-03-02 07:05:45 UTC
We won't change this behavior either, Baloo is deactivated by default and we actually wanted to remove it completely. Even if the stability has improved in the current KF5 versions, Baloo has brought us many bug reports with crashes over the years.

Maik
Comment 3 Maik Qualmann 2021-03-02 07:13:19 UTC
If you only want to use Baloo, you can switch off reading of the rating tags in the advanced metadata settings, then we will only read Baloo.

Maik
Comment 4 tagwerk19 2021-03-02 08:20:57 UTC
Thank you for the replies...

> ... Baloo has brought us many bug reports with crashes
> over the years ...
I don't think I was using it in the bad years. Now, from my perspective, I think "hat's off". It's an incredibly messy and challenging project.

My feeling here is that digiKam is reading/writing xattr on the filesystem, the data you can see with "getfattr -d". Baloo reads this data, digiKam can read and write this data. For this bit of the work it _seems_ the two are independent.

> ... you can switch off reading of the rating tags in the
> advanced metadata settings ...
It could be that I've written changes done within digiKam back into the embedded metadata, and digiKam is reading this info in preference to (modified) extended attributes. I will check. I am happy to carry on troubleshooting

As an initial suggestion. If people see:

    Read metadata from Baloo

there'll be an expectation that digiKam will try to read the info. Could there be an option in the advanced settings, to show xattr as a source - and list things like user.baloo.rating, user.xdg.tags etc explicitly, although by default unchecked, at the bottom of the list?

Further on perhaps, if "Read metadata from Baloo" is selected, these options could be checked
Comment 5 Maik Qualmann 2021-03-02 20:21:40 UTC
Git commit 583c39033bec33e6bcca7e475ebb9d9d91a8f2c9 by Maik Qualmann.
Committed on 02/03/2021 at 20:20.
Pushed by mqualmann into branch 'master'.

cleanup Baloo API and fix QString memory leak

M  +5    -4    core/libs/fileactionmanager/metadatahub.cpp
M  +6    -21   core/utilities/extrasupport/filesindexer/baloowrap.cpp
M  +1    -7    core/utilities/extrasupport/filesindexer/baloowrap.h

https://invent.kde.org/graphics/digikam/commit/583c39033bec33e6bcca7e475ebb9d9d91a8f2c9
Comment 6 Maik Qualmann 2021-03-02 20:47:31 UTC
Git commit cf9f68128d5f5fe87d0fec67a912ca8a39703c7a by Maik Qualmann.
Committed on 02/03/2021 at 20:46.
Pushed by mqualmann into branch 'master'.

fix sync rating value from Baloo
FIXED-IN: 7.2.0

M  +2    -1    NEWS
M  +6    -1    core/libs/database/item/scanner/itemscanner_baloo.cpp

https://invent.kde.org/graphics/digikam/commit/cf9f68128d5f5fe87d0fec67a912ca8a39703c7a
Comment 7 Maik Qualmann 2021-03-02 21:11:42 UTC
Git commit 67426dc24ab1a4c2c1b1528258fba63dbcf7bcb6 by Maik Qualmann.
Committed on 02/03/2021 at 21:11.
Pushed by mqualmann into branch 'master'.

factoring Baloo API

M  +5    -7    core/libs/fileactionmanager/metadatahub.cpp
M  +7    -10   core/utilities/extrasupport/filesindexer/baloowrap.cpp
M  +7    -9    core/utilities/extrasupport/filesindexer/baloowrap.h

https://invent.kde.org/graphics/digikam/commit/67426dc24ab1a4c2c1b1528258fba63dbcf7bcb6
Comment 8 tagwerk19 2021-03-03 15:50:03 UTC
> I am happy to carry on troubleshooting

Yes, my test file had had the metadata written back into them, that and the advanced settings meant that reading extended attributes would have been "at the bottom of the list"

Testing, step by step (using the "old build" of 2021-03-02)

    Configured digiKam metadata settings, leaving 'Advanced' setting
    as default, close digiKam

    Created clean test files (confirming no embedded metadata), set Tags
    and Ratings values in xattr for some of the test files

    Ran digiKam, confirm that it picked up the images.

        ... It had not picked up the xattr values

        ... but that can be explained by the default 'Advanced' settings

    Reconfigured digiKam settings, clearing the selections to
    read/write embedded metadata under 'Advanced' for tags and ratings

    Changed xattr Tags/Ratings (via Dolphin), reran digiKam.

        ... Changes were not seen. Ohh... :-(

    Updated timestamps on all test files, with "touch *" ...

        ... Changes became visible after an F5 refresh. Ahhh :-)

    Changed tags and ratings within digiKam (for one test file)

        ... extended attributes also changed (Good)

So that's reasonably clear. Need to be careful about embedded metadata when testing, it's not obvious at what point and in what situations it gets written to the file. Need to know about the "Advanced' settings and also realise that changing extended attributes is "invisible" to digiKam; it checks when it sees the timestamp changes.

Equivalent run with today's build (2021-03-03) to follow....
Comment 9 tagwerk19 2021-03-03 16:05:48 UTC
Repeating for today's, 2021-03-03, build

    Configured digiKam metadata settings, leaving 'Advanced' settings
    as default, close digiKam. 

    Created clean, new test files (without embedded metadata), set Tags
    and Ratings values in xattr for some of the files as before...

    Ran digiKam, confirm that it picked up the images, and ...

        ... It *has* picked up the xattr values.

That's good stuff :-)

    Changed the Tags and Ratings values with Dolphin, checked in
    digiKam 

        ... Changes were not seen. Ohh... :-(

    Update the timestamps ...

        ... Changes became visible after an F5 refresh. Ahhh :-)

So digiKam should ideally(!) read the extended attributes while it is reading the modification time in order to pick up changes. I'll submit this as a 'wishlist' change

I've not looked to see what happens when an image file has both embedded metadata and external attributes. That's probably something to come back to...
Comment 10 Maik Qualmann 2021-03-03 17:42:54 UTC
> So digiKam should ideally(!) read the extended attributes while it is
> reading the modification time in order to pick up changes. I'll submit this
> as a 'wishlist' change

No, digiKam will definitely not read the Baloo information when the modification date is read. digiKam also doesn't read metadata every time. You will not be able to avoid re-reading the metadata. And F5 does not read in the metadata again, it creates new thumbnails and finds new and changed elements based on the modification date.

Maik
Comment 11 tagwerk19 2021-03-03 20:01:34 UTC
Sorry, I was trying to be very careful (and making sure I was getting repeatable results). It's possible though that my wording was amiss.

The 2021-03-03 build works far better, thank you for the quick patches

The 'gotcha' with extended attributes is that if you change them (in Dolphin/Gwenview/etc) that does not change the modification time of the file. The file itself is not changed.

In order to follow changes in the extended attributes, you _also_ need to change the modification time and do a refresh. Does that make sense? Or am I missing something?

I also had a look to see if I could see any dependencies on baloo as a service. If I run the tests with the baloo services disabled and the database deleted, it seems not to make a difference. DigiKam is just reading the extended attributes...