Bug 221547 - Metadata should be saved in extended attributes
Summary: Metadata should be saved in extended attributes
Status: RESOLVED UNMAINTAINED
Alias: None
Product: nepomuk
Classification: Miscellaneous
Component: general (show other bugs)
Version: unspecified
Platform: Compiled Sources Linux
: NOR wishlist
Target Milestone: ---
Assignee: Sebastian Trueg
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-01-06 17:24 UTC by Rafał Rzepecki
Modified: 2015-01-23 16:25 UTC (History)
4 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Rafał Rzepecki 2010-01-06 17:24:10 UTC
Version:            (using Devel)
OS:                Linux
Installed from:    Compiled sources

Currently nepomuk metadata is only stored in the ontology. Using dedicated ontology storage is certainly necessary, but it has the problem of being system-local; moving or copying objects (specifically, filesystem objects) across machines destroys all accumulated metadata in the process. This makes semantic desktop a lot less reliable; if I want semantic information to replace filesystem hierarchy as means of organizing data, I need for this organization to be transferable as easily as filesystem hierarchy is.

This could be alleviated by storing a copy of metadata in the file extended attributes, support for which is nearly ubiquitous in modern operating system and many of their tools. The xattrs needn't (and shouldn't) be primary storage for this information, but only a duplicate to allow transparent transfer of some metadata along with objects when transferring between boxes.

('Some', because obviously only metadata not needing external local referents are of any use here. Ie., tags and ratings certainly are ok, source-email-id (or whatever the name) -- not necessarily, although email-ids are, perhaps, unique enough. In short, only metadata which can be uniquely resolved when read as key-value pairs could and should be stored that way.)

The way I see it would work is that, whenever a property is assigned to a filesystem object it would also be stored in its xattrs; the xattrs would be read by strigi on scanning and fed into local ontology. (Strigi could also watch files for xattr changes to keep xattrs and ontology in sync, effectively enabling usage of standard xattr tools for limited ontology manipulation.)
Comment 1 Kubuntiac 2011-10-06 00:57:24 UTC
Agreed. Until metadata becomes highly persistent *even when moving files across different computers / os's), few people will bother to put the time needed into adding the most important metadata to their files, blocking Nepomuk's full potential.

I suggest a combination of:

1. Adding metadata to files extended attributes where possible
2. Using sidecar files (ie storing metadata for files in .directory or a file unique to indexed files like .filename.extension.metadata ). Adjust KDE to make sure that if someone copies file foo.bar that the appropriate sidecar files are edited / copied with it.
3. Using the current database style storage

This way, 
1. makes much of the data available, even on non KDE systems
2. ensures that a folder copied to another kde system (even if copied onto a non-kde system first) keeps it's metadata for automatic reimporting.
3. Can be used for speed when doing things like desktop search

Related wish: Bug #269365
Comment 2 Thiago Jung Bauermann 2012-12-31 13:55:15 UTC
One idea is to save an UUID as an extended attribute in the file, and reference that UUID in the Nepomuk database. That is how Amarok tracks files and it seems to work well:

http://techbase.kde.org/Amarok/AmarokFileTracking

The OS moves and copies extend attributes along with the file, so there's no need to adapt KDE or KIO to transport the nepomuk metadata along with the file, which would be suboptimal and fragile since people don't use only KDE tools to manage their files.

To enable the metadata to be transfered to another computer, the metadata related to the files can be stored in a database in the removable media and again referenced by the UUID.

Not all filesystems support external attributes, but a lot of them do and it would be a waste not to take advantage of that when available to enable such an important feature.
Comment 3 Vishesh Handa 2015-01-23 16:25:48 UTC
Thank you for taking the time to file a bug report.

The Nepomuk project is no longer included in the KDE Software Compilation. With Plasma 5, we have replaced most of the underlying technology with Baloo and other components. Hopefully this will have addressed your concern.

We encourage you to try to Plasma 5 (+Baloo) and let us know if your problem persists.