Bug 161403 - Nepomuk Tags are lost when copying a file
Summary: Nepomuk Tags are lost when copying a file
Status: RESOLVED UNMAINTAINED
Alias: None
Product: nepomuk
Classification: Miscellaneous
Component: filewatch (show other bugs)
Version: 4.1
Platform: Compiled Sources Linux
: NOR normal
Target Milestone: ---
Assignee: Nepomuk Bugs Coordination
URL:
Keywords:
: 152777 304100 320164 (view as bug list)
Depends on:
Blocks:
 
Reported: 2008-04-29 10:53 UTC by Benoît Jacob
Modified: 2015-01-23 16:22 UTC (History)
18 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Benoît Jacob 2008-04-29 10:53:27 UTC
Version:            (using Devel)
Installed from:    Compiled sources
OS:                Linux

How to reproduce:
1) launch dolphin
2) select some file
3) in the 'information' panel, add some comment and some tag to that file
4a) copy that file, paste to another directory
4b) drag-and-drop that file to another directory, select "Copy Here"
4c) drag-and-drop that file to another directory, select "Move Here"
5) in any of the above 3 ways 4a,4b,4c, the resulting file does not remember the comment and tags of the original file.

I think that at least for file operations done by KDE 4.x (x>=1) applications it would be nice (and reasonable to expect) that nepomuk metadata be preserved.
Comment 1 Sebastian Trueg 2008-04-29 14:19:07 UTC
please try playground/base/nepomuk-kde/filewatch and tell me about your experiences.
Comment 2 Benoît Jacob 2008-05-04 08:31:01 UTC
kde4@kiwi:~/filewatch-build$ cmakekde /home/gaston/cuisine/trunk/filewatch/
-- Check for working C compiler: /usr/bin/gcc
-- Check for working C compiler: /usr/bin/gcc -- works
-- Check size of void*
-- Check size of void* - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
CMake Error: Error in cmake code at
/home/gaston/cuisine/trunk/filewatch/kinotify/CMakeLists.txt:14:
Unknown CMake command "kde4_add_library".
-- Configuring done


I tried for an hour copying what the main KDE modules do in order to use "kde4_add_library", without success.
Comment 3 Sebastian Trueg 2008-05-05 09:12:55 UTC
well, to compile it you need the kde4 devel environment. And also it does not work with KDE 4.0.x anymore.
Comment 4 Benoît Jacob 2008-05-05 10:00:31 UTC
I do have a current kde devel environment, and this is against current kde trunk.

Could it be that I shouldn't run cmake directly on the filewatch/ directory, and should instead run it on some parent directory such as nepomuk-kde.... I'll try.
Comment 5 Sebastian Trueg 2008-05-05 10:17:11 UTC
> Could it be that I shouldn't run cmake directly on the filewatch/
> directory, and should instead run it on some parent directory such as
> nepomuk-kde.... I'll try.


Yes, sorry, I forgot to mention that. :)
Comment 6 Benoît Jacob 2008-05-05 10:21:27 UTC
Now it builds (sidenote: in nepomuk-kde/pimoshell/resourcepropertydelegate.cpp, I had to comment out the line kDebug() << parent << option << index; because operator<< wasn't defined for QStyleOption).

How do I use it? Should I manually enable something, or just re-login into KDE?

I can't really test because of a separate problem -- so I am very grateful if you see a solution: a few days ago, nepomukservices started eating 100% CPU constantly (and more and more RAM). This makes dolphin completely unusable (response times above 10s) so I had to disable nepomuk, and now when I re-enable it I still have the same problem. So I can't test in Dolphin right now. Maybe you would at least know how to reset nepomuk to its original state before that bug appeared?
Comment 7 Sebastian Trueg 2008-05-05 10:30:25 UTC
> How do I use it? Should I manually enable something, or just re-login into
> KDE?


It will automatically be used after next login, yes.

> I can't really test because of a separate problem -- so I am very grateful
> if you see a solution: a few days ago, nepomukservices started eating 100%
> CPU constantly (and more and more RAM). This makes dolphin completely
> unusable (response times above 10s) so I had to disable nepomuk, and now
> when I re-enable it I still have the same problem. So I can't test in
> Dolphin right now. Maybe you would at least know how to reset nepomuk to
> its original state before that bug appeared?


Is Strigi enabled? Does "ps aux|grep strigi" show the "strigidaemon"?
Comment 8 Benoît Jacob 2008-05-05 10:37:02 UTC
Strigi wasn't enabled when that bug appeared. Today when I re-enabled Nepomuk I also enabled Strigi, and got the same result. So it seems strigi-independent.

I just re-enabled nepomuk to check: CPU usage went again to 100%, and strigi wasn't running (according to ps aux).

I have only a few files nepomuk-tagged in my home dir.

FYI, here is part of the cmake output in kdesupport:

-- Soprano Components that will be built:                                       
   * Redland storage backend                                                    
   * Raptor RDF parser                                                          
     (including TriG parser)                                                    
   * Raptor RDF serializer                                                      
   * The CLucene-based full-text search index library                           

-- Soprano Components that will NOT be built:
   * Sesame2 storage backend (java-based)    
Comment 9 Sebastian Trueg 2008-05-05 10:45:37 UTC
Could you please debug into the process taking all the cpu to see what it is 
doing?
Comment 10 Benoît Jacob 2008-05-05 11:30:21 UTC
You mean run it in gdb ? Then what is the appropriate command-line, just "gdb nepomukservices" ? nepomukservices is something that KDE launched for me and I am just not sure how to properly launch it myself. Also, is it OK to run it while nepomuk is disabled in the KCM?
Comment 11 Sebastian Trueg 2008-05-05 11:50:22 UTC
I thought about debugging into the running 100% cpu process.
You can do that by issuing:

gdb nepomukservicestub <pid>
Comment 12 Benoît Jacob 2008-05-08 17:57:08 UTC
Today I did svn up and rebuilt kde trunk, but in "debugfull" mode (formerly I used "release" mode).

I also did svn up in nepomuk-kde from playground, rebuilt and installed, and re-logged into kde. In systemsettings, I enabled nepomuk but not strigi.

The second problem (nepomuk eating 100% CPU) is gone: nepomuk now only takes negligible CPU time. I don't know if it's because you fixed it, or if it's because this issue only exists in "release" mode (which would be strange!) I am willing to test but wouldn't like to rebuild all of kde again, so I'd need to know what parts to rebuild, if you're interested.

Unfortunately, the problem that I describe in the original text of this bug report, still exists: metadata is lost when moving a file (i tried both moving and copying in dolphin).

Here are the nepomuk-related processes running:

gaston@kiwi:~$ ps aux | grep nepomuk | grep -v grep
gaston   10221  0.0  1.1  67336 11512 ?        Sl   17:41   0:00 /home/kde4/kde/bin/nepomukserver
gaston   10241  0.8  1.4  41760 14636 ?        S    17:41   0:07 /home/kde4/kde/bin/nepomukservicestub nepomukstorage
gaston   10246  0.0  1.0  32612 10696 ?        S    17:41   0:00 /home/kde4/kde/bin/nepomukservicestub nepomukalignment
gaston   10248  0.4  1.3  36792 14408 ?        S    17:41   0:03 /home/kde4/kde/bin/nepomukservicestub nepomukfilewatch
gaston   10249  0.0  0.9  27540  9468 ?        S    17:41   0:00 /home/kde4/kde/bin/nepomukservicestub nepomukontologyloader
Comment 13 Alessandro Morandi 2008-06-19 10:22:06 UTC
I can confirm that Nepomuk-related metadata do not follow the file when it is moved, renamed or deleted.

If you
1. Create a file
2. Associate some comment/tag/rating to it
3. Delete it
4. Create a new file with the same name
you will see the metadata associated to the new file, while it should have been discarded along with the deleted one.

I'm currently experiencing this behavior on Kde4Daily, built against SVN revision 821684.
Comment 14 Sebastian Trueg 2009-01-19 20:39:06 UTC
it should work fine whenever KIO is used to move or delete the files. Moving or deleting on the command line or via gnome tools is not yet handled. I am still open for good suggestions here. Maybe even something which guesses move locations and presents some gui where the user can then choose.
Or even better: a kernel patch that allows us to track all file operations. ;)
Comment 15 Benoît Jacob 2009-01-20 15:19:40 UTC
Frankly, I believe that this is a very important issue: I don't see myself tagging files if the metadata can be lost at anytime if the files are moved by a non-KDE application.

What is the status of inotify support? Isn't that the kind of kernel support that was needed? (totally uninformed about inotify)
Comment 16 Icekiss 2009-08-17 19:28:43 UTC
I can confirm that copying or moving tagged files with Dolphin 1.3 under KDE 4.3.0 does _not_ preserve meta information.
I actually have no idea at all how to copy/move files in a way that takes meta informations with them (I tried it with konqueror as well.)
In my case, I tagged a few pictures with gwenview, and then wanted to move the folder they are in. (I tried to copy a single file too, just as a test).

To reproduce the problem:
1. tag a picture in gwenview
2. move the file with dolphin
3. open with gwenview again -> tags gone
( 4. move file back -> tags are there again)
Comment 17 Icekiss 2009-08-17 19:35:14 UTC
Damnit, correction:
Copy the file, do not move it.
Moving a single file with Dolphin does indeed preserve meta information.
It's the first action I found to actually do that (Moving the whole folder did not). So I actually found a solution for my current problem: Just mark all files in the folder, and move them that way.
That does not mean however, that all those other cases (moving folders, copying files, ...) should not work as well.
Comment 18 Sebastian Trueg 2009-08-17 19:35:37 UTC
please provide the output of "qdbus|grep nepomuk"
Comment 19 Icekiss 2009-08-19 22:47:37 UTC
$ qdbus|grep nepomuk
 org.kde.nepomuk.services.nepomukstorage
 org.kde.nepomuk.services.nepomukfilewatch
 org.kde.nepomuk.services.nepomukontologyloader
 org.kde.nepomuk.services.nepomukqueryservice
Comment 20 Sebastian Trueg 2009-08-20 15:47:12 UTC
So you want a copy have the same metadata as the original?
I thought about that but came to no good conclusion. So I decided to not do it.
As for moving folders: that should work and I will have a look.
Comment 21 Benoît Jacob 2009-08-20 15:55:50 UTC
To me, preserving the metadata across copy operations is the "obviously" wanted behavior; I wonder what can be arguments against this?

Just an example, if nothing else, the copy operation could be made to archive important personal files; then the user also wants to keep the metadata.

There is also the consistency-with-other-metadata-systems argument: e.g. tags embedded in music files are preserved when the file is copied, and this has set a standard of what the user expects.

Finally, the user may reasonably assume that moving a file has the same effect as copying it and then deleting the original. (Of course it's very different in the way it works, and moving is much faster, but most users don't know that).
Comment 22 Icekiss 2009-08-20 17:32:56 UTC
>So you want a copy have the same metadata as the original?

Yes, I do. Ideally, I would want the metadata to be embedded within the file, so that no matter what tool I use to move, copy, backup, ... the file, the metadata is preserved. As long as that is not the case, I must use only kde applications to work with the files.
But at least these applications I expect to treat the metadata as part of the file. So yes, I do want the copy to have the same metadata as the original.

What argues against that?
Comment 23 mutlu inek 2009-09-14 06:45:05 UTC
I agree that metadata needs to be copied with a file copy, everything else does not make much sense to me.
Comment 24 GJ 2010-02-05 09:49:27 UTC
I agree that the meta data should be preserved when copying the file. To me that is expected behaviour.
Comment 25 mutlu inek 2010-06-07 22:39:13 UTC
I just tested this with KDE SC 4.4.82 (4.5 beta 2). The good news is that tags are now persistent: moving and renaming files does not destroy the metadata!

However, while all four commenters above agreed that it is obviously desirable to copy metadata as well when copying files, this is not the case.

Sebastian, thank you for fixing this. But could you please consider to change the behavior and copy metadata as well? [Use case/example: I can't see why I would want the copy of a picture tagged "Frank" to be disassociated from the name since the person is still in the image.]

Should we open a new bug report for this so this one can be closed?
Comment 26 Sebastian Trueg 2010-06-08 11:49:51 UTC
Copying metadata is not trivial as there are multiple ways to copy a file in KDE. It may even be necessary to patch certain applications in order to get the desired behavior.
Comment 27 Thiago Jung Bauermann 2010-08-07 17:24:41 UTC
I saw an idea in another bug report of writing an UUID to the file's xattr, then nepomuk could use the UUID to find the file's metadata. Wouldn't that work?
Comment 28 Sebastian Trueg 2011-01-06 15:40:44 UTC
*** Bug 152777 has been marked as a duplicate of this bug. ***
Comment 29 Patrick Shanahan 2011-05-13 22:01:51 UTC
I too, feel that metadata should follow a file.  If one wants different metadata for two copies of the same file, they should change the name of the 2nd file.
Comment 30 Richard Bos 2011-05-13 22:41:48 UTC
I just encountered this problem recently.  I've:

# nepomukserver --version
Qt: 4.7.3
KDE Development Platform: 4.6.3 (4.6.3)
Nepomuk-server: 0.2


# qdbus|grep nepomuk
 org.kde.digikamnepomukservice-11053
 org.kde.nepomuk.services.digikamnepomukservice
 org.kde.nepomuk.services.nepomukqueryservice
 org.kde.nepomukqueryservice-11057
 org.kde.nepomuk.services.nepomukfilewatch
 org.kde.nepomukfilewatch-11061
 org.kde.nepomuk.services.nepomukbackupsync
 org.kde.nepomukbackupsync-11054
 org.kde.nepomuk.services.nepomukremovablestorageservice
 org.kde.nepomukremovablestorageservice-11059
 org.kde.nepomuk.services.nepomukontologyloader
 org.kde.nepomuk.services.nepomukstorage
 org.kde.nepomukstorage-10737
 org.freedesktop.Akonadi.Agent.akonadi_nepomuk_contact_feeder
 org.kde.akonadi_nepomuk_contact_feeder-10759

I asked about it on the openSUSE kde emaillist first:
http://lists.opensuse.org/opensuse-kde/2011-05/msg00015.html and was pointed to this bug report.

In short, I moved many files with dolphin from folder A to folder B and the tags (metadata) does not move with the file.  After reading this bug report I tried to move only one file, using Dolphin, from folder A to folder B with the same result.

I really hope that this functionality can be made to work.
Comment 31 Richard Bos 2011-05-14 09:29:56 UTC
For the people that think that this bug / feature is important to solve, it might be a good idea to give your votes to this bug report.
Comment 32 Will Stephenson 2011-05-17 16:13:34 UTC
Before anyone suggests "fs.inotify.max_user_watches = 524288" to increase the number of folders KDirWatch can monitor for moves, each inode watched is un-swappably pinned into memory - 1 or 4 KB each, i don't remember exactly.
Comment 33 Benoît Jacob 2011-05-17 16:17:24 UTC
Can someone please remind me (also see comment 27) why xattrs couldn't be used to store metadata in a way that ensures that it follows files?

If xattrs doesn't work, then, doesn't that mean that it would be useful to work with kernel developers to implement something what would work?
Comment 34 Benoît Jacob 2011-05-17 16:23:41 UTC
To be clear: as time passes, I am less and less interested in a solution that works only for KIO, and more and more thinking that it's vital to find a solution that integrates with the system VFS. Even people like me who are full-time KDE users may use cp/mv commands, and with a KIO-only solution, that would lose metadata. So if xattrs is not good enough then surely kernel developers should be interested in making a better replacement for it? I'm thinking about Linus' claims about how the desktop is where things are really interesting. Time to put his desktop commitment to the test ;-)
Comment 35 Richard Bos 2011-05-22 12:08:24 UTC
@Benoît: it is indeed desired to have a general solution, but this looks like outside the scope of this bug report.  Let's use this bug report to obtain a KIO only solution and have another (new) bug report / feature request to track the general solution that you propose / requires in comment #23.  If you agree (and hopefully you  do) will you open a new bug report requesting the general solution?
Comment 36 Frank Reininghaus 2012-08-03 06:42:29 UTC
*** Bug 304100 has been marked as a duplicate of this bug. ***
Comment 37 Vishesh Handa 2012-12-31 07:41:05 UTC
Update on the bug -

Tags are preserved when moving files (This has been the case for quite some time now). They are not preserved when copying because it is simply not possible on a technical level. Maybe we could integrate into KIO for file copies, but we have no way of doing it at the kernel level, nor will that ever be possible.

There is already bug 221547 for saving the metadata in the extended attributes.

Marking this bug as CONFIRMED. We still need someone to implement the added integration with KIO for copying.
Comment 38 Thiago Jung Bauermann 2012-12-31 14:05:09 UTC
I think using an UUID in the extended attributes to reference files in the Nepomuk database would make it easier to copy the Nepomuk metadata along with the file. I describe it a bit in bug 221547 comment 2.

At the moment the file is copied, there would be two files with the same UUID and thus referencing the same metadata. That wouldn't be a problem since then Nepomuk could be implement a copy on write scheme: when editing metadata information that is referenced by two files, then a new UUID is generated for the file being edited and the metadata is copied and associated with that UUID.
Comment 39 Jekyll Wu 2013-05-23 12:02:04 UTC
*** Bug 320164 has been marked as a duplicate of this bug. ***
Comment 40 Vishesh Handa 2015-01-23 16:22:17 UTC
Thank you for taking the time to file a bug report.

The Nepomuk project is no longer included in the KDE Software Compilation. With Plasma 5, we have replaced most of the underlying technology with Baloo and other components. Hopefully this will have addressed your concern.

We encourage you to try out Plasma 5 (+Baloo) and let us know if your problem persists.