Bug 397742

Summary: Silent data loss when using a program that deletes and re-creates the document after every save operation when the document is located on a samba share accessed with KIO/Dolphin
Product: [Frameworks and Libraries] frameworks-kio Reporter: tempel.julian
Component: generalAssignee: David Faure <faure>
Status: RESOLVED FIXED    
Severity: critical CC: aleixpol, bronsonmathews, bugs.kde.org, bugseforuns, cyberbeat, edera, jens-bugs.kde.org, jtamate, kde, kdelibs-bugs, kilgore.trout, luzius, matthewtrescott, nate, omarplummer
Priority: VHI Keywords: usability
Version: git master   
Target Milestone: ---   
Platform: Arch Linux   
OS: Linux   
Latest Commit: Version Fixed In: 5.51
Sentry Crash Report:
Attachments: 1
2

Description tempel.julian 2018-08-22 12:49:08 UTC
Browse an SMB share with Dolphin with a LibreOffice document in it and open it.
Edit the file, e.g. add some characters. Then save the file and close LO.
Open the document again -> the changes haven't been saved. There is no warning for the user about this, the changes to the document get just silently lost.

LibreOffice 6.1 ships with a KDE 5 filepicker. When opening files on SMB shares with it, changes actually get saved. Maybe this can be useful to improve the situation with Dolphin?
Comment 1 Nate Graham 2018-08-22 16:42:18 UTC
So I presume that you using a version of LibreOffice prior to 6.1? Does this reproduce with any other non-KDE apps? For example does the same thing happen with Inkscape, GIMP, or Audacity?
Comment 2 tempel.julian 2018-08-22 17:02:16 UTC
I'm with LO 6.1 and gtk3_kde5 VCL.

I tried KWrite and KolourPaint, they both save correctly on the SMB share.
Comment 3 Nate Graham 2018-08-22 17:04:13 UTC
Yeah, I would expect (well, hope) that KDE apps behave properly. What I'd like to know is whether non-LibreOffice GTK apps have the same problem that you're having the LibreOffice. Can you test Inkscape, GIMP, or Audacity?
Comment 4 tempel.julian 2018-08-22 17:14:02 UTC
I'm sorry, I've just realized I missed that part you wrote.

GTK apps don't seem to react so well:
Mousepad text editor doesn't save directly and instead triggers a Dolphin prompt if I want to replace the file. The change gets saved, but it's not comfortable.

Gedit: It silently saves to "~/.cache/kioexec/krun/2557_0/file.txt", the change doesn't get saved to the actual SMB share. So apparently the same behavior as LO.
Comment 5 tempel.julian 2018-08-22 17:25:15 UTC
Inkscape: The same as Mousepad. After pressing the save button, it recognizes that the file has changed (in KIO cache?) and asks if it should be reuploaded. If the reupload prompt is answered with yes, the Dolphin replace prompt appears.
Comment 6 Nate Graham 2018-08-22 17:44:59 UTC
> Mousepad text editor doesn't save directly and instead triggers a Dolphin
> prompt if I want to replace the file. The change gets saved, but it's not
> comfortable.

> Inkscape: The same as Mousepad.
OK, this is (unfortunately) exactly what is supposed to happen. I know it's not ideal, but at least there's no data loss. We're tracking improvements to that with Bug 40115.

> Gedit: It silently saves to "~/.cache/kioexec/krun/2557_0/file.txt", the
> change doesn't get saved to the actual SMB share. So apparently the same
> behavior as LO.

Aha. So we've found the problem: apps that use GNOME's GIO don't work because they don't communicate with KIO. As a result, KIO doesn't know, "hey, this is a local file that needs to be re-uploaded to the server," and GIO thinks "oh hey, I have no concept of local vs remote files because I assume that all remote files are locally mounted with GvFS". So LibreOffice saves the file to the cache and KIO for some reason never finds out that it needs to be re-uploaded to the server.

I suspect that the reason why this just started happening with LibreOffice is because LO 6.1 gained native GIO support via the GTK3 VCL plugin. So now it no longer talks to KIO.

I see three possible solutions here:
1. LibreOffice gains native KIO support too. (Would fix LibreOffice only)
2. We figure out some way to have KIO notice that the file has changed in the cache anyway even when the app is using GIO (would fix all GIO-using apps)
3. The GNOME devs patch GIO so that it notices when it's being asked to save a file to a KIO cache and it either alerts KIO to this so that KIO can re-upload it, or else it silently redirect the save operation to the actual mounted location (would fix all GIO-using apps, but seems unlikely for both political and technical reasons).

Let's forget about #3 for now and use this Bugzilla ticket to track #2. Can you track down the ticket for #1 on the LibreOffice bug tracker, or barring that, file one? Thanks!
Comment 7 tempel.julian 2018-08-22 17:53:17 UTC
I'll gladly suggest to them to implement native KIO support for the given reasons and link the ticket here.
Thanks for your very gentle and timely support!
Comment 8 Nate Graham 2018-08-22 18:12:17 UTC
You're very welcome. One more request: could you share a screenshot of the prompt that appears asking whether or not you want to save the edited file back to the samba share?
Comment 9 tempel.julian 2018-08-22 20:01:38 UTC
Created attachment 114550 [details]
1
Comment 10 tempel.julian 2018-08-22 20:01:48 UTC
Created attachment 114551 [details]
2
Comment 11 tempel.julian 2018-08-22 20:02:44 UTC
The file replacement dialog says that the target, which is about to be replaced, would be newer than the to be copied file. That's a bit misleading.
Comment 12 Nate Graham 2018-08-22 20:17:26 UTC
Thanks. I'll dig through the code and see if I can find whether there's an easy fix for some of this...
Comment 13 tempel.julian 2018-08-22 20:19:59 UTC
Bug on the LO tracker:
https://bugs.documentfoundation.org/show_bug.cgi?id=119429
Comment 14 Nate Graham 2018-08-30 13:54:28 UTC
*** Bug 398057 has been marked as a duplicate of this bug. ***
Comment 15 Jaime Torres 2018-08-30 15:12:55 UTC
In a simple test:
gimp and inkscape work with .png and .jpg (I'm asked if I want to overwrite the modified file in the samba server), but gvim, emacs, libreoffice and winrar(using wine) doesn't work, even gvim modifying the same .jpg image as gimp.
Comment 16 Nate Graham 2018-08-30 15:40:42 UTC
Thanks so much! If you examine the KIO cache directory that the file is being read from, can you see if these programs are doing something like swapping the file out for a backup file or something? I wonder if KIO is getting confused by some activity on the cache file because the app isn't expecting that its file is in a local cache or something.
Comment 17 Nate Graham 2018-08-30 16:08:17 UTC
*** Bug 383724 has been marked as a duplicate of this bug. ***
Comment 18 Kai Uwe Broulik 2018-08-30 16:10:39 UTC
I can reproduce with gedit, open a file on a remote location (e.g. smb or sftp) with gedit, it downloads the file but never prompts to reupload when saved.

The issue lies in kioexecd which is monitoring the file temp file for changes. It only watches for KDirWatch::dirty. However, gedit deletes and recreate the file. When the file is removed, the watcher is removed and it doesn't see that the file reappeared later.

Ideally, it would also handle a delete-and-create cycle as "dirty", but care must be taken not to leak the watchers (e.g. finally discard a file when it hasn't reappared within 30 seconds or so)
Comment 19 Jaime Torres 2018-08-30 16:38:43 UTC
Would it be enough if kioexecd monitors the temporary directory for changes instead of the copied file in the temporary directory?
It could also have the advantage or annoyance of being asked if you want to copy the backup files.
Comment 20 Nate Graham 2018-09-03 21:49:40 UTC
There's an open patch that fixes this: https://phabricator.kde.org/D15180

Hopefully it'll get in for Frameworks 5.51!
Comment 21 tempel.julian 2018-09-04 17:03:10 UTC
Great!

Btw: There is a tool called samba-mounter, which integrates itself into Plasma system settings:
https://cgit.kde.org/scratch/afiestas/samba-mounter.git
https://aur.archlinux.org/packages/samba-mounter-git/

It seems to work just fine and allows LibreOffice to successfully edit files when started out of Dolphin.
Since it's real handy, perhaps it can officially be integrated into Plasma?
Comment 22 Nate Graham 2018-09-04 17:17:13 UTC
(In reply to tempel.julian from comment #21)
> Great!
> 
> Btw: There is a tool called samba-mounter, which integrates itself into
> Plasma system settings:
> https://cgit.kde.org/scratch/afiestas/samba-mounter.git
> https://aur.archlinux.org/packages/samba-mounter-git/
> 
> It seems to work just fine and allows LibreOffice to successfully edit files
> when started out of Dolphin.
> Since it's real handy, perhaps it can officially be integrated into Plasma?

Aleix, would you care to comment on this? :)
Comment 23 Jaime Torres 2018-09-06 14:40:43 UTC
Git commit 820f622e86bb0b7d44a39a3f90f84c99736b30c7 by Jaime Torres Amate.
Committed on 06/09/2018 at 14:40.
Pushed by jtamate into branch 'master'.

kioexecd: watch for creations or modifications of the temporary files

Summary:
When a non KIO friendly program opens a non local file, the file is copied to an user temporary folder by kioexec.
Watch any creation or modification or deletion of that temporary file, because some programs delete the old file and create a new one when saving the file. When the file is recreated or modified, the user is asked if he/she wants to upload the modified file.

If the temporary file keeps deleted more than 30s then forget about that file and try to delete the temporary directory.

Test Plan:
Tested logging out and in several times and opening two or three remote files with xed and libreoffice.
After removing manually some of the temporary files and wait around ~31s the empty temporary directories are deleted.

Reviewers: #frameworks, broulik, ngraham, dfaure, elvisangelaccio

Reviewed By: dfaure, elvisangelaccio

Subscribers: bruns, anthonyfieroni, elvisangelaccio, kde-frameworks-devel

Tags: #frameworks

Differential Revision: https://phabricator.kde.org/D15180

M  +41   -3    src/kioexec/kioexecd.cpp
M  +7    -0    src/kioexec/kioexecd.h

https://commits.kde.org/kio/820f622e86bb0b7d44a39a3f90f84c99736b30c7
Comment 24 Nate Graham 2018-11-05 21:42:22 UTC
*** Bug 369122 has been marked as a duplicate of this bug. ***
Comment 25 Nate Graham 2018-11-05 21:42:43 UTC
*** Bug 397704 has been marked as a duplicate of this bug. ***
Comment 26 Nate Graham 2018-11-05 21:43:02 UTC
*** Bug 377033 has been marked as a duplicate of this bug. ***
Comment 27 Nate Graham 2018-11-05 21:43:50 UTC
*** Bug 390058 has been marked as a duplicate of this bug. ***
Comment 28 Nate Graham 2018-11-05 21:44:08 UTC
*** Bug 378546 has been marked as a duplicate of this bug. ***
Comment 29 Nate Graham 2018-11-05 21:44:15 UTC
*** Bug 400691 has been marked as a duplicate of this bug. ***
Comment 30 Nate Graham 2018-11-05 21:44:18 UTC
*** Bug 380349 has been marked as a duplicate of this bug. ***
Comment 31 Nate Graham 2018-11-07 00:50:46 UTC
*** Bug 377033 has been marked as a duplicate of this bug. ***