| Summary: | Temporary files (for previewing files in the trash or remote files) aren't deleted | ||
|---|---|---|---|
| Product: | [Unmaintained] kio | Reporter: | Alvaro Aguilera <alvaro.aguilera> |
| Component: | general | Assignee: | David Faure <faure> |
| Status: | RESOLVED FIXED | ||
| Severity: | normal | CC: | a.t.page, aaugusto, adawit, ares, david.marin.mty, DJm00n, elvis.angelaccio, finex, hawake, ht990332, karl.r.ernst, kde.bugzilla.2012, kde, kde, lamarque, lambdae2, lameventanas, mail, martin.frueh, rlerallut, serhiy.int, toresoft, unggnu, vo.zaeb, willemw12 |
| Priority: | NOR | ||
| Version: | 4.8 | ||
| Target Milestone: | --- | ||
| Platform: | unspecified | ||
| OS: | Linux | ||
| Latest Commit: | https://commits.kde.org/kio/1620032772465be475ae0746aff63a566ef2a546 | Version Fixed In: | 5.35 |
| Sentry Crash Report: | |||
|
Description
Alvaro Aguilera
2009-09-26 19:10:39 UTC
Way to reproduce this braindamage: 1. rm /tmp/kde-user/dolphin* 2. open dolphin and split the window 3. go to a remote machine via SFTP 4. copy a file from the remote machine to the local one I did this for a file with a size of 700MB, at 250MB copied this was the situation: /home/file.part ==> 250MB /tmp/kde-user/ ===> 0 dolphinB13600.tmp 348M dolphinB13600.tmp.part 0 dolphinC13600.tmp 343M dolphinC13600.tmp.part 0 dolphinD13600.tmp 347M dolphinD13600.tmp.part 0 dolphinE13600.tmp 9.7M dolphinE13600.tmp.part 0 dolphinH13600.tmp 138M dolphinH13600.tmp.part 0 dolphinI13600.tmp 108M dolphinI13600.tmp.part 0 dolphinL13600.tmp 97M dolphinL13600.tmp.part 0 dolphinN13600.tmp 285M dolphinN13600.tmp.part 0 dolphinb13600.tmp 294M dolphinb13600.tmp.part 0 dolphind13600.tmp 344M dolphind13600.tmp.part 0 dolphine13600.tmp 335M dolphine13600.tmp.part 0 dolphinl13600.tmp 286M dolphinl13600.tmp.part 0 dolphinm13600.tmp 287M dolphinm13600.tmp.part 0 dolphino13600.tmp 148M dolphino13600.tmp.part 0 dolphins13600.tmp 351M dolphins13600.tmp.part 0 dolphint13600.tmp 293M dolphint13600.tmp.part 0 dolphinu13600.tmp 96M dolphinu13600.tmp.part 0 dolphinz13600.tmp 114M dolphinz13600.tmp.part yes dolphin was coping the file 18 "additional" times. As a bonus, with a cpu utilization of 50%, even after closing dolphin and stopping the copy. I don't like to criticize, but if there is some money left at KDE e.V. they should put some dollars dolphin and make it work. At version 4.3.1 it's already time to have a reliable file manager. To be able to copy a file shouldn't be this problematic, not on a "state of the art file manager" like dolphin. This bug looks quite strange, can you reproduce using other protocols like samba or fish? Using smb and fish dolphin creates a lot of temporary files, but all are very small (even zero length). About the high CPU utilization it should be a problem of the notification plasmoid, would you try to disable the plasma notification and try to copy a big file using sftp? Thanks! I can only access the remote machine through SSH, either with sftp:// or fish:// and the problem happens with both. Unfortunately I can't test it with any other protocol. I don't think the CPU utilization is the fault of the plasmoid. Would you like to disable the notification protocol for file operation and try it anyway? Thanks. what exactly do you want me to disable? the "kded notification item watcher"? Right click on the system tray, and un-select "File transfer and other jobs". Thanks :-) disabling the notification is somewhat better, instead of the 18, only 3 extra copies are created on /tmp/kde-user: 0 dolphinH21511.tmp 0 dolphinK21511.tmp 0 dolphinl21511.tmp 268M dolphinl21511.tmp.part 0 dolphinr21511.tmp 267M dolphinr21511.tmp.part 0 dolphinV21511.tmp 265M dolphinV21511.tmp.part 0 dolphinw21511.tmp 0 dolphinW21511.tmp 0 dolphinx21511.tmp the cpu-utilization remains high, with several kio-sftp's taking around 20% each. same with me. protocol - smb. KDE4.3.1 Same problem here, dolphin 1.3 (KDE 4.3.2), protocol samba Same problem in KDE 4.3.3 (openSUSE 11.2 RC1) when browsing samba shares. Probably dolphin downloads remote files to create previews. It's not a problem when it downloads 1 file and creates 1 tmp file, but when it tries to create previews for several files at the same time, browsing is inconvenient (cpu load is too high). May be restriction on previews count is needed? In data domenica 08 novembre 2009 17:35:51, Dmitry Kholodilov ha scritto:
> when it tries to create previews for several files at the same time,
> browsing is inconvenient (cpu load is too high). May be restriction on
> previews count is needed?
An alternative could be disabling thumbs on remote filesystem.
> > May be restriction on previews count is needed? > An alternative could be disabling thumbs on remote filesystem. Yes, it also refers to bug 211007: https://bugs.kde.org/show_bug.cgi?id=211007 I have a similar problem in which whole directory trees can be found in /tmp/kde-username/dolphinFUBAR.tmp/ , but they don't contain any file ! Only empty directories. I haven't tracked down the suspect but it's probably due to browsing SMB shares with dolphin (and smb://). It took at least 20+ minutes) to cleanup /tmp, which caused boot "failures" (delays, actually) and much anguish among my coworkers that share this machine. I can't believe its been more than 4 months and such a serious bug is still in UNCONFIRMED state. This reminds me of KDE3, in which some bugs were there for years, and by the time some of them were resolved, KDE3 was not supported anymore. @alancio: the key is that nobody do this for work. If developers have time to spend on this project, they can fix bugs (if they want), otherwise patch are always appreciated :-) P.S: you could give your vote to this bug in order to underline the severity. P.P.S: I've finished my available votes :-) P.P.P.S: I've reproduced this bug too, using RC version of KDE 4.4. Can you reproduce with KDE 4.4 too? I can confirm this bug using the smb protocol with kde 4.3.5. next week i will try to confirm it using kde 4.4! it already happened two times that dolphin completely filled up my root partition (around 13G temporary dolphin* files), which caused plasma to frequently crash :-) sadly, no "disk full" or other warnings appeared - this might be a real problem for users without enough computer knowledge. a solution which works for me is to disable the information sidebar. FiNeX: Yes, I used to do the voting thing, bug reporting, et al with KDE3, but like you said, developers don't really care because they are not paid for this, they do it for fun, and I guess supporting KDE3 is not fun anymore. I just come here to check if anybody has a useful patch, I'm tired of my bug reporting and votes being ignored, but knock yourself out. *** This bug has been confirmed by popular vote. *** I can confirm this bug with kde 4.4.1, my wild guess is that this bug is related with browsing samba shares. my girlfriend had this problem a few minutes ago on her kubuntu 9.10 install (dolphin 4:4.3.5-0ubuntu1~karmic1). she was deleting local files, no samba involved at all! the files she deleted where not saved on the /-filesystem, but mounted at /data. she rebooted, so i cant provide and more details right now. Dolphin creates several empty files here with format /tmp/kde-lamarque/dolphinXXXXXX.tmp when I browse my cell phone using bluetooth (bluedevil), just browsing, I do not even need to open a file. still present in 4.4.5 :-/ Still present in 4.8, I just had to delete 5Gb worth of dolphin temporal files. Cannot reproduce this at all. I tried three different remote protocols ftp, sftp and smb with dolphin and it does not create a single temp file under /tmp/kde-<user>/ for me. Same thing with Konqueror. Nothing. Do any of you still see this issue in KDE v4.10 or is this something that was fixed? still present for me in KDE SC 4.11.3 arch linux Still present in KDE: 4.13.1 Temporal files can also grow very big and fill the partition. Confirming with KDE 4.13.3 on Ubuntu 14.04: I just got a "no space left on device" for /tmp and had to delete two dolphin temp directories from /tmp/user/1000/kde-username The two directories were dolphinK11352.tmp and dolphinH11352.tmp - note the related names. Each of them contained 1.2 GB of files; They were files have once been put into trash a while ago. The directories contained a similar of equal copy of the files. The trash in dolphin was empty at the time. Over the last couple of months on my desktop, I accumulated 100+ GB in /tmp/kde-user... i.e. most of my SSD was being used by that single folder. This is with dolphin/KDE 4.13.2, and many accesses to smb shares. A good solution would probably be to automatically delete those tmp files in FIFO order, keeping only the newest X MB or GB (where X is a value in settings). Until we have something like that, I created a cronjob to purge that directory nightly, which should keep it down to just a few GB. I believe I found out how to reproduce this bug, these temp files are generated whenever I visit the trash:/ folder with dolphin, and generates new ones for every time I press F5 in there. If image thumbnails aren't enabled, then they don't take much space, but if you enable it and you have a bunch of images then they become quite large. How much space the temp files take depends of the contents of the trash, since it seems to be copying them over. In short, - delete some files and images (so the trash isn't empty) - go to trash:/ - enable thumbnails - see how dolphin creates temp files and never deletes them. Qt: 4.8.6 KDE Development Platform: 4.13.3 This bug is particularly bad for those of us with /tmp as tmpfs. I just discovered I had 1.3 GiB of Dolphin temp files sucking up my RAM. I run without disk-based swap, too — swap only on zram devices — so this is really terrible. Hi, I've found the same bug on KDE 4.11.5 on openSUSE 13.1. I've noticed that moving those tmp files to trash:/ through Dolphin itself, loads the CPU for the 70% to 90%. While deleting those files from trash:/ the whole system is really slow, i had to use rm from a terminal window. openSUSE 13.2 (Harlequin) (x86_64) 3.16.7-24-desktop x86_64 GNU/Linux Qt: 4.8.6 KDE: 4.14.9 Dolphin: 15.04.0 I can confirm that this behavior too. If open the Trash:/ folder with preview (thumbnails) activated, dolphin starts to create plenty of dirs inside of /tmp/kde-user dir, and in seconds (not minutes!) it fills 4GB. Then all the other applications complains or misbehave, due to the lack of space in /tmp. I've also reported this misbehavior in other related bugs, because this is something really complicated to address as user. Implies stopping the dolphin file manager (also this happens with konqueror when browsing Trash:/ ) and manually remove the files inside /tmp And as far as I believe that thumbnais is a very useful feature, resources granted to this process (cpu cycles / thread / processes / tmp space) must be somehow tuned in order not to overload the machine resources. Still 100%-reproducible on Dolphin 16.04.3 (kubuntu backports PPA) I don't see it as files not being removed but as the files being copied to /tmp Why would you do that? Please close this bug as a duplicate: https://bugs.kde.org/show_bug.cgi?id=238309 OK, let's see. First, trying to reproduce Alvaro's initial steps (with the path in step 1 adjusted for Kf5) : 1. rm /tmp/dolphin* 2. open dolphin and split the window 3. go to a remote machine via SFTP 4. copy a file from the remote machine to the local one (using drag-n-drop) I copied a 102 MB file and a 3.7 MB file and no temp file was created, not even during the copy. Then Dmitry's comment says it might be about previews, I tried enabling previews in the toolbar, works locally, no effect on sftp. This is because Dolphin's preferences dialog says "Skip previews for remote files over 0MB". If I change that to 10 MB, then indeed I get a preview of the image I uploaded - but still no dolphin temp file left over in /tmp. Now trying the same with trash:/, preview works, no temp file left over. Now looking at the code: there is a QTemporaryFile in KIO::PreviewJob, obviously, for remote files where the thumbnail plugin doesn't support KIO. But the temp file is deleted in the next step. KIO::PreviewJobPrivate::getOrCreateThumbnail: Using temp file "/tmp/dolphin.B12772" KIO::PreviewJob::slotResult: Deleting temp file "/tmp/dolphin.B12772" (additional debug output I just added) So a temp file leak would only happen if the first step happens but not the second step. Reading the code, I don't see how this can ever happen. There is no code path where we wouldn't get to the second step. The only course of action I can suggest is that someone who can reproduce the bug applies the patch below to their kio build and reports the results. http://www.davidfaure.fr/2017/previewjob.cpp.diff *** Bug 238309 has been marked as a duplicate of this bug. *** *** Bug 352975 has been marked as a duplicate of this bug. *** *** Bug 272249 has been marked as a duplicate of this bug. *** (In reply to David Faure from comment #34) > The only course of action I can suggest is that someone who can reproduce > the bug applies the patch below to their kio build and reports the results. > > http://www.davidfaure.fr/2017/previewjob.cpp.diff Okay, I applied it and reproduced the bug. When I browse to "trash:" and enable thumbnails, I get thousands of these log messages: Using temp file "/tmp/dolphin.c25868" KIO::TransferJob(0x2bb5350) for "/tmp/dolphin.c25868" Deleting temp file "/tmp/dolphin.c25868" After all the log activity subsides, I find that 1.3 GiB are now consumed in two directories inside /tmp. Both of these directory paths are reported in "Deleting temp file" messages (multiple times, presumably because the temp file names are reused). Are you certain that QFile::remove(const QString &) supports recursively removing a non-empty directory? Oops, no, it definitely doesn't. Well done, that's the issue. This code was written at the time where only files were getting previewed. Now it's getting triggered for entire directories too. I'm not sure we want that -at all- for non-local directories (trash is non local too, the way it's implemented). There's a choice between two possible fixes here: - properly removing temp dirs - skipping dirs for previews (In reply to David Faure from comment #39) > Oops, no, it definitely doesn't. Well done, that's the issue. This code was > written at the time where only files were getting previewed. Now it's > getting triggered for entire directories too. I'm not sure we want that -at > all- for non-local directories (trash is non local too, the way it's > implemented). > > There's a choice between two possible fixes here: > - properly removing temp dirs > - skipping dirs for previews I vote for the second fix, better safe than sorry. One can always explicitly enter in the remote directory, to trigger the previews of the files in there. Git commit 78c45a1ea0e28a98f34c6d113c807f14700b22d4 by David Faure. Committed on 14/05/2017 at 13:41. Pushed by dfaure into branch 'master'. PreviewJob: clean up empty temp file when get() fails. (e.g. because it's a directory) M +13 -4 src/widgets/previewjob.cpp https://commits.kde.org/kio/78c45a1ea0e28a98f34c6d113c807f14700b22d4 Git commit 1620032772465be475ae0746aff63a566ef2a546 by David Faure. Committed on 14/05/2017 at 13:48. Pushed by dfaure into branch 'master'. PreviewJob: skip remote directories. Too expensive to preview. For some protocols, file_copy() would end up copying the whole directory locally! FIXED-IN: 5.35 M +6 -0 src/widgets/previewjob.cpp https://commits.kde.org/kio/1620032772465be475ae0746aff63a566ef2a546 |