Bug 207361 - KIO is failing to detect file(filelist) changes with Kernel 2.6.31 (inotify reimplementation)
Summary: KIO is failing to detect file(filelist) changes with Kernel 2.6.31 (inotify r...
Status: RESOLVED FIXED
Alias: None
Product: kio
Classification: Frameworks and Libraries
Component: general (show other bugs)
Version: unspecified
Platform: Unlisted Binaries Linux
: NOR normal with 143 votes (vote)
Target Milestone: ---
Assignee: David Faure
URL:
Keywords:
: 206890 210284 210952 211778 212838 214073 216641 216838 (view as bug list)
Depends on:
Blocks:
 
Reported: 2009-09-14 17:30 UTC by Mikes
Modified: 2010-02-02 12:15 UTC (History)
20 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Mikes 2009-09-14 17:30:26 UTC
Version:            (using KDE 4.3.1)
OS:                Linux
Installed from:    Unspecified Linux

When i copy or delete files to desktop, they do not appear or disappear automaticaly and i have to manualy refresh the desktop with f5 or right click on the desktop.

i have to mention that i have folder view enabled.

My system is archlinux 64 bit, and this issue is new for my system. The strange fact is that it did not use to happen since the first day i used kde 4.3.1 and i noticed that few days ago. I cannot remember what might have changed in order to cause that problem
Comment 1 Mikes 2009-09-14 19:03:10 UTC
the problem only seems to be happening with kernel 2.6.31. i downgraded to kernel 2.6.30.6 and the issue disappeared.

how is it possible that the newer kernel causes this problem?


ps. apart from the kernel package downgrade i also downgraded the nvidia driver package, but it is just the same nvidia version built for another kernel.
Comment 2 Mikes 2009-09-14 19:37:05 UTC
reading the changelog of 2.6.31 from 

http://kernelnewbies.org/LinuxChanges#head-0c30611399be8bbb1b2c618211b8bf2d342527fb

i read about a reimplementation of inotify and dnotify on top of a new filesystem notification infrastructure, so it seems that this is the reason for the problem.
Comment 3 Mikes 2009-09-16 12:06:03 UTC
i did some extra tests in order to see if it's nvidia fault.

1. i installed nvidia-beta driver 190.32 driver
2. i tested nv driver from xorg
3. i disabled composite and i deleted my xorg.conf file

in all the above i kept having the same problem using kernel 2.6.31.
Comment 4 Dario Andres 2009-09-17 01:45:57 UTC
If it is a kernel thing it should be affecting all the applications using KDirWatch.

Can you test if this also happens on Dolphin ? (example, open a folder in Dolphin, from the outside of the application (a shell, ex. Konsole) try to remove or add a file and see if the Dolphin file list is updated too.

If it is not, then we should move this to kdelibs. 
Thanks
Comment 5 koso 2009-09-26 21:59:06 UTC
I can confirm, that Dophin is also affected. For a few weeks using Kubuntu Karmic 9.10 (kernel 2.6.31), I must manualy refresh directory views.
Comment 6 Dario Andres 2009-09-26 22:01:09 UTC
Thanks
Comment 7 Mikes 2009-10-06 10:04:51 UTC
i just tested kernel 2.6.31.2 and the problem remains.
Comment 8 Mikes 2009-10-08 15:15:42 UTC
i tested kernel 2.6.31.2 with new kde 4.3.2 and the copy problem is gone. the only remaining issue is when i cut and then paste a file. The desktop remains unrefreshed.
Comment 9 Dario Andres 2009-10-10 17:22:34 UTC
Mh, if it only happens with the Desktop (desktop:/ kioslave may be) it could be a different issue. Can you confirm you are talking about "desktop:/" instead of "~/Desktop" (or does it fails on both?)
Thanks
Comment 10 Hugo Lobo 2009-10-19 16:16:45 UTC
I can confirm this issue on my computer with the same kernel. It happens not only to ~/Desktop or any other folder (inside Dolphin, Krusader, etc..) but on desktop:/ kioslave too.

Procedure:
1- Open desktop:/ on Dolphin
2- Open konsole, "cd" to ~/Desktop and create a file or directory from there.
3- It won't appear on both "Desktop Folder View" and "desktop:/"
Comment 11 Jay LaCroix 2009-10-21 17:21:44 UTC
Same problem here in Arch Linux (64-bit) as well with all updates as of today. This is VERY annoying. There is a few weeks left until 4.3.3, so is there any work around I might try to rectify it now?

Arch Linux report:
http://bugs.archlinux.org/task/16193
Comment 12 Dario Andres 2009-10-25 23:07:17 UTC
*** Bug 211778 has been marked as a duplicate of this bug. ***
Comment 13 Kamil Neczaj 2009-10-30 19:44:18 UTC
I can confirm the problem on arch linux, but it works ok in kubuntu 9.10.
Comment 14 Kamil Neczaj 2009-10-30 22:05:38 UTC
For me the problem occurs on x86 arch linux with kernel 2.6.31 and nvidia drivers, and doesn't occur in kubuntu 9.10 amd64 livedvd also with kernel 2.6.31 and nvidia drivers 185.18.36. Both distros using the same version of kde - 4.3.2. Summing up, we must try to find some patches from kubuntu guys.
Comment 15 Kamil Neczaj 2009-10-30 22:10:24 UTC
I must add that for me downgrading to 2.6.30 and changing driver from nvidia to nv on my arch linux doesn't solve the problem.

One more thing:
The easiest way to reproduce is opening dolphin and then downloading some file with firefox to the folder opened in dolphin. After that the file should immediatly appear in dolphin but it doesn't.
Comment 16 Dario Andres 2009-10-31 19:23:30 UTC
If this is also happening with Kernel < 2.6.31 (comment 15), then the cause is at another place.. I wonder if you have FAM or other related daemon running.
Comment 17 Kamil Neczaj 2009-10-31 21:52:54 UTC
I've tried fam and gamin with kernel 2.6.31 without any effect. I don't remember whether I've tested 2.6.30 with or without fam/gamin. I had downgraded only kernel, udev, hal etc. stayed at same version.
Comment 18 Beat Wolf 2009-11-05 13:00:33 UTC
*** Bug 210952 has been marked as a duplicate of this bug. ***
Comment 19 Beat Wolf 2009-11-05 13:02:56 UTC
*** Bug 212838 has been marked as a duplicate of this bug. ***
Comment 20 Médéric Boquien 2009-11-14 03:48:37 UTC
*** Bug 214073 has been marked as a duplicate of this bug. ***
Comment 21 Jithin Emmanuel 2009-11-14 07:30:51 UTC
*** This bug has been confirmed by popular vote. ***
Comment 22 David García 2009-11-17 12:59:42 UTC
Hi,

This bug has been linked in Launchpad: https://bugs.launchpad.net/ubuntu/+bug/479527
Please have a look there for more details.

As far as I know, it doesn't seem to be kernel related (happens in kernel 2.6.28 and 2.6.31), it's only triggered when you follow a symlink with Dolphin and has been there at least since KDE 4.2.4.
Comment 23 Jay LaCroix 2009-11-17 14:48:27 UTC
I posted this comment in Launchpad and I will post here too to make sure the proper people see it.
-----
I am using Kubuntu 9.10 on my laptop, and Arch on my desktop. Both 64-bit. I am not following any symlinks when this happens, so I am not convinced that it's related to symlinks. I have the problem with both Kubuntu and Arch. In Kubuntu 9.10, it starts off working fine but for some reason, something gets out of sync and the problem starts. With Arch, the problem is always there.

For me, here are a few examples of my reproducing this problem in both Arch and Kubuntu:

Example 1:
1.) Either set the desktop to a Folder View or include a Folder View on the desktop
2.) Open Firefox
3.) Download a file to the desktop
4.) The download completes
5.) The file is not on the desktop
6.) Right click the desktop and click "refresh"
7.) The file you downloaded now appears

Example 2:
1.) Either set the desktop to a Folder View or include a Folder View on the desktop
2.) Find a random file on your desktop
3.) Open Dolphin
4.) Right-click the file chosen on the desktop in step 2, and click "cut"
5.) Go into the Dolphin window and click "Paste"
6.) The file is now moved to the folder you opened in Dolphin
7.) The file is still showing on the desktop
8.) Click on the file on the desktop
9.) It will say the file is not there
10.) Right-click the desktop and click "refresh"
11.) The file disappears

Example 3:
1.) Open KTorrent
2.) Open Firefox
3.) Find a torrent to download in Firefox
4.) Send the Firefox open dialogue for the torrent to KTorrent
5.) Choose the desktop as the place KTorrent will save the downloaded torrent to
6.) KTorrent starts downloading the torrent
7.) No files have appeared on the desktop
8.) Right-click the desktop and click "refresh"
9.) The files that KTorrent is downloading now appears on the desktop

I deal with this problem in both Kubuntu 9.10 AND Arch with KDE 4.3.3, even after reinstalling my OS completely and/or deleting my .kde and .kde4 folders. This is extremely annoying. I have to babysit my desktop and constantly refresh it manually to see any changes. If I never refresh it, I will never see any changes.
Comment 24 Sebastien ROHAUT 2009-11-17 15:30:01 UTC
My two cents. The problem doesn't occurs only in desktop but everywhere. I found it using ksnapshot / KDE file chooser.

As it occurs with 2.6.31 and its new fsnotify/inotify implementation, the problem should come from the kernel or from an old and/orr incorrect use of notify system calls in kdelibs.

Three bugs reports are in Novell Bugzilla for openSUSE 11.2 :

Two for KDE file refresh
https://bugzilla.novell.com/show_bug.cgi?id=547903
https://bugzilla.novell.com/show_bug.cgi?id=544153

One for a true problem with 2.6.31 fsnotify :
https://bugzilla.novell.com/show_bug.cgi?id=546491

I don't think it's a coincidence.

kdelibs4 has been updated sunday in openSUSE KDE 4.3.x repositories :

* dim. nov. 15 2009 dmueller@suse.de
- refresn patches

It's seems it works now. I don't know what are these patches.

From  Novell bugzilla #546491, Andreas Gruenbacher:
I have pushed two fixes into the 11.2 git tree; they should show up in the KOTD
in a while. Could you please see if they make a change? Changelog entry:

    - patches.fixes/dnotify-ignore-FS_EVENT_ON_CHILD.diff: dnotify:
      ignore FS_EVENT_ON_CHILD.
    - patches.fixes/inotify-fix-coalesce-duplicate-events-into-a-single.diff:
      inotify: fix coalesce duplicate events into a single event in
      special case.

So there was a problem. I didn't test the kernel fix. i will do tomorrow. But I can confirm current 2.6.31 has a problem with inotify, because my own code doesn't work.
Comment 25 David Faure 2009-11-17 21:35:49 UTC
(In reply to comment #22)
> As far as I know, it doesn't seem to be kernel related (happens in kernel
> 2.6.28 and 2.6.31), it's only triggered when you follow a symlink with Dolphin
> and has been there at least since KDE 4.2.4.

You're confusing bugs. You're talking about bug 213799, which I fixed for 4.4.0.
However there -is- a kernel-related bug, I'm sure of that, since kdelibs/kio/tests/kdirlistertest worked fine before I upgraded from 2.6.28 to 2.6.31 (kubuntu karmic). Comment #24 has more details, I see - very cool.
Comment 26 David García 2009-11-18 08:36:22 UTC
Agreed. Thanks for fixing that bug.
Just one question: do you know whether the fix for bug 213799 is being backported to a 4.3.X release?

(In reply to comment #25)
> (In reply to comment #22)
> > As far as I know, it doesn't seem to be kernel related (happens in kernel
> > 2.6.28 and 2.6.31), it's only triggered when you follow a symlink with Dolphin
> > and has been there at least since KDE 4.2.4.
> 
> You're confusing bugs. You're talking about bug 213799, which I fixed for
> 4.4.0.
> However there -is- a kernel-related bug, I'm sure of that, since
> kdelibs/kio/tests/kdirlistertest worked fine before I upgraded from 2.6.28 to
> 2.6.31 (kubuntu karmic). Comment #24 has more details, I see - very cool.
Comment 27 David Faure 2009-11-18 12:28:57 UTC
I didn't backport the fix for bug 213799 because it's a bit too intrusive (-> risk of regressions)
Comment 28 Tom Arnold 2009-11-22 15:31:17 UTC
I would like to risk the regressions .. this is totally annoying. Makes KDE pretty much worthless for file management.
Comment 29 Jay LaCroix 2009-11-22 18:17:56 UTC
I agree. Waiting to fix this would be silly because KDE is essentially broken right now. This worked fine in the initial release of KDE 4.3.0 so it should work fine now. This bug has basically ruined my KDE 4.3.x experience. This is not just a small flaw, this is HUGE.
Comment 30 Martin Samek 2009-11-23 09:55:11 UTC
Hey, please backport the fix. Now it's total pain to use KDE.
Comment 31 Dario Andres 2009-11-27 22:21:44 UTC
The fix which is not going to be backported is the one for bug 213799 (only related to *symlinks*....). The inotify issue was not even fixed on KDE SC 4.4 yet... If you want that issue(symlinks) to be backported then your comments should be in *that bug report* and not into this... otherwise this was a misunderstood... Regards
Comment 32 Dario Andres 2009-11-30 18:50:45 UTC
*** Bug 216838 has been marked as a duplicate of this bug. ***
Comment 33 Dario Andres 2009-12-07 01:20:30 UTC
*** Bug 216641 has been marked as a duplicate of this bug. ***
Comment 34 Dario Andres 2009-12-10 22:01:03 UTC
*** Bug 218180 has been marked as a duplicate of this bug. ***
Comment 35 David Faure 2009-12-22 22:54:07 UTC
SVN commit 1065275 by dfaure:

Investigating the inotify problems on kernels >= 2.6.31: the problem only happens
when removing+re-adding a directory in kdirwatch (which happens all the time in kdirlister).
CCBUG: 207361


 M  +50 -24    kdirwatch_unittest.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=1065275
Comment 36 David Faure 2009-12-24 18:30:12 UTC
SVN commit 1065898 by dfaure:

The inotify implementation in 2.6.31/32 changed: it's possible after rm+add to get the same value of "wd" again.
Which makes this processing of IN_IGNORED problematic, since it makes us forget about the 'new' watch.
With many thanks to Eric Paris (who works on inotify in the kernel) for his hints about this.

About the fix: it seems safe to me to just ignore IN_IGNORED ;-), but I might adjust
this if Eric Paris advises otherwise next week.

Fixed for: 4.4 rc1
BUG: 207361


 M  +2 -1      kdirwatch.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=1065898
Comment 37 Pino Toscano 2010-01-03 19:05:17 UTC
*** Bug 206890 has been marked as a duplicate of this bug. ***
Comment 38 Sebastien ROHAUT 2010-01-16 21:26:00 UTC
Are you sure it's fixed ? I installed KDE 4.3.90 (RC1 from openSUSE 11.2 repositories) and the problem remains in file chooser. I must use F5 to refresh.
Comment 39 Sebastien ROHAUT 2010-01-16 21:32:27 UTC
Oh I'm stupid. I had this problem in pseudo-kde4 file chooser from openoffice.org opensuse package. In fact it works everywhere else.
Comment 40 David Faure 2010-01-21 17:17:01 UTC
SVN commit 1078133 by dfaure:

Backport r1065898: Fix inotify regression with linux kernel 2.6.31/32.
Fixed for: 4.3.5
CCBUG: 207361


 M  +2 -1      kdirwatch.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=1078133
Comment 41 Frank Reininghaus 2010-02-02 12:15:26 UTC
*** Bug 210284 has been marked as a duplicate of this bug. ***