Bug 254126 - PLEASE implement disk I/O quota for virtuoso-t
Summary: PLEASE implement disk I/O quota for virtuoso-t
Status: RESOLVED UNMAINTAINED
Alias: None
Product: nepomuk
Classification: Miscellaneous
Component: general (show other bugs)
Version: unspecified
Platform: Debian testing Linux
: NOR wishlist
Target Milestone: ---
Assignee: Sebastian Trueg
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-10-14 10:44 UTC by Boris
Modified: 2015-01-23 16:25 UTC (History)
10 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Boris 2010-10-14 10:44:29 UTC
Version:           unspecified (using KDE 4.4.5) 
OS:                Linux

virutoso-t can often completely saturate disk I/O (and I have a fast disk). The most recent case was when reorganising large mail folders. It was impossible to do any work for about 15 minutes after the move because of frantic indexing activity, with saturation of disk I/O (and to a lesser extent CPU) being the main problems.

Indexing should be a BACKGROUND activity. To ENSURE civic behaviour, it would be really good to implement quotas for disk I/O (and probably CPU usage etc) by default.

See also #228081, #227985, #253103, #241443 re: excessive resource usage.

Best regards,

Reproducible: Sometimes

Steps to Reproduce:
Move a large directory with many files.

Actual Results:  
Machine slows to a crawl with atop showing virtuoso-t reading and writing to disk saturating and also significant CPU usage.

Expected Results:  
A limit to 5% of resources (I/O, CPU, memory) would be nice.
Comment 1 Boris 2011-06-18 16:51:04 UTC
This still causes big problems at start up. System is unresponsive for minutes after logging in, recalling the worst days of windows (pretending to be ready but not really). For a while I had worked around the problem by disabling the indexing, but now if I want to use kontact it needs akonadi which needs indexing, it seems.

renice reports a -19 priority already (might be a bug according the man). In any case, the main problem currently seems to be saturation of disk I/O rather than CPU usage.

(debian testing kde-plasma-desktop Version: 5:68)
Comment 2 Alejandro Nova 2011-07-27 02:36:45 UTC
Akonadi doesn't need indexing. It needs Nepomuk, though, so you can enable Nepomuk and disable indexing for the time being.
Comment 3 Boris 2011-11-30 08:28:51 UTC
This is still a major problem in kde 4.6.5 (I track Debian testing). Even with strigi file indexer DISabled in the system settings (as per comment above), upon each restart I am faced with a horribly unresponsive system for many minutes. This is because disk access is saturated (atop)

DSK |         sdb | busy    100% | read       6 | write    600 | avio    1 ms 

I know this is free software, but maybe you could give a thought to users who wish, you know, to use their computer. I'm not saying that "semantic" will never be useful, but preventing email/browsing/any other work is really crazy. The indexing has to be more discreet (and discreet by default).

Is it so difficult to sleep between disk accesses? I really couldn't care if the index is incomplete for an hour. And doesn't it persist anyway? Elsewhere it is mentioned that the activity is to place monitors on each file. If this is true, I can see why you try to do so quickly, but you still have a huge race condition, so a different way of managing it seems necessary anyway.

Regarding the theoretical benefits of the semantic desktop, have you considered that the computer-illiterate users who are more likely to benefit from the semantic stuff (i.e. they don't understand file systems and don't have a defined workflow) are also those who will be unable to understand why KDE doesn't respond and will quickly change to some over desktop environment that does.

I've been a faithful KDE user for ~10 years, so thank you for those, but the current situation is really making me question the future.
Comment 4 Sebastian Trueg 2011-11-30 09:56:13 UTC
4.6.5 is rather old. 4.7.3 should provide a much better experience.
Comment 5 Franz Trischberger 2012-02-05 11:26:34 UTC
Running kde-4.8.0. Login takes over a minute, due to virtuoso-t starting up. This is on a recent WD caviar black, so a quite fast HDD. When I disable nepomuk, login goes quite fast (passwd -> <10sec -> desktop). But I was watching this behaviour over the last releases, feeling says it even got worse.
Comment 6 Dennis Schridde 2012-08-03 00:20:43 UTC
I experience the same issue. Needless to say that on a 5.4k rpm drive like mine the issue is even worse. It would be really nice if I could start my computer and do something with it, not having to wait for the metadata system to become available. Now that even plasma-desktop seems to use Nepomuk, the issue indeed got worse, as even after login Plasma tends to freeze for many seconds (30-60 — just a guess).
Comment 7 Boris 2012-09-22 15:42:45 UTC
"4.7.3 should provide a much better experience"

Well, a slightly less awful start up experience, maybe (I'm now on 4.7.4). Disk i/o is still saturated for minutes after startup. This is free software and you can do what you like or you can. But if you were to ask me "is it acceptable to saturate the user's disk for minutes when (s)he has switched on the computer to do some work?", my answer would be "no". Other family members use this computer, and I have witnessed first hand many times "Why doesn't it work? Your linux is crap!". If the aim is to attract new users, you simply _have_ to get them to the useful work stage before they will even begin to notice the wonderful semantic desktop... Otherwise all will have been for nought.

Typical atop output. Needless to say that killing virtuoso-t (properly) solves the problem.

DSK |         sdb | busy     98% | read    1155 | write   2701 | avio    2 ms |

  PID  SYSCPU  USRCPU  VGROW  RGROW USERNAME THR  ST EXC  S  CPU CMD     1/1   
 4525   0.17s   1.70s     0K    24K boris      7  --   -  S  19% virtuoso-t
Comment 8 Alejandro Nova 2012-09-22 18:10:06 UTC
We are at 4.9.1 by now, and according to Vishesh Handa, there are massive improvements there, specially when you mix KDE 4.9.1 with Soprano 2.8.1.

Please, update and retest. Your mileage WILL vary ;).
Comment 9 Dennis Schridde 2012-09-22 19:55:44 UTC
I have only Soprano 2.8.0 (2.8.1 is not in Gentoo/KDE yet), and KDE startup still takes literally minutes, before I get a usable desktop.
Comment 10 Craig Magina 2013-05-08 17:06:21 UTC
I am running KDE 4.10.2 on Kubuntu 13.04 amd64 with soprano 2.9.0 and experience minutes of unresponsiveness on system startup while virtuoso-t and nepomuk are the top cpu consumers. I have a brand new 2TB WD data disk for my home directory and a SSD for my root disk. I use kontact with a large downloaded e-mail account, google services, ldap and owncloud all configured as akonadi resources.
Comment 11 Alejandro Nova 2013-05-08 19:05:04 UTC
We should focus on LDAP and OwnCloud, since everything else is behaving here. Can you install KDE 4.10.3 and be sure that you have Akonadi 1.9.2 before we continue? KDE 4.10.3 has a rewritten Nepomuk Feeder, and that could be very useful here.
Comment 12 Vishesh Handa 2015-01-23 16:25:09 UTC
Thank you for taking the time to file a bug report.

The Nepomuk project is no longer included in the KDE Software Compilation. With Plasma 5, we have replaced most of the underlying technology with Baloo and other components. Hopefully this will have addressed your concern.

We encourage you to try to Plasma 5 (+Baloo) and let us know if your problem persists.