Version: 1.5.2 (using KDE 3.1.2) Installed from: compiled sources Compiler: gcc version 2.95.4 20020320 [FreeBSD] OS: FreeBSD (i386) release 4.8-STABLE Yes, I realize this is a duplicate of several other bug reports. Each and every one claims file system responsibility. I do not believe this is a sufficient level of reasoning as this apparently has occurred to a number of different kinds of files systems, to include my UFS with SoftUpdates enabled. KMail is simply too important an application to be this unstable. I'm assuming the problem lies with KMail holding open the config file rather than closing it after it has been read or written to. What should happen is that each time kmailrc is accessed the file is closed out. There simply can't be that big a performance hit by doing this, and it goes a lot further to insuring that the file system actually has a chance to recover a glitch. If that isn't possible, KMail should at least maintain a copy of it's configuration for comparison purposes. Several word processing programs do this in order to recover from crashes, and I would argue that none of them are as critical to end users as KMail. A third approach would seperate out different aspects of KMail to seperate files. There are areas, such as account information, that need only be dealt with at startup, where other portions need constant access. Why does it all need to be placed within a single file? If KMail forgets which directory it was last looking at that's one thing. When it loses over 50 filter rules and a half dozen account settings plus every single look and feel option, that is an entirely different beast.
kmail is _not_ keeping the file open. It's saving to a copy and them moving the result in an atomic operation. If something in that goes wrong, it's _definitly_ the fault of your file system. The older bug reports were most likely more valid as KDE2 used to write to the destination config file. Splitting the config into several files is still TODO though - and I was under the impression it's a duplicate, but can't find another bug.
Here's some of what I was referring to... http://bugs.kde.org/show_bug.cgi?id=32896 http://bugs.kde.org/show_bug.cgi?id=41112 http://bugs.kde.org/show_bug.cgi?id=44064 http://bugs.kde.org/show_bug.cgi?id=44381 Especially interesting... http://bugs.kde.org/show_bug.cgi?id=56874 These are mostly post 3.x reports. All KMail, all complete configuration loss due to an unexpected shutdown. Nobody seems to have actually addressed the problem other than to divert the blame to the OS or file system. This leaves two basic possibilities. Either Suse, Redhat, and FreeBSD are all totally screwed up, or there is something intensely wrong with what KMail is doing with it's configuration file. To date, the filesystems blamed for KMail's data loss are ReiserFS, XFS, and now UFS with SoftUpdates. Are all of these filesystems really that unstable? I should also note here that I had several other applications running at the time of my crash (battery went dead on me). The only application that did not recover fully was KMail. I strongly disagree with making this bug a wishlist item. This app sufferred data loss under a common mishap and has done so in the past. I would hope that it gets treated like the data loss problem that it is.
Subject: Re: Split kmailrc (was: All settings lost on power loss) On Monday 26 May 2003 10:16, metrol@metrol.net wrote: > I strongly disagree with making this bug a wishlist item.
FWIW, a _power cut_ cannot count as a common mishap (and a laptop dying with a drained battery is nothing but a power outage). A power outage is one of the worst and most unpredictable things that can happen to an application, an OS, a filesystem and a harddisk. If power cuts happen so often that you mistake them for a common mishap, you should investigate how to shut down clean before the battery dies - or at the very least turn off write caching for your ide harddisks to assist your filesystem and harddrive.
Subject: Re: Split kmailrc (was: All settings lost on power loss) As Stephan pointed out KDE tries to make sure that no data is lost by first writing to a new file and then replacing the old file with a new file. Splitting kmailrc will not fix this problem. I see no reason why KMail's configuration should be stored in several files. This will surely not minimize the risk of losing the configuration in case of a severe system crash. In fact it will increase the risk because each file could be damaged during the crash. BTW, the only guaranteed way to prevent data loss is regular backups.
One problem in my eyes is, that the file is currently used both as a cache (for the folders) and a config file. Splitting the config file would make it for a cleaner data separation and at least lower the probability of a lost config: For one is the file pretty big (mine for example has 5883 lines and is 119kB big) and takes longer to be saved back to disk, so you have a longer timespan where your file is in danger. Several files also have the advantage that only the settings which have changed need to be written back to disk -- each file could be damaged only if you changed a setting involving every file. And it always gives me headaches if I look at that file manually and have to scroll 4000 lines before I finally find that option I've been looking for ;-) Separate files additionally had the advantage of a little possible increased security: If a file containing passwords may not be world/group readable (like it's already the case in many apps), it's less likely that somebody gets his hands on my passwords on a multi-user machine. (Right, there's no reason to have the kmailrc group-readable at all, data separation makes it easier to enforce this.) I'd suggest splitting that file into four separate ones: kmailrc -- general settings kmail_folderrc -- [Folder] related settings and "cache" entries kmail_filterrc -- the [Filter]s kmail_transportrc -- [Account]s, [Transport]s and maybe [Identity]s (this file may not be group/world-readable) But that's all just me three cents :)
I agree with Malte here. Currently the config file is invalidated on every new message to a folder. Config data should only be invalidated by change to setting and the folderrc can be restored very easily (by just removing it and starting kmail).
Subject: Re: Split kmailrc (was: All settings lost on power loss) Sorry, but only very few items in the [Folder xxx] sections are non-config items, namely Current Top TotalMsgs UnreadMsgs isOpen Compactable All others are folder settings: DateWidth=170 ExpireMessages=false Identity=0 MailingListAdminAddress= MailingListEnabled=false MailingListPostingAddress= NormalIconPath= OrderOfArrival=false ReadExpireAge=7 ReadExpireUnits=0 SenderWidth=170 SizeWidth=80 SortColumn=3 Status=false SubjectWidth=310 UnreadExpireAge=28 UnreadExpireUnits=0 UnreadIconPath= UseCustomIcons=false WhoField= htmlMailOverride=false threadMessagesOverride=false So only very few items should be moved to a separate file. BTW, splitting kmailrc could make updating it between releases difficult or impossible. For example during the identity update the occurrences of identities in the whole file had to be changed to unique ids. If the folder configuration (which also contains an identity) would have been stored in another file this wouldn't have been possible (AFAICS) without changes in kconf_update to allow passing more than one file to a perl script.
At least have the program keep a backup copy of the config file(s), possibly updated once when opening Kmail. At least this would provide a chance to get most if the setting back after loss of the master. Believe me, this is a _very_ disappointing issue to deal with, and the size of the issue definitely appears related to how the kmail handles it files - IOW, it looks like the situation can very easily be improved, if not solved. Please don't mention "regular backups". As an installer I need to make a setup in a limited time, usually with only the distro disks and all default settings. I don't get to tell the cuatomer to make his bed, clean up his room, and back up his files - usually I'm just happy to get the business. Stuff like this makes it harder to do that next time. This problem has already been solved by most Windoze applications; it should at least be possible to manually copy an old version of the settings. If this kind of problem can be mostly eliminated by a simple copy on startup by the application, it will give everybody in the Linux camp a better chance to survive and go far. Rufus
@Rufus (and everyone else requesting keeping a backup of the config file): This has been done on the level of KConfig (which is part of kdelibs). So if you want this then file a feature request against kdelibs.
Stephan Kulow 2003-05-26 09:35: > kmail is _not_ keeping the file open. It's saving to a copy and them moving > the result in an atomic operation. If something in that goes wrong, it's > _definitly_ the fault of your file system. The older bug reports were most > likely more valid as KDE2 used to > write to the destination config file. Today, it happened to me as well. I kept searching and searching for that "backup file" but as I don't know its name, I didn't succeed yet. It will take 2-3 hours of concentrated work to manually resore (=reenter) identites, POP/SMTP/IMAP accounts, filters, addresses, distribution lists (everything is gone!). This happened with 1.6.2, the most recent version and XFS file system. As most settings don't change too often, I suggest having one automatically generated kmailrc.gz that is only updated whenever kmailrc is changed. I know of applications that offer automatted backup cycles of up to 9 steps. Please, don't disregard this thread, it is really crucial.
Another reason to split kmailrc is this IMHO: I am running KDE 3.5, but installed KDE4 for another user. In my KDE3 session I started kopete via F2. Seems Opensuse has a flaw, because Kopete of KDE4 was started, which triggered possibly Akonadi and whatnot, at least the KDE4 update scripts for kmailrc was run in the back (and Kopete never started, hu?). On the next start of KMail my accounts were gone (in the view), (Pop/Imap instead of pop/imap as values for type key in the config), on exit even removed from kmailrc (due to accounts=0). Old version of kmailrc to the rescue - well, including a lot of work, because of all the filters and folder settings which had changed as well, so I had a hard time to tell what the changes were (and I was to lazy to search for the update scripts and find out what was supposed to be changed, now that I think of) So please consider storing the settings of the accounts/transports, those of the templates, those of the filters and those of the folders in different files, as they can get very large. Otherwise there is not much difference between to the Win registry ;)
Disappointing to see such a real world PITA that's so clearly unnecessary (based on the performance of other major applications) continue on like this. You say there's a backup? Not if it's not in a logical place with a logical name with a (simple) logical usage. Nobody but KDE developers knows where the bits and pieces are hidden and where to patch them in and (gawd forbid) edit them. If something is not readily available and useable (menu item, anybody?)... It doesn't exist. This hit me three times, the last in 2006. I'd been hoping it was fixed but silly me. It looks like KDE is clueless. If anybody cares, it's the filters that really kill you. They can change and grow hugely after the initial installation and they are quite important to many email users.
Currently running KDE 4.2 because it was announced as stable for daily usage. But I lost my KMail settings the third time now (in 3 days!). First startup it didn't import my KMail settings from KDE 3.5.9 correctly. All settings (filters etc) knew about the folders, but the actual mail accounts having been there. Accidently in trying to recover from that I moved my old kmailrc to the new place so that's gone now. I restarted alomst from scratch using a very old backup I had. Accounts were there, but I needed to readd all filters (hell of a work). Then, kmail crashed. On start all my folders, settings, filters etc were gone. Yeah, very funny! I decided to start from scratch. I set up my accounts, filters, folders etc, it was a lot of work. Now after several shutdown, startups, reboots it kept config. Until shortly kmail came up with just an empty configuration - everything gone again. This is in no sense stable! I'm making heavy use of dimap - if that helps to find out what happened. Is kmail keeping any backups of this single(!) important(!) file around? I don't think so although it should. This needs to be fixed asap - at least a quickfix should keep some backups around so I can recover easily.
Guess Ext4 is another good reason to split config from caching: https://bugs.edge.launchpad.net/ubuntu/+source/linux/+bug/317781
So reading the URL from comment #15 I have another suspect: kmailrc handling is completely okay, but the filesystem may destroy it. I'm running on XFS, not Ext4 - but it probably has the same behaviour (consistence over preservation), as I read XFS zeroes out files with unwritten data blocks during log replay. So a good idea would be to keep a backup of the original file which kmail should use if the current file looks unusable. I assume kmail already tries an atomic update by creating a tmp file and then renaming to the original file name. Should be easy to rename the original file to the backup first. This is not strictly atomic now but with my suggestion kmail can easily recover from the missing-rc-file-case by using the backup if no standard rc file can be found.
You're right, KConfig should generally handle long commit times better and use temporary files and fsync. But I still think that mixing config data and cache data as it is done in the kmailrc is wrong and the latter should go to a dedicated cache file.
I have had this several times as well now, using KDE 4.2.3 (but starting with earlier versions) on OpenSuse 11.1. My filesystem is XFS on linux. Problem happens when the system crashes during bootup (while restoring the previous session). I have the same problem with plasma-desktop.
Maybe it would be generally good practice (if not best practice) to split all config files in that way, that writing pure configuration (which only occurs when I reconfigure something) to another config file which is thus pretty stable and not subject to change often, and writing session specific configuration (like ids, position data, column ordering etc) to its own config file. The idea behind it is: It wouldn't be that hard to loose position data of plasmoids (I can simply move them where they've been) or mail listing settings (just a few clicks to reconfigure that), but it is very very annoying to find a completely zero state default plasma desktop, or find all your filter and account settings in kmail lost. :-( Of course this solution only alleviates the symptoms, the long term target should be to cure the root cause (wrong interaction with the filesystem, bad filesystem behaviour). One cannot guarantee that desktop systems are totally stable - there are many causes for power outage or hard locks: Instable graphics drivers (this doesn't only relate to closed-source drivers!), lack of ups'es (shutdown due to vis major), running out of battery (laptop). It's pretty normal that desktop systems run in hostile environments so the desktop environment should be implemented in a way to support this. Strange enough is that server software components are designed to support hostile environments better although these are usually run in friendly environments (good cooling, ups, no console with a human "attached").
Thank you for your feature request. Kmail1 is currently unmaintained so we are closing all wishes. Please feel free to reopen a feature request for Kmail2 if it has not already been implemented. Thank you for your understanding.
Instead of creating a new feature request, please confirm here if the wishlist is still valid for kmail2.