Bug 72441 - Hide Mail and dead.letter
Summary: Hide Mail and dead.letter
Status: RESOLVED FIXED
Alias: None
Product: kmail
Classification: Applications
Component: general (show other bugs)
Version: unspecified
Platform: Compiled Sources Linux
: NOR wishlist
Target Milestone: ---
Assignee: kdepim bugs
URL:
Keywords:
: 74300 (view as bug list)
Depends on:
Blocks:
 
Reported: 2004-01-12 04:23 UTC by Luke Sandell
Modified: 2007-09-14 12:17 UTC (History)
4 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments
moves kmail storage to appdata (3.37 KB, patch)
2004-09-29 23:11 UTC, Aaron J. Seigo
Details
moves kmail storage to locateLocal("data", "kmail/mail") (4.23 KB, patch)
2004-10-06 21:41 UTC, Aaron J. Seigo
Details
updated patch addressing ingo's comments (4.29 KB, patch)
2004-10-14 00:43 UTC, Aaron J. Seigo
Details
ingo suggests, i code (5.00 KB, patch)
2004-10-14 21:36 UTC, Aaron J. Seigo
Details
Aaron-Seigo-2004-10-14-use-hidden-Mail-folder-and-move-old-mail-ik-utf8.diff (5.96 KB, text/x-diff)
2004-10-18 21:48 UTC, Ingo Klöcker
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Luke Sandell 2004-01-12 04:23:49 UTC
Version:            (using KDE KDE 3.1.94)
Installed from:    Compiled From Sources

Recent discussion on kde-usability seems to conclude that placing the "dead.letter" file directly in the home directory is intrusive to the user's personal files. It would be much better to make it ".dead.letter" so that it is hidden, or move it into a kmail-specific directory. 

Additionally, users generally have no reason to directly access the Mail folder, and therefore it should also be hidden away under the name ".mail" or in ".kde" by default.
Comment 1 Don Sanders 2004-01-12 05:18:59 UTC
Subject: Re:  New: Hide Mail and dead.letter

In this discussion was the issue of dead.letter being a de facto 
standard that pine, sendmail etc follow covered?

Comment 2 Ingo Klöcker 2004-01-12 21:42:13 UTC
Sometimes it makes sense to follow a defacto standard. But sometimes it doesn't make sense. Neither pine nor sendmail is used by the majority of users that we write KMail for. Pine users are usually experienced with Unix and are not confused by a file in their home directory. But GUI users could be confused. And to be honest, even though I'm an experienced user, applications that clutter my home directory with not hidden files annoy me.

About ~/Mail:
a) this is a duplicate
b) since there are other mail clients that use Mail as well and since KMail doesn't cope very well with others messing around with its files it would make sense to use another directory, e.g. ~/.kmail by default.

Anyway, I don't have a preference for or against using another Mail directory by default.
Comment 3 Paul Pogonyshev 2004-02-06 00:01:14 UTC
Agreed with the inital post absolutely.  There is absolutely no reason I need to see both `dead.letter' and `Mail' in my home directory.  They are often annoying.  I'd really like them moved away or hidden.
Comment 4 Don Sanders 2004-02-06 05:19:38 UTC
*** Bug 74300 has been marked as a duplicate of this bug. ***
Comment 5 Thomas Zander 2004-09-29 22:16:27 UTC
Another "Please fix this" from a usability person... :)
Comment 6 Aaron J. Seigo 2004-09-29 23:11:06 UTC
Created attachment 7732 [details]
moves kmail storage to appdata

make that two 'ayes' from the usability peanut gallery ;) 

in fact, how about the attached patch? it puts it into locateLocal("appdata",
"mail/") which is a bit more radical. there's some code dupe in the patch and
its 100% untested (just compiled to check syntax). if this is palatable to the
kmail hackers, i'll test, clean it up a bit more and then commit once i've
verified that it works properly .... but first... what do you think of the
concept?
Comment 7 Don Sanders 2004-09-30 11:22:22 UTC
I'm okay with renaming .deadletter. Renaming ~/Mail to ~/KMail is just 
too bold for me. I wonder what will happen if a mutt/pine/elm user 
decides to try out KMail ... they won't be able to see their old mail 
anymore in mutt/pine/elm.

Don.

Comment 8 Thomas Zander 2004-09-30 11:34:51 UTC
On Thu, Sep 30, 2004 at 09:22:23AM -0000, Don Sanders wrote:
>Renaming ~/Mail to ~/KMail is just 
> too bold for me. I wonder what will happen if a mutt/pine/elm user 
> decides to try out KMail ... they won't be able to see their old mail 
> anymore in mutt/pine/elm.

That argument does not hold since the folder structure of KMail is already
very different from what mutt/pine expect.  In its current version sharing
is not practical anyway.

The incompatability I am referring to is concerning the subdir structure
KMail enforces.  Creating ".foo.directory" directories.
Furthermore, altering mailbox 'foobar' with mutt and leaving the
foobar.ids and foobar.sorted files untouched can lead to loss of email.

Now you point it out, these basic differences makes renaming it (out of
$HOME, as Aaron proposed) a very good idea to avoid trouble.

Comment 9 Don Sanders 2004-10-01 07:12:19 UTC
On Thursday 30 September 2004 19:34, Thomas Zander wrote:
> ------- You are receiving this mail because: -------
> You are the assignee for the bug, or are watching the assignee.
>
> http://bugs.kde.org/show_bug.cgi?id=72441
>
>
>
>
> ------- Additional Comments From zander kde org  2004-09-30 11:34
> -------
>
> On Thu, Sep 30, 2004 at 09:22:23AM -0000, Don Sanders wrote:
> >Renaming ~/Mail to ~/KMail is just
> > too bold for me. I wonder what will happen if a mutt/pine/elm
> > user decides to try out KMail ... they won't be able to see their
> > old mail anymore in mutt/pine/elm.
>
> That argument does not hold since the folder structure of KMail is
> already very different from what mutt/pine expect. 

I don't see how that makes a difference. If the ~/Mail directory is 
renamed then mutt/pine won't be able to find old mail. That's true no 
matter what the folder structure of KMail is.

> In its current 
> version sharing is not practical anyway.

That's correct.

> The incompatability I am referring to is concerning the subdir
> structure KMail enforces.  Creating ".foo.directory" directories.

Yes there's no standard for subdirectories, that's a general problem 
for mail clients that use ~/Mail.

> Furthermore, altering mailbox 'foobar' with mutt and leaving the
> foobar.ids and foobar.sorted files untouched can lead to loss of
> email.

I think you mean foobar.index, but I understand what you mean.

> Now you point it out, these basic differences makes renaming it
> (out of $HOME, as Aaron proposed) a very good idea to avoid
> trouble.

It's a bad idea to rename ~/Mail for existing users for the reason I 
gave above.

Don.

Comment 10 Luciano Montanaro 2004-10-01 16:07:08 UTC
I'd rather have pine/mutt compatibility improved instead.
Comment 11 Aaron J. Seigo 2004-10-01 18:22:47 UTC
> I wonder what will happen if a mutt/pine/elm user 
> decides to try out KMail

i thought that was what IMAP was for? ;-)

i'd also suggest that this is such a rare case with fairly easy ways around the larger problem (e.g. back up your email, use IMAP, whatever) that it really doesn't outweigh the issue of having a user-visible folder in ~/. not only does this degrade the utility of that folder for your average desktop user as it becomes a place where "magic" folders live that you shouldn't touch, but it also does result in real, non-theoretical data loss as i've heard of a few cases where people have simply deleted ~/Mail not know what it was since they couldn't remember putting it there. yes, this silly behaviour, but we can help avoid it by putting the mail somewhere safe.

btw, i've reworked the patch so it's actually half decently non-ugly ;-)
Comment 12 Aaron J. Seigo 2004-10-06 21:41:31 UTC
Created attachment 7802 [details]
moves kmail storage to locateLocal("data", "kmail/mail")
Comment 13 Aaron J. Seigo 2004-10-06 22:13:18 UTC
p.s. i wonder if the move should be done more or less silently, preventing the majority of users from having to interpret the message in that dialog which is complete but complex =)
Comment 14 Aaron J. Seigo 2004-10-06 22:37:34 UTC
just to spam this report with more detail: the other half of this solution (as agungl pointed out nicely on IRC) is a way to easily locate and backup the mail data. something like kdenonbeta/KeepIt is that other half ;-)

so the "users won't find it" isn't an issue.
Comment 15 Ingo Klöcker 2004-10-07 23:15:34 UTC
I'm okay with changing the default location for the mail storage to locateLocal("data", "kmail/mail"). But this mail storage moving code makes my belly ache. Is this really necessary? Wouldn't it be sufficient to just change the default, but leave the old mail storage where it is?

I have several Gigs of mail on an NFS mounted /home. I don't even want to know how long KMail would block while hundred thousands files are moved. And what happens if something goes wrong during the move? IMO this moving is just to risky. Mandrake made their KMail use ~/.Mail (or whatever) and we received several bug reports because obviously it didn't work in all cases.
Comment 16 Paul Pogonyshev 2004-10-08 00:07:22 UTC
> I have several Gigs of mail on an NFS mounted /home. I don't even want to
> know how long KMail would block while hundred thousands files are moved.

Well, moving a directory is instant to the best of my knowledge.

> And what happens if something goes wrong during the move? IMO this moving
> is just to risky. Mandrake made their KMail use ~/.Mail (or whatever) and
> we received several bug reports because obviously it didn't work in all
> cases.

But here I agree.  KMail shouldn't play with this.  It is too risky
for this little gain.  Let me move my mail by hands.

Comment 17 David Faure 2004-10-08 00:18:12 UTC
> Well, moving a directory is instant to the best of my knowledge.

Only when the source and destination are on the same partition.
Granted, this should be the case for most people, when moving from
$HOME/Mail to $HOME/.kde/share/apps/kmail.

Comment 18 Aaron J. Seigo 2004-10-08 00:41:03 UTC
the patch as it stands lets the user choose with a Yes or No. i'm cool with leaving it like that, as that should be a satisfactory compromise. i'll probably clean up the language a bit to make it more easily decipherable as to what is about to happen for the average user.

as for "something going wrong" during the move, if the move crosses partitions, if the move fails the mail will stay where it is. if its on the same partition, the only way it can fail is if there are permissions problems which are not only unlikely due to it being KDEHOME but will also prevent the mail from being moved from the original location.
Comment 19 Ingo Klöcker 2004-10-13 23:00:42 UTC
Problems I see with the patch:
a1) If the move crossing partitions fails then the user's mail storage will be fscked up because the following code will pick up the partially moved mail. This can be prevented by storing the valid location of the mail storage in the config file. Additionally, the partially moved mail should be deleted (from the new location).

+    foldersPath = locateLocal("data", "kmail/mail/");
+    if (!QFile::exists(foldersPath))
+    {
+      // only do the transfer mail trickery if we haven't already
+      // set up the dir.
+      transferMail(foldersPath);
+    }


a2) If the user answers No to the move then he will be asked again on the next start. You have to store the old folder in the "folders" config entry, so that the following if statement is skipped:

+  QString foldersPath = cfg->readEntry("folders");
 
   if (foldersPath.isEmpty())
   {


a3) KMail should store the folder location in any case in the config file. Otherwise KMail will ask whether $HOME/Mail should be moved to locateLocal("data", "kmail/mail/") even if locateLocal("data", "kmail/mail/") does already exist and $HOME/Mail was created by another MUA (KMail is not the only MUA using $HOME/Mail).


b) The following is dangerous because destinationDir might contain "%1" (or "%2" or "%3"). Too bad there's no QString::arg() with 5 parameters.

+  QString msg = i18n("The %4 folder exists. KMail now uses the %1 folder for its messages.\n"
+                     "KMail can move the contents of %5 into "
+                     "%2, though this may replace any existing files with the same "
+                     "name in %3.\n"
+                     "Should KMail move the mail files now?")
+                     .arg(destinationDir, destinationDir, destinationDir).arg("%1", "%2");


c) The following looks wrong (copy&paste error).
+  else if(QFile::exists(dir + "/Mail"))
+  {
+      rc = KMessageBox::questionYesNo(0, msg.arg(dir + "/KMail", dir + "/KMail"), title, buttonText);

This should surely be
+      rc = KMessageBox::questionYesNo(0, msg.arg(dir + "/Mail", dir + "/Mail"), title, buttonText);


d) Typo:
+  void transferMail(const QString& distinationDir);


e) Please make the new lines of code adhere to http://korganizer.kde.org/develop/hacking.html.
Comment 20 Aaron J. Seigo 2004-10-14 00:43:42 UTC
Created attachment 7865 [details]
updated patch addressing ingo's comments

here you go ... should address all the issues brought up =)
Comment 21 Ingo Klöcker 2004-10-14 20:34:17 UTC
Looks good now except for one little thing. The warning in the following message ("this may replace any existing files...") is superfluous and unnecessarily worries the user if destinationDir doesn't exist yet (which is the normal situation). Therefore this warning should be omitted if the destination folder doesn't exist.

+  QString msg = i18n( "The %1 folder exists. KMail now uses the %2 folder for "
+                      "its messages.\n"
+                      "KMail can move the contents of %3 into this folder for "
+                      "you, though this may replace any existing files with "
+                      "the same name in %4.\n"
+                      "Should KMail move the mail files now?" )
+                     .arg( dir, destinationDir, dir, destinationDir );

And another thing: If moving fails then the user should be informed about the failure. Hmm, if moving fails then we should probably not store the mail storage in the configuration, so that the user can give it another try on the next KMail start. He can still cancel the move if he doesn't want to try again.
Comment 22 Aaron J. Seigo 2004-10-14 21:36:05 UTC
Created attachment 7878 [details]
ingo suggests, i code

i also did some markup on the dialog message so it's a bit more readable.
Comment 23 Ingo Klöcker 2004-10-18 01:09:06 UTC
JFYI, I've tested the patch, found a few more issues and fixed most of them. I'll add a detailed comment and my updated patch tomorrow.
Comment 24 Ingo Klöcker 2004-10-18 21:48:51 UTC
I've finally tested the patch, found a few more issues and fixed most of 
them:
a) locateLocal( "data", "kmail/mail/" ) creates the folder if it doesn't 
exist. We have to use locateLocal( "data", "kmail/mail/", false ) 
instead. Fixed.

b) We have to use cfg->readPathEntry( "folders" ) and 
cfg->writePathEntry( "folders", foldersPath ) instead of readEntry and 
writeEntry in order to support roaming user profiles. Fixed.

c) The messages:
c1)
+                  "its messages. KMail can move the contents of %3 into 
"
should be
+                  "its messages. KMail can move the contents of 
<i>%3</i> into "
Fixed.
c2) Use kapp->aboutData()->programName() instead of hard-coding "KMail" 
in the messages which would be wrong for Kontact users. Fixed.

d) Tests:
d0) Keeping ~/KMail or ~/Mail works.
d1) ~/KMail is a folder on the same partition as $KDEHOME. Moving works.
d2) ~/Mail is a symbolic link to ~/Mail.test. Moving fails and then 
KMail aborts (SIGABRT):
Error message: "Symbolic link [...]/share/apps/kmail/mail/ could not be 
created. Please check the permissions."
Debug output:
kio (KIOJob): kio_uiserver registered
kio (KIOJob): stat file:/home/kdecvs/.kde-cvs/share/apps/kmail/mail/
kio (KSycoca): Trying to open ksycoca 
from /var/tmp/kdecache-kdecvs/ksycoca
kio (KIOJob): error 11 /home/kdecvs/.kde-cvs/share/apps/kmail/mail
kio (KIOJob): This seems to be a suitable case for trying to rename 
before stat+[list+]copy+del
kio (KIOJob): error 40 /home/kdecvs/Mail
kio (KIOJob): Couldn't rename, reverting to normal way, starting with 
stat
kio (KIOJob): stat file:/home/kdecvs/Mail
kio (KIOJob): copying /home/kdecvs/.kde-cvs/share/apps/kmail/mail/
kio (KIOJob): error 59 /home/kdecvs/.kde-cvs/share/apps/kmail/mail/
kio (KIOJob): Observer::open_SkipDlg job=0x818d080 progressId=3

--> Error message is shown. Click on Cancel.

kio (KIOJob): stat file:/home/kdecvs/.kde-cvs/share/apps/kmail/mail/
kio (KIOJob): error 11 /home/kdecvs/.kde-cvs/share/apps/kmail/mail
kmail: /home/ingo/cvs/kde/head/kdepim/kmail/kmfoldermgr.cpp:126: virtual 
void KMFolderMgr::setBasePath(const QString&): 
Zusicherung !aBasePath.isNull() nicht erfllt.
KCrash: crashing... crashRecursionCounter = 2
KCrash: Application Name = kmail path = <unknown> pid = 18925
ERROR: Communication problem with kmail, it probably crashed.

The failed move could probably be considered a bug in 
KIO::NetAccess::move(). 
Nevertheless we have to workaround this bug.

I've fixed the crash by changing the API of transferMail() from
  QString KMKernel::transferMail( const QString & destinationDir )
to
  bool KMKernel::transferMail( QString & destinationDir )
The crash (actually a failed ASSERT) occurred because transferMail 
returned an empty string which was then used as path to the mail 
storage. Now transferMail returns the valid path of the current mail 
storage in the call-by-ref parameter in any case while the bool return 
value indicates whether the transfer was successful.

The remaining issue is failed move. I didn't investigate the reason for 
the failed move.


Created an attachment (id=7943)
Aaron-Seigo-2004-10-14-use-hidden-Mail-folder-and-move-old-mail-ik-utf8.diff
Comment 25 David Faure 2004-10-18 21:59:05 UTC
On Monday 18 October 2004 21:48, Ingo Klöcker wrote:
> ------- Additional Comments From kloecker kde org  2004-10-18 21:48 -------
> I've finally tested the patch, found a few more issues and fixed most of 
> them:
> a) locateLocal( "data", "kmail/mail/" ) creates the folder if it doesn't 
> exist. We have to use locateLocal( "data", "kmail/mail/", false ) 
> instead. Fixed.

Can't one simply use locateLocal( "data", "kmail/" ) + "mail/" to test for existence
without creating?

> The failed move could probably be considered a bug in 
> KIO::NetAccess::move(). 

How so? What's the bug?
~/Mail is a symbolic link to ~/Mail.test and you're calling
move( "~/Mail", "/some/dir/on/another/partition" )?
This should work fine... I just tried in konqueror.
Maybe the dest dir doesn't exist?

Comment 26 Ingo Klöcker 2004-10-19 00:59:07 UTC
> Can't one simply use locateLocal( "data", "kmail/" ) + "mail/" to test for existence without creating?

Yes, we should probably do this so that $KDEHOME/share/apps/kmail/ is created if it doesn't exist.

> > The failed move could probably be considered a bug in 
> > KIO::NetAccess::move(). 
 
> How so? What's the bug?

Please reread d2). It contains all information (the error message and kio's debug output).

> ~/Mail is a symbolic link to ~/Mail.test and you're calling 

Yes:
lrwxrwxrwx   1 kdecvs users        8 2004-10-16 18:26 Mail -> Mail.test/
drwx------   9 kdecvs users      936 2004-09-27 00:04 Mail.test/

> move( "~/Mail", "/some/dir/on/another/partition" )?

No, KMail is calling NetAccess::move( "/home/kdecvs/Mail", "/home/kdecvs/.kde-cvs/share/apps/kmail/mail/" ).

Of course, /home/kdecvs/.kde-cvs/share/apps/kmail/mail/ doesn't exist. Okay, let's try NetAccess::move( "/home/kdecvs/Mail", "/home/kdecvs/.kde-cvs/share/apps/kmail/mail" ). In this case the link is moved, but of course now it points to the non-existing Mail.test folder in /home/kdecvs/.kde-cvs/share/apps/kmail/.

So, moving links doesn't work (either way). We have two options:
a) Don't propose to move ~/Mail if it's a symbolic link.
b) Propose to move the folder ~/Mail really points to.

I think we can safely go for a).
Comment 27 Don Sanders 2004-10-27 08:41:49 UTC
Just to reiterate my earlier position. I'm against moving ~/Mail to 
~/KMail for existing users. I think it is a bad idea and will lead to 
mail loss for first time KMail users for the reasons I gave before.

If I'm outvoted so be it, but I do want to 'be on the record' as 
objecting to changing ~/Mail to ~/KMail for existing users.

Don.

Comment 28 Thomas Zander 2004-10-27 09:10:27 UTC
And to reiterate my former reply (last parag #8) this is not about moving mail to ~/KMail
I'm really confused you still think it is; since there is a plethora of hints in the bugreport (last attachment name?) that this is about moving the user-visible folder out of the homedir..
Comment 29 Don Sanders 2004-12-21 09:28:59 UTC
> ------- Additional Comments From zander kde org  2004-10-27 09:10 -------
> And to reiterate my former reply (last parag #8) this is not about moving
> mail to ~/KMail I'm really confused you still think it is; since there is a
> plethora of hints in the bugreport (last attachment name?) that this is
> about moving the user-visible folder out of the homedir..

The location of the directory is irrelevant other than it being different from 
the current default. It's moving the default directory for storing mail that 
concerns me.

As I said for KMail to move that directory would be extremely rude to the 
point of being hostile from the point of a non KMail user trying KMail for 
the first time. 

Beyond that I noticed after Mandrake moved the directory several Mandrake 
users posted to usenet complaining that all their mail had been lost after 
upgrading. I infer that many others lost their mail but didn't bother to 
write about it.

Don.

Comment 30 David Faure 2004-12-21 11:15:52 UTC
On Tuesday 21 December 2004 09:29, Don Sanders wrote:
> Beyond that I noticed after Mandrake moved the directory several Mandrake 
> users posted to usenet complaining that all their mail had been lost after 
> upgrading. I infer that many others lost their mail but didn't bother to 
> write about it.

That's because Mandrake's patch was broken - don't confuse that, and a clean patch 
that achieves the same in a stable way.

Comment 31 Thomas Zander 2004-12-21 13:22:03 UTC
On Tuesday 21 December 2004 09:29, Don Sanders wrote:
> As I said for KMail to move that directory would be extremely rude to the
> point of being hostile from the point of a non KMail user trying KMail
> for the first time.

We have been here before, KMail does not currently behave well for those 
people anyway, since it has a different mail storing mechanism.  See 
comments #8, #9, #11
Additionally, using that as a reason to hold back this usability refinement 
leaves us in a deadlock, I would like to work with solutions instead of 
problems.
Aaron already provided a solution.  Lets work from there, please.

Thanks.
Comment 32 Don Sanders 2004-12-22 05:20:38 UTC
I'm not sure why the Mandrake patch being broken is supposed to increase my 
confidence that the ~/Mail folder can be moved without problems. It has the 
opposite effect of reinforcing my view that it's dangerous.

Regardless my main concern remains "for KMail to move that directory would be 
extremely rude to the point of being hostile from the point of a non KMail 
user trying KMail for the first time. "

Beyond that if Ingo and another core KMail developer support it then I think 
you have the right to commit the patch. (Please understand that, see the 
COMMITPOLICY in kmail cvs).

I simply want to be on the record that I think the patch is dangerous, will 
lead to mail loss, hurt feelings and harm the reputation of KMail. If there's 
support for renaming the Mail directory then I think the sensible thing to do 
would be to change the default rather than move the directory for existing 
users.

Don.

Comment 33 S. Burmeister 2004-12-22 10:53:31 UTC
I have to agree that having a .maildirectory is more dangerous than the current Mail directory. When unexperienced users are told to save their home-directory before re-installing a distro, they will most likely use konqueror and copy only the non-hidden files, which will let them lose all their mail. Yet I see the point about using a different dir, as well as hiding it. This might actually prevent non-experienced users from deleting it by mistake.

I think that the best solution for this issue is actually not so much about the where and how of the folder, but about providing a functionality that makes it possible not to care about the whereabouts of the folder.

http://bugs.kde.org/show_bug.cgi?id=85656 for example is aiming for a export functionality that allows the user to export ones mail and settings. This way all users have a very easy and comfortable way of saving their data, restoring it, as well as do not need to know where the folder is situated, i.e. it can be hidden.

For moving people's folder: there should be a notification when starting kmail, in order to inform the user about the move.
Comment 34 David Faure 2004-12-22 11:28:54 UTC
> I have to agree that having a .maildirectory is more dangerous than the current Mail directory. 
> When unexperienced users are told to save their home-directory before re-installing a distro, they will most likely 
> use konqueror and copy only the non-hidden files, which will let them lose all their mail 

If you do that you lose all your bookmarks, wallet passwords, and of course configuration etc.
I don't think this is really a valid point.

Comment 35 Patrick Audley 2004-12-23 05:01:23 UTC
I think hiding something as critical as a user's mail is a really, really bad idea..  Configuration files can be reproduced, but mail generally can't.
Comment 36 Gav Wood 2005-01-21 17:10:20 UTC
i notice this thread appears to have stalled. it would be (have been?) nice to have the patch ready for 3.4.0.

to address #33: i wouldn't expect to have my mail contained in my visible file heirarchy, since i dont use file management paradigms to access my mail. since i only ever see my mail in kmail, as a user i would expect the logical place to access may mail, including for backup to be kmail.

in any case, if the user is clever enough to think about copying all their files in their home directory as a backup (and not e.g. just their desktop), then i reckon there's probably a good chance they'll find the "Show hidden files" option too, which would backup their mail, configuration, bookmarks, etc. successfully.

please please please get this sorted out soon, if only for the dead.letter rubbish that gets left behind on my desktop (which for right or wrong i have set to $HOME). with 285 votes, call my totalitarian, but i reckon it's time to bite the bullet and give the patch a go in the primetime.

gav
Comment 37 S. Burmeister 2005-01-21 17:39:43 UTC
> > I have to agree that having a .maildirectory is
> > more dangerous than the current Mail directory. 
> > When unexperienced users are told to save their
> > home-directory before re-installing a distro, they will most likely 
> > use konqueror and copy only the non-hidden files, which will let
> > them lose all their mail 
 
> If you do that you lose all your bookmarks,
> wallet passwords, and of course configuration etc.
> I don't think this is really a valid point.

Actually you support my comment. First of all users should not be forced to copy any files but get a tool instead. Anyway, you are only right about losing the config. Yet I want to lose my config when changing distro version, because a clean KDE install worked better for me in the past.

The most important point you made, without the intention of doing so I guess, is that one does not lose bookmarks and/or passwords when not knowing where they are because both can be exported to a file and then re-imported.

Regarding comment #36: A lot of people who now switch to Linux are used to XP which hides files as well. I frequently see postings in newsgroups where people do not even know about hidden files.

An export-functionality is the only proper way to allow every user to save and migrate its data. I guess nobody will deny that chances of forgetting/not knowing about data is increased by hiding it.
Comment 38 Gav Wood 2005-01-21 17:54:09 UTC
> The most important point you made, without the intention of doing so I
> guess, is that one does not lose bookmarks and/or passwords when not
> knowing where they are because both can be exported to a file and then
> re-imported.

yet we don't have a $HOME/Bookmarks or $HOME/Passwords.

then any question of backup or export should be dealt with internally in 
kmail. moving $HOME/Mail out of the way, as with every other internal data, 
should proceed unhindered.

gav

Comment 39 S. Burmeister 2005-01-21 18:07:26 UTC
> > The most important point you made, without the intention of doing so I
> > guess, is that one does not lose bookmarks and/or passwords when not
> > knowing where they are because both can be exported to a file and then
> > re-imported.
>
> yet we don't have a $HOME/Bookmarks or $HOME/Passwords.
>
> then any question of backup or export should be dealt with internally in
> kmail. moving $HOME/Mail out of the way, as with every other internal data,
> should proceed unhindered.

Nope, IMHO safety comes first and as hiding data increases the risk, the 
export function has to come first. It does not make sense to increase the 
risk of losing data only to get an icon or two less on the desktop or in a 
directory-view.
There is no need for $HOME/Bookmarks as there is an export-functionality 
already. And email is not every other data, as another commentator already 
pointed out. Email cannot be brought back, configs, bookmarks, passwords etc. 
can. If you want to compare email-data you have to compare it to documents or 
photos (which most people also only access via a GUI), which is not a hidden 
directory either.

Comment 40 Gav Wood 2005-01-21 18:24:00 UTC
> Nope, IMHO safety comes first and as hiding data increases the risk, the
> export function has to come first.

sadly as there are others apart from me that use kde, i gotta respect this 
view :-(

then perhaps the patch could be introduced as a visible, non-default (for now) 
option, for those of us that either:

1. dont use local mail storage (imap only) (me)
or
2. understand enough to know to backup ~/.kde/ (me)
or
3. understand enough to be able to move local mails to a remote (imap) folder 
for backup. (me)
(or, for the cynical, "4. haven't a clue about backing up and would lose their 
mail anyway.")

gav
Comment 41 Thomas Zander 2005-01-21 19:19:49 UTC
On Friday 21 January 2005 18:07, S.Burmeister wrote:
> Nope, IMHO safety comes first and as hiding data increases the risk, the
> export function has to come first. It does not make sense to increase the
> risk of losing data only to get an icon or two less on the desktop or in
> a directory-view.

There is an application that does do backups for you already, in the manner 
you request. Its not currently being shipped in the default KDE setup, 
though.
Also note that we have had a number of people remove the (K)Mail dir since 
they did not understand that it contained their mail exactly due to the 
point made earlier; they never use the Mail dir themselves.

In the comments you can find quite a number of ways that the current setup 
can cause the user to loose his email in daily usage.  From using other 
mail clients to people cleaning up their homedir.  If safety comes first; 
these points should not be dismissed because no dedicated backup is in 
place just yet.
People can still backup their complete homedir, which is still advised 
anyway.
Comment 42 S. Burmeister 2005-01-21 20:11:09 UTC
> There is an application that does do backups for you already, in the manner
> you request. Its not currently being shipped in the default KDE setup,
> though.

So it should be integrated and the functionality accessible via the same menu 
the import-functionality is accessible. I would even prefer a GUI that 
incorporates all export-functionalities so that the user can easily save and 
migrate all its data (email, bookmarks, passwords, documents, news-feeds, 
addresses, calendar etc.) and config 
(email-identities/transport-methods/filters etc.).

By how much do you think this would ease migration as well as save people from 
losing data?

> Also note that we have had a number of people remove the (K)Mail dir since
> they did not understand that it contained their mail exactly due to the
> point made earlier; they never use the Mail dir themselves.

Certainly there are a bunge of possibilities to delete mail. Yet I still claim 
that those people deleting the Mail-directory are the minority in the group 
of people that lose their email. One might even be tempted to say that a 
person who deletes a directory called Mail would lose its email anyway, so 
the hiding would only pospone the loss.

> In the comments you can find quite a number of ways that the current setup
> can cause the user to loose his email in daily usage.  From using other
> mail clients to people cleaning up their homedir.  If safety comes first;
> these points should not be dismissed because no dedicated backup is in
> place just yet.

So one would have to do a survey to know which method is less safe. Yet that 
would not really help. Although I cannot prove it my thesis is that if one 
looks at the total number of people losing their email-data, the decrease 
because of not deleting it by mistake, would be exceeded by the increase of 
data-loss due to it being hidden and thus forgotten/not known about.
The only thing that really helps, is to provide both mechanisms simultaniously 
in order to avoid the shortcommings of either way to solve this issue.

Remove the data from the space it can easily be deleted, offer a functionality 
that gives the user easy access to migrate and save its data.

> People can still backup their complete homedir, which is still advised
> anyway.

You are right, yet unfortunately unexperienced users hardly read any docs and 
thus do not find any advice. One could even argue that whoever finds that 
kind of advice will have a look at google or the docs before deleting the 
Mail-folder anyway.

Comment 43 Thomas Zander 2005-01-21 20:37:34 UTC
On Friday 21 January 2005 20:11, S.Burmeister wrote:
> So one would have to do a survey to know which method is less safe. Yet
> that would not really help. Although I cannot prove it my thesis is that
> if one looks at the total number of people losing their email-data, the
> decrease because of not deleting it by mistake, would be exceeded by the
> increase of data-loss due to it being hidden and thus forgotten/not known
> about.

Do you base that on the assumption that people migrate to new 
installations/computers a lot?
That seems out of touch with reality; any linux distro allows upgading 
without formatting, for example.
Please ask around how many people that use 1 computer operating system 
(which is an assumption if you have your mail locally) have migrated to new 
hardware or a new OS.  Formatting their harddrive.  Let us know how many 
you found (that are not system-maintainers themselves :)

No, all the prove points in the other direction.
Keeping a directory out it the open in the end means people can and will use 
it, which they are not suppost to do.  And this started because we want to 
encourage people to do as they please with non-hidden directories.

> Remove the data from the space it can easily be deleted, offer a
> functionality  that gives the user easy access to migrate and save 
> its data. 
This we agree on; and the first part is what this bug is suppost to fix; the 
other part is out of scope for this bugreport.  But see comment #14 for a 
solution in that department.

Comment 44 S. Burmeister 2005-01-21 21:21:30 UTC
> > So one would have to do a survey to know which method is less safe. Yet
> > that would not really help. Although I cannot prove it my thesis is that
> > if one looks at the total number of people losing their email-data, the
> > decrease because of not deleting it by mistake, would be exceeded by the
> > increase of data-loss due to it being hidden and thus forgotten/not known
> > about.
>
> Do you base that on the assumption that people migrate to new
> installations/computers a lot?

How often do you think people think about deleting their Mail directory? 
Anyway, you got a point although a lot of people I know get a new distro 
every six months, which is approximately the SuSE schedule. If you have a 
look at the newsgroups you will also notice that although upgrading might be 
standard behaviour it is not the best way to go, as it always leads to 
problems. Me updating KDE to 3.4 beta1 caused problems with kickerrc, quanta 
shortcuts etc. I should have done a clean install of KDE thus deleting 
the .kde directory. So actually one does not even have to change/upgrade the 
distro but simply a clean install of KDE including deleting 
the .kde-directory.
Still, I see your point and I am not against hiding the directory but still 
favour the option to pospone it until a proper export-function is available.

> That seems out of touch with reality; any linux distro allows upgading
> without formatting, for example.
> Please ask around how many people that use 1 computer operating system
> (which is an assumption if you have your mail locally) have migrated to new
> hardware or a new OS.  Formatting their harddrive.  Let us know how many
> you found (that are not system-maintainers themselves :)
>
> No, all the prove points in the other direction.
> Keeping a directory out it the open in the end means people can and will
> use it, which they are not suppost to do.  And this started because we want
> to encourage people to do as they please with non-hidden directories.

As I said, I am not against hiding it just against seeing this issue seperate 
from an export-funtionality. If both are introduced, I would appreciate it.

> > Remove the data from the space it can easily be deleted, offer a
> > functionality  that gives the user easy access to migrate and save
> > its data.
>
> This we agree on; and the first part is what this bug is suppost to fix;
> the other part is out of scope for this bugreport.  But see comment #14 for
> a solution in that department.

I could not find any real information on KeepIt with google. Anyway, I guess 
it is a program that incorporates all export features, as well as import 
features that are available accross the main KDE apps and that it will be 
available from the kmail-menus. When is going to be shipped with KDE? Who 
counts the complaints about data-loss until then? ;)

Comment 45 Ingo Klöcker 2005-01-26 23:37:59 UTC
CVS commit by kloecker: 

Change autosaving:
- Autosave messages in the maildir folder $KDEHOME/share/apps/kmail/autosave instead of in $HOME/dead.letter (mbox-style).
CCBUG: 72441
- The composers are now responsible for timer-based autosaving.
- The changes in recoverDeadLetters are partially cosmetic and partially due to the change from an mbox-style autosave folder to a maildir autosave folder.
- Add KMKernel::localDataPath() which simply returns locateLocal( "data", "kmail/" )
- The actual code which does the autosaving is simplified by using KSaveFile. Moreover, the buggy code which wrote the message (the bug being that the saved message isn't From-escaped) gets easier because with maildir we just need to write the message, but nothing else.
- Show an error message when autosaving fails.


  M +141 -60   kmcomposewin.cpp   1.915
  M +35 -8     kmcomposewin.h   1.273
  M +39 -60    kmkernel.cpp   1.320
  M +7 -3      kmkernel.h   1.129



Comment 46 Ingo Klöcker 2005-02-03 00:23:24 UTC
CVS commit by kloecker: 

Use /home/ingo/.kde-cvs/share/apps/kmail/mail by default as folder storage (instead of ~/Mail). Moving is disabled for now because it doesn't always work and is IMO anyway to dangerous because there's no safe rollback possible unless we copy the whole mail storage and only delete the origin if the transfer succeeded.
Comment 47 Ingo Klöcker 2005-02-03 00:43:38 UTC
I just want to add one comment:
The user should always backup $HOME before updating. If only selected files/folders are backed up then the chances are extremely high that at least one important file/folder is forgotten.

In fact the best solution is putting /home on a separate partition. I'm doing this since a long time and therefore updating never was a problem for me although I'm using SuSE since ever.
Comment 48 Jedd 2005-03-01 14:13:49 UTC
Ingo's right, but there's a lot of things that the user *should* do .. and a roughly equal number of things that the user typically does *not* do.

I operate between a desktop and a laptop, and shuffle data (via a .tgz) between the two - private, work, & config files - each time I land back home, and then again in the other direction when I leave.

I include ~/Mail in that shuffle.

Because my laptop's typically on dial-up, and the desktop's always on ADSL, the latter tends to be slightly ahead in versions of KDE.  They're *usually* in sync, but it's not guaranteed.  I'm using Debian unstable/experimental, so the disparity is sometimes significant.

Was I pleased when ~/dead.letter started doing weird things between the two machines?  No, not particularly.  Would I be pleased to have ~/Mail gone from my home directory?  Yes, probably.  Would I be thrilled to have to find it and work out what other files I may now need to add to my shuffle.sh script?  Unlikely.  Will I be ecstatic to find that my 1GB collection of Mail suddenly is duplicated, and my shuffle.tgz file ends up being twice the size one rainy afternoon?  (That's a rhetorical one...)

There are already significant problems with the mix of user-specific and machine-specific files that are scattered under ~/.kde/ -- there's plenty of examples of config files that contain both types of information, and consequently make migration painful.  Once-off migration as well as the regular scheduled variety.

Changes like this should be very carefully considered, with lots of information (or at least the option to see lots of information) given to the user.  Sometimes people operate between two relatively minor versions of KDE - I'm sure I'm not alone in this kind of oscillation, but there's also the far more common case of people backing out updates, and reverting to an earlier package (to use the vernacular).  My point is -- not everyone uses their computer exactly the same way you* do.

Jedd.

* Note that 'you' is not aimed at any one person.  I've read through this thread and seen lots and lots of generalisations uttered by lots of people.  I also see lots of people saying out loud how careful they need to be, so if you prefer to, then treat this as an echo of those cautions.  ;)


Comment 49 Don Sanders 2005-03-17 05:57:57 UTC
Thanks Ingo. I didn't look at the diff yet, but from reading your last 
few comments I think the changes you made were appropriate, desirable 
and sensibly restrained, thank you.

Don.