Bug 79357 - Folder compression/dismissal wish in kmail
Summary: Folder compression/dismissal wish in kmail
Status: CONFIRMED
Alias: None
Product: kmail2
Classification: Applications
Component: general (show other bugs)
Version: unspecified
Platform: unspecified Linux
: NOR wishlist
Target Milestone: ---
Assignee: Volker Krause
URL:
Keywords:
: 48217 95611 (view as bug list)
Depends on:
Blocks:
 
Reported: 2004-04-09 23:09 UTC by kaplun
Modified: 2011-11-13 19:21 UTC (History)
7 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description kaplun 2004-04-09 23:09:06 UTC
Version:           1.6.2 (using KDE 3.2.2, compiled sources)
Compiler:          gcc version 3.3.3
OS:          Linux (i686) release 2.4.25-lck1

Will it be possible to compress folders in order to create archives of old mail? Maybe passing them though bzip2...
Comment 1 Jens Bäckman 2004-04-10 21:55:44 UTC
I'm looking for something small to get my feet wet in KDE, so I might just take on this feature and implement it.

As I understand your request, you want to be able to right-click a folder, select "Archive...", enter a filename and receive an archive.tar.bz2 with all the messages in separate files. Perhaps even store subfolders as a selectable option.

Have I interpreted your request correctly? What does the KMail wizards say - any objections to such a solution?
Comment 2 kaplun 2004-04-11 13:47:15 UTC
Well, I was thinking about giving the option of telling KMail we want some folder to be kept compressed (Am I using the right tense?). This could result in less space used on disk but more time need to read emails because of the compression (this option would look like the one to tell kmail that a folder contains a mailing list). When you select it, such folder would be immediatly compressed and every new mail added too. If deselect it would return as a normal folder.
I was thinking about something nearly transparent to the user. He/she tell kmail to keep a folder compressed and then he/she could forget about it :-)
Comment 3 kaplun 2004-04-11 14:43:43 UTC
Your idea is good, too. But isn't it more related to konqueror and its service menus?
Comment 4 Michel Nolard 2004-08-12 22:32:04 UTC
I am precisely looking for such a feature. For my own, I clearly see a simple solution to be implemented into kmail : make use of the Kio slaves : bzip2:/, zip:/, file:/, ...

If this is done in a proper manner, there are the things that should be added to kmail in order to give to a user a complete control over the process, while he/she do not have to bother with complex options.

* In the main windows, the folder should have a specific property giving to the user a mean to choose wheither or not they want to compress it.
A good way is to put a checkbox stating "compress folder" which unreveals a disabled zone in the dialog box. This zone would contain the following abilities : specify the default algorithm (choosing the default one in the list, see the next "*"), a radio buttons group giving the user the choice to compresse "always" or "when size limit reached" (in this case a spin button is given to select the size in Kb, Mb or in Messages).

Beware that the user does not know that when using compression his mail can become unreadable with other mail clients. So, the "folder properties" dialog box should display a remark for this ...

* In the KMail Settings dialog box, a choice for "by default compression folder algorithm" should be given. The help hints for this should state something like :
 "gzip" = "this algorithm is faster than bzip2 although it compresses less.
 "bzip2" = "this algorithm uses the fastest compression algorithm available to KDE, while this is the slower one."
There should be a way for the user to tell how to set the default compression properties for the new folders. By example, he/she can tell KMail to do the following by default : "always compress folders with the default compression algorithm" or "compress folders with the default compression algorithm when their size is 200 messages or 50Mb". I hope you see the idea.

* There should be a kmail action to "force disabling compression throughout KMail", thus putting every compressed folder into uncompressed state, and another one to "force folder compression throughout KMail", so changing the compression of each folder to the default compression scheme. Maybe a dialog box should ask if the user wants to enforce the compression algorithm or only replace not compressed folders with compressed ones using the default scheme.

* All this should be done with the idea that a user can make a mistake. If he has 10 000 of messages and select "compress folder" by mistake, having choosen the bzip2 compression algorithm, he/she can be very disappointed to see that minutes pass and the compression is still running. So, there should be a clean way for the user to cancel the operation without loosing too much time.

* Moreover, the user does not always know what size a folder takes. So, the size of each individual folders and subfolders should be displayed in a human readable manner, using units like Kb, Mb, Gb.

* Finally, it would be VERY wise to give the user a mean to make archives of the folders, in the sense that was proposed earlier in this thread. Something like "Archive every mail in this folder (and possibly subfolders) to a file and possibly remove them from their actual place".

I hope all this long text will help. I really think that this option is VERY useful.
Comment 5 Tom Albers 2004-08-12 22:43:13 UTC
Michael Jahn found out these are all very similar, yet not similar enough to be dupes: bug 55827, bug 77744, bug 85656, bug 34382 and bug 76690. Just so you know.
Comment 6 Thiago Macieira 2005-01-07 01:14:57 UTC
Actually, this bug here is a closer duplicate to Bug #95611. All the other ones mentioned in comment #5 ask for archiving (i.e., exporting) all emails, some emails, and/or KMail configuration. This bug here and 95611 ask for compressed maildir folders.

I think I will try and hack this myself.
Comment 7 Helge Hielscher 2005-03-25 12:59:20 UTC
http://wiki.dovecot.org/CoreFeatures is also talking about gziped mbox and gziped maildir features. Personally I think that are better choices than gzip, like lzo. 
On the other hand it is questionable if it makes sense to implement compression at an application level since there are many filesystems with built in compression, as Reiser4.1 [1], e2compr [2], NTFS, cramfs [3], CVF-FAT [4], Squashfs [5], zisofs [6], cloop [7], etc.p.p.

[1] http://www.namesys.com/v4/v4p1boem.html
[2] http://e2compr.sourceforge.net/ 
[3] http://sourceforge.net/projects/cramfs/
[4] http://www.kernel.org/pub/linux/kernel/people/marcelo/linux-2.4/Documentation/filesystems/fat_cvf.txt
[5] http://squashfs.sourceforge.net/
[6] http://freshmeat.net/projects/zisofs-tools/
[7] http://www.alextreme.org/phpwiki/index.php/cloop
Comment 8 Michel Nolard 2005-03-25 13:42:59 UTC
IMHO, there is are big differences between a partition and a file in terms of use, efficiencies (regarding time AND space),... as both were not designed while expecting the same kind of use.

I think you've forgotten that this [Wish] does not imply dropping compressed file systems, nor forcing the user to compress its mail folders. This is only a [Wish] for a feature which would have the form of kmail options, nothing more.

May I recall you, too, that on embedded machines, this can be quite interesting to spend time to uncompress data only in very specific areas, like mail folders, and not on the entire file system.

Moreover if my mailbox takes too much space using gzip compression algorithms, I want to be able to change this and choose another compression system. This can't be achieved easily and seamlessly with file systems.

For the differences between file system and regular files, I dare to list some capabilities which do not fill standard KDE usability requirements, like
 - creation
 - renaming
 - moving
 - copying
 - encryption
 - digital signing
 - remote access
 - backup

To be sure to make myself clear, I think the debate may only take place based upon users respect which encompasses ease of use, seamless operations, this should not be mandatory to be Unix/Linux guru, etc. I remember to the reader that RTFM (Read The F*** Manual, for those who don't know) is neither a user friendly reply, nor does it enforce the Responsible Principles by the developer...
Comment 9 Helge Hielscher 2005-03-25 16:26:27 UTC
As a reference – the discussions of Mozilla and Ximian:
https://bugzilla.mozilla.org/show_bug.cgi?id=76852
http://bugzilla.ximian.com/show_bug.cgi?id=23621
Comment 10 Martin Steigerwald 2005-08-06 00:01:56 UTC
This bug is a duplicate of bug #48217. Seems to be a long term wish. ;)

I would also like such a feature - like it is available in YAM for AmigaOS for example (http://yamos.sourceforge.net offering different compression technologies the modular library system XAD). 

I am currently using mbox folder for mail archival as I believe for archival on XFS one big file is better than tons of small files. Thus however archival is done, I would like when the result would rather be one big file than lots of small files.

I just packed a mbox folder with the first 5 months (almost, I lost some mails for January) of debian-user-german, thats 17119 e-mails with bzip2. The results look promising:

-rw-------  1 martin martin 83900466 2005-08-05 23:34 debian-user-german-2005-1
-rw-------  1 martin martin 10837074 2005-08-05 23:34 debian-user-german-2005-1.bz2

The choice of the compression technology needs some thought tough. While bzip2 yields good compression ratios, it wouldn't do well when a certain file of an archive with lots of files is to be accessed cause it has no directory index. That's one of the reasons why OpenOffice.org choosed ZIP (there is a comparison of advantages and disadvantages of compression technology in regard to the needs of OpenOffice.org somewhere on the net, I did not find it right now).
Comment 11 Thiago Macieira 2005-08-06 18:32:58 UTC
*** Bug 95611 has been marked as a duplicate of this bug. ***
Comment 12 Thiago Macieira 2005-08-06 18:33:08 UTC
*** Bug 48217 has been marked as a duplicate of this bug. ***
Comment 13 Thiago Macieira 2005-08-06 22:00:50 UTC
*** This bug has been confirmed by popular vote. ***
Comment 14 Maciej Pilichowski 2008-11-08 12:07:52 UTC
Few notes:
a) the compression algorithm should be transparent to KMail, i.e. user should pick up the program for compression and pass the arguments, hardcoding the compression is not a good choice because each time there is new alg. fixes to KMail should have been done

a.1) not matter what KMail should always provide size guarantee, i.e. if after compression the size is bigger, no compression is done. Maybe even user could set some limit, saving should be bigger than 5% and 500B

b) while maildir is obvious (compression per file), mbox is maybe not -- mbox should not be compressed as a file, but incrementally (each part separately), otherwise deletion of the mail in the middle will kill the user

c) maybe better idea would be to start from "compress this mail" feature, having that, folder compression is just a matter of GUI, this would also lead to more flexible behaviour, user could have uncompressed folder with some bigger mails compressed 

c.1) this shows that even having in KDE compressed folders/partitions does not help to solve this fully, because logic lies in mail and only KMail knows the meaning of the files 
Comment 15 subscryer 2008-12-28 17:02:59 UTC
This would be very nice. I'm using squasfs to store very large maildirs as compressed files, however when kmail is up for a reindex of those folders it becomes an IO hog therefore it's a sub-par solution.
Should someone implement this feature the indexing/reindexing of folders should be taken into account.
Comment 16 Thiago Macieira 2008-12-28 19:49:30 UTC
Forget this.

KMail will use Akonadi, which in turn uses MySQL, though I don't know if also for bulk storage. In any case, reassigning to Akonadi so we can get a new opinion.