Bug 358116

Summary: messageviewer leaks temporary files with mail parts in it
Product: [Applications] kdepim Reporter: Martin Steigerwald <Martin>
Component: messageviewerAssignee: kdepim bugs <kdepim-bugs>
Status: RESOLVED FIXED    
Severity: normal CC: faure, montel, simonandric5, sknauss, thomas, winter
Priority: NOR    
Version: GIT (master)   
Target Milestone: ---   
Platform: Other   
OS: Linux   
Latest Commit: Version Fixed In:
Sentry Crash Report:

Description Martin Steigerwald 2016-01-17 11:04:08 UTC
(Would be nice to have a messageviewer component, I may file an admin request about that later.)

messageviewer leaks temporary files in /tmp with parts of mails in it. According to Sandro already in KDEPIM 4 times it needs these temporary files to open attachments, but it should clean a file when it opens another mail. Yet, here it appears it doesn´t.

Reproducible: Always

Steps to Reproduce:
1. Run KMail.
2. Read some mails. (Maybe also some with attachments).

Actual Results:  
merkaba:~> date                                                                                                                                              
So 17. Jan 11:50:10 CET 2016

merkaba:~> ls -ld /tmp/messageviewer_* | wc -l
117

merkaba:~> LANG=C du -sch /tmp/messageviewer_* | tail -1
368K    total

This looks like this. I changed user and group names. At the time of this listing "user2" was not even logged in anymore, neither KMail nor Akonadi running for the user at the time.

# ls -ld /tmp/messageviewer_*
drwx------ 2 user2 users  60 Jan 13 11:17 /tmp/messageviewer_a16100.index.1
drwx------ 2 user1 group1 60 Jan 15 18:19 /tmp/messageviewer_A22454.index.
drwx------ 2 user1 group1 60 Jan 17 11:32 /tmp/messageviewer_a24855.index.1.1.1
drwx------ 2 user1 group1 60 Jan 12 18:17 /tmp/messageviewer_AJ6125.index.1
drwx------ 2 user2 users  60 Jan 13 09:42 /tmp/messageviewer_B16100.index.
drwx------ 2 user1 group1 60 Jan 12 18:56 /tmp/messageviewer_Bc6125.index.
drwx------ 2 user2 users  60 Jan 13 11:37 /tmp/messageviewer_C16100.index.
drwx------ 2 user1 group1 60 Jan 16 10:32 /tmp/messageviewer_c24855.index.1
drwx------ 2 user1 group1 60 Jan 12 15:11 /tmp/messageviewer_CD6125.index.1
drwx------ 2 user1 group1 60 Jan 14 19:49 /tmp/messageviewer_d21362.index.
drwx------ 2 user1 group1 60 Jan 13 21:07 /tmp/messageviewer_DM6125.index.1
drwx------ 2 user1 group1 60 Jan 13 14:05 /tmp/messageviewer_Dp6125.index.1
drwx------ 2 user2 users  60 Jan 13 17:10 /tmp/messageviewer_e16100.index.
drwx------ 2 user1 group1 60 Jan 15 22:07 /tmp/messageviewer_e22454.index.
drwx------ 2 user1 group1 60 Jan 14 20:55 /tmp/messageviewer_E22454.index.
drwx------ 2 user1 group1 60 Jan 15 19:08 /tmp/messageviewer_e22454.index.1
drwx------ 2 user1 group1 60 Jan 16 14:23 /tmp/messageviewer_E24855.index.1
drwx------ 2 user1 group1 60 Jan 12 18:55 /tmp/messageviewer_eQ6125.index.
[… rest skipped, but available on request…]


Expected Results:  
No file leaks.

While messageviewer protects the files with chmod 600, it is still an information leak when the original data is saved crypted on disk, like for example I have it with user2. The home directory of user2 is crypted via ecryptfs.

This happens in git master as of de33276f801787d5b54cb42728137d17bfd59762, but also before already I think.
Comment 1 Martin Steigerwald 2016-01-17 11:09:41 UTC
Almost all of the directories still have files in it:

merkaba:/tmp> find -name "*unnamed*" | wc -l 
116

However quite some of them are empty:

merkaba:/tmp> find -name "*unnamed*" -empty | wc -l
50

merkaba:/tmp> find -name "*unnamed*" | head
./messageviewer_V24855.index./unnamed
./messageviewer_t24855.index.1/unnamed
./messageviewer_y24855.index./unnamed
./messageviewer_i24855.index.1/unnamed
./messageviewer_Z24855.index.1/unnamed
./messageviewer_o24855.index.1/unnamed
./messageviewer_J24855.index./unnamed
./messageviewer_R24855.index./unnamed
./messageviewer_p24855.index./unnamed
./messageviewer_P24855.index./unnamed
[…]

The non empty files contain the body text of mails (even of ones without attachments).
Comment 2 Laurent Montel 2016-01-17 16:37:29 UTC
Do you have a test case when it leaks ?
Because it cleans them when we switch from mail (after 10 secondes)
Comment 3 Martin Steigerwald 2016-01-17 16:58:09 UTC
Hm, Laurent, I clicked through some more mails and the count of messageviewer directories in /tmp didn´t increase:

merkaba:/tmp> ls -ld messageviewer_* | wc -l
117

Hmmm, okay, it does when I open a mail with attachments:

merkaba:/tmp> ls -ld messageviewer_* | wc -l
123

Hmmm, and indeed it cleans up:

merkaba:/tmp> ls -ld messageviewer_* | wc -l
117

Okay, so now good question why it didn´t with the 117 directories in there. I have no idea yet.
Comment 4 Laurent Montel 2016-01-17 18:01:20 UTC
For me it's when kmail crash.
I will look at how to clean up it during crash.
Comment 5 Martin Steigerwald 2016-01-17 18:31:17 UTC
Hmmm, I had some crashes when I had KMail running during kdesrc-build. But 117 files – even it at one day I had two copies of KMail run (one for work user, one for private user)? Well, I am happy to test any fix you implement from git master :). Thank you.
Comment 6 Sandro Knauß 2016-01-29 10:17:15 UTC
Well I can reproduce this behaviour with a simple testcase.  This one fails:
    QString path;
    {
        NodeHelper helper;
        path = helper.createTempDir(QStringLiteral("foo"));
        QVERIFY(QDir(path).exists());
    }
    QEventLoop loop;
    QTimer::singleShot(0, &loop, &QEventLoop::quit);
    loop.exec();
    QVERIFY(!QDir(path).exists());

if I change the singleShot to something like 11000 (11secs) if passes.

The problem here is that if the NodeHelper instance is deleted the tempfiles are deleted also only 10secs after the application and not directly with the deletion of NodeHelper. I see no need to wait 10secs when the object is already deleted.

NodeHelper::~NodeHelper()
{
    //Don't delete it it will delete in class with a deleteLater;
    if (mAttachmentFilesDir) {
        mAttachmentFilesDir->removeTempFiles();
        mAttachmentFilesDir = 0;
    }
}

void AttachmentTemporaryFilesDirs::removeTempFiles()
{
    QTimer::singleShot(d->mDelayRemoveAll, this, &AttachmentTemporaryFilesDirs::slotRemoveTempFiles);
}

^^ d->mDelayRemoveAll is 10secs.
Comment 7 Sandro Knauß 2016-01-29 10:30:12 UTC
I've created a patch that should improve the situation:
https://phabricator.kde.org/D883

Still we have no handling, if the application crashes, but we now delete the temp dirs directly, if the object is deleted. It would be great to test it in the wild.
Comment 8 David Faure 2016-08-01 09:48:54 UTC
This was fixed (for the case of cleanly quitting kmail) in c803a7a (v16.07.80). Please test.

We can't do much in case of crashes -- except fixing those crashes in the first place, of course :-)
Comment 9 Thomas Monjalon 2017-08-31 21:10:32 UTC
I still see some messageviewer files leaked in /tmp:
    messageviewer_ATD826.index.1
    messageviewer_Fkk826.index.1.1
    messageviewer_GzU826.index.1
    messageviewer_HrO826.index.
    messageviewer_JXF826.index.1
    messageviewer_JhV826.index.

It seems created for every mails I open, even without attachment.

I use Kmail 5.6.0 from Archlinux.

Can we re-open the bug?
Which information should I provide to help finding the root cause?
Comment 10 Allen Winter 2017-08-31 21:18:07 UTC
confirmed.
I have dozens of messageviewer_foo.index.1 dirs in /tmp
each of them contains a file called "unnamed"
Comment 11 Laurent Montel 2017-09-01 05:28:26 UTC
Git commit 1a6d1763e10e6b231fccca9b01d351c2c94b77a7 by Montel Laurent.
Committed on 01/09/2017 at 05:28.
Pushed by mlaurent into branch 'master'.

Fix Bug 358116 - messageviewer leaks temporary files with mail parts in it

when we crash kmail too.

M  +21   -0    src/kmkernel.cpp
M  +2    -4    src/kmkernel.h

https://commits.kde.org/kmail/1a6d1763e10e6b231fccca9b01d351c2c94b77a7
Comment 12 Thomas Monjalon 2017-09-01 07:22:09 UTC
Wow! So efficient!
Thanks Laurent
Comment 13 Laurent Montel 2017-09-01 08:00:20 UTC
But there is still a case where we can't fix, it's when you launch kmail and use CRTL+C => no crash and kmail can't delete by itself temp file.

it's not possible to manage this case.
But we reduce the number of temporary file.
Comment 14 Thomas Monjalon 2017-09-01 08:11:42 UTC
About Ctrl+C, not a big issue, but strange. Issue with catching SIGINT?

About your fix, is it cleaning temp files only on crash?
My issue is that these files are filling /tmp when kmail is running.
Is it possible to avoid creating these files at the first place?
Or could they be created in a subfolder of /tmp?
Comment 15 Laurent Montel 2017-09-01 08:55:26 UTC
(In reply to Thomas Monjalon from comment #14)
> About Ctrl+C, not a big issue, but strange. Issue with catching SIGINT?
> 
> About your fix, is it cleaning temp files only on crash?

Yep and others fix was already done.

> My issue is that these files are filling /tmp when kmail is running.
> Is it possible to avoid creating these files at the first place?

nope it's not possible as we need to extract text from message or attachement for message.

> Or could they be created in a subfolder of /tmp?

messageviewer_XXX is not enough ?
What is the advantage to create a subfolder ?
Comment 16 Thomas Monjalon 2017-09-03 19:47:12 UTC
> > Or could they be created in a subfolder of /tmp?
> 
> messageviewer_XXX is not enough ?

No it's not enough because these files are never removed.
So my /tmp is filling and it prevents me from seeing my new files in /tmp.

> What is the advantage to create a subfolder ?

The advantage is to keep /tmp clean.
I use /tmp a lot and I need to keep it clean.