Bug 332013 - NFS with NetApp FAS: please split payload files in file_db_data into several directories to avoid reaching maxdirsize limit on Ontap / WAFL filesystem
Summary: NFS with NetApp FAS: please split payload files in file_db_data into several ...
Status: RESOLVED FIXED
Alias: None
Product: Akonadi
Classification: Frameworks and Libraries
Component: server (show other bugs)
Version: 1.11.0
Platform: Debian unstable Linux
: NOR wishlist
Target Milestone: ---
Assignee: kdepim bugs
URL:
Keywords:
Depends on:
Blocks: 332624
  Show dependency treegraph
 
Reported: 2014-03-11 10:17 UTC by Martin Steigerwald
Modified: 2016-09-14 09:04 UTC (History)
5 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments
mysql.err which hinted my at double MySQL database (6.67 KB, application/octet-stream)
2014-03-26 09:10 UTC, Martin Steigerwald
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Martin Steigerwald 2014-03-11 10:17:03 UTC
On moving a mail from one folder to a newly created folder in an IMAP account for Exchange I get an error message instead of a moved mail.

Reproducible: Always

Steps to Reproduce:
1. Have an Exchange server.
2. Create a folder called "_Eingangskorb" (I tried with another folder without underscore, and it happened as well)
3. Move one mail from inbox to that folder.
Actual Results:  
~/.xsession-errors shows the following – sometimes I also give a dialog window with the ItemRetriever-Exception error message:

akonadi_imap_resource_0(5641)/kdepimlibs (kimap) KIMAP::StoreJob::handleResponse: We asked for UID but the server didn't give it back, resultingFlags not stored. 
akonadi_imap_resource_0(5641)/kdepimlibs (kimap) KIMAP::StoreJob::handleResponse: We asked for UID but the server didn't give it back, resultingFlags not stored. 
ItemRetrieverException :  Unable to retrieve item from resource: NO PartHelperException: Could not open '/home/ms/.local/share/akonadi/file_db_data/2811475_r0' for writing, error was 'Die Datei ist zu groÃ'
ItemRetrieverException :  Unable to retrieve item from resource: NO PartHelperException: Could not open '/home/ms/.local/share/akonadi/file_db_data/2811475_r0' for writing, error was 'Die Datei ist zu groÃ'
ItemRetrieverException :  Unable to retrieve item from resource: NO PartHelperException: Could not open '/home/ms/.local/share/akonadi/file_db_data/2811475_r0' for writing, error was 'Die Datei ist zu groÃ'
X Error: BadWindow (invalid Window parameter) 3
  Major opcode: 20 (X_GetProperty)
  Resource id:  0x620025b
ItemRetrieverException :  Unable to retrieve item from resource: NO PartHelperException: Could not open '/home/ms/.local/share/akonadi/file_db_data/2718925_r0' for writing, error was 'Die Datei ist zu groÃ'
ItemRetrieverException :  Unable to retrieve item from resource: NO PartHelperException: Could not open '/home/ms/.local/share/akonadi/file_db_data/2718925_r0' for writing, error was 'Die Datei ist zu groÃ'
ItemRetrieverException :  Unable to retrieve item from resource: NO PartHelperException: Could not open '/home/ms/.local/share/akonadi/file_db_data/2718925_r0' for writing, error was 'Die Datei ist zu groÃ'
X Error: BadWindow (invalid Window parameter) 3
  Major opcode: 20 (X_GetProperty)
  Resource id:  0x6200291
plasma-desktop(10397)/plasma StatusNotifierItemSource::refreshCallback: DBusMenu disabled for this application 

(process:24691): GLib-CRITICAL **: g_slice_set_config: assertion 'sys_page_size == 0' failed
ATTENTION: default value of option force_s3tc_enable overridden by environment.
OpenGL version detected: 210
OpenGL version detected: 210
OpenGL version detected: 210
OpenGL version detected: 210
OpenGL version detected: 210
plasma-desktop(10397)/plasma StatusNotifierItemSource::refreshCallback: DBusMenu disabled for this application 
akonadi_imap_resource_0(5641)/kdepimlibs (kimap) KIMAP::StoreJob::handleResponse: We asked for UID but the server didn't give it back, resultingFlags not stored. 
ItemRetrieverException :  Unable to retrieve item from resource: NO PartHelperException: Could not open '/home/ms/.local/share/akonadi/file_db_data/2718925_r0' for writing, error was 'Die Datei ist zu groÃ'
ItemRetrieverException :  Unable to retrieve item from resource: NO PartHelperException: Could not open '/home/ms/.local/share/akonadi/file_db_data/2718925_r0' for writing, error was 'Die Datei ist zu groÃ'
plasma-desktop(10397)/plasma StatusNotifierItemSource::refreshCallback: DBusMenu disabled for this application 
akonadi_imap_resource_0(5641)/kdepimlibs (kimap) KIMAP::StoreJob::handleResponse: We asked for UID but the server didn't give it back, resultingFlags not stored. 
Insert: payload file  "/home/ms/.local/share/akonadi/file_db_data//2918291_r0"  could not be open for writing!                                                
Error:  "Die Datei ist zu groß"                                                                                                                               
ItemRetrieverException :  Unable to retrieve item from resource: NO Unable to add item part                                                                                             
Insert: payload file  "/home/ms/.local/share/akonadi/file_db_data//2918292_r0"  could not be open for writing!
Error:  "Die Datei ist zu groß"
ItemRetrieverException :  Unable to retrieve item from resource: NO Unable to add item part

Expected Results:  
The mail is moved.

These files do not exist:
- /home/ms/.local/share/akonadi/file_db_data/2718925_r0
- /home/ms/.local/share/akonadi/file_db_data//2918294_r0 

ms@mango:~> ls -l /home/ms/.local/share/akonadi/file_db_data/2811475_r0
-rw-rw-r-- 1 ms teamix 4184 Mär 11 11:07 /home/ms/.local/share/akonadi/file_db_data/2811475_r0

/home is located in NetApp FAS NFS.

ms@mango:~#2> apt-show-versions | egrep "(kmail|akonadi)"
akonadi-backend-mysql:all/sid 1.11.0-1 uptodate
akonadi-backend-postgresql:all/sid 1.11.0-1 uptodate
akonadi-dbg:amd64/sid 1.11.0-1 uptodate
akonadi-server:amd64/sid 1.11.0-1 uptodate
akonadiconsole:amd64/experimental *manually* upgradeable from 4:4.12.2-1+b1 to 4:4.12.3-1
kmail:amd64/experimental *manually* upgradeable from 4:4.12.2-1+b1 to 4:4.12.3-1
libakonadi-calendar4:amd64/experimental *manually* upgradeable from 4:4.12.2-1 to 4:4.12.3-1
libakonadi-contact4:amd64/experimental *manually* upgradeable from 4:4.12.2-1 to 4:4.12.3-1
libakonadi-kabc4:amd64/experimental *manually* upgradeable from 4:4.12.2-1 to 4:4.12.3-1
libakonadi-kcal4:amd64/experimental *manually* upgradeable from 4:4.12.2-1 to 4:4.12.3-1
libakonadi-kde4:amd64/experimental *manually* upgradeable from 4:4.12.2-1 to 4:4.12.3-1
libakonadi-kmime4:amd64/experimental *manually* upgradeable from 4:4.12.2-1 to 4:4.12.3-1
libakonadi-notes4:amd64/experimental *manually* upgradeable from 4:4.12.2-1 to 4:4.12.3-1
libakonadiprotocolinternals1:amd64/sid 1.11.0-1 uptodate

I will upgrade those and try again. If it works then I will close the bug.
Comment 1 Martin Steigerwald 2014-03-11 11:22:17 UTC
I did these upgrades now and the issue still happens. I found out something more:

It doesn't happen on all moves. I just have seen it when moving from Inbox to another folder. It doesn't seem to matter if newly created or not. But it also only seems to happen on some mails.

I can't test at the moment, cause Akonadi is again occupied with folder synchronization and blocks out KMail completely now, even for minutes (see bug #331848).
Comment 2 Christian Mollekopf 2014-03-25 09:44:12 UTC
FYI: "Die Datei ist zu gross" means "The file is too large"
Comment 3 Martin Steigerwald 2014-03-25 14:12:23 UTC
Thanks for translation, I thought I added it, but I forgot.

I now have this again, but already with trying to display a mail. Even directly after restarting Akonadi. I get things like this in ~/.xsession-errors:

ms@mango:~> Insert: payload file  "/home/ms/.local/share/akonadi/file_db_data//3026619_r0"  could not be open for writing!
Error:  "Die Datei ist zu groß"
ItemRetrieverException :  Unable to retrieve item from resource: NO Unable to add item part
Insert: payload file  "/home/ms/.local/share/akonadi/file_db_data//3026620_r0"  could not be open for writing!
Error:  "Die Datei ist zu groß"
ItemRetrieverException :  Unable to retrieve item from resource: NO Unable to add item part
Insert: payload file  "/home/ms/.local/share/akonadi/file_db_data//3026621_r0"  could not be open for writing!
Error:  "Die Datei ist zu groß"
ItemRetrieverException :  Unable to retrieve item from resource: NO Unable to add item part
Insert: payload file  "/home/ms/.local/share/akonadi/file_db_data//3026622_r0"  could not be open for writing!
Error:  "Die Datei ist zu groß"
ItemRetrieverException :  Unable to retrieve item from resource: NO Unable to add item part
Insert: payload file  "/home/ms/.local/share/akonadi/file_db_data//3026623_r0"  could not be open for writing!
Error:  "Die Datei ist zu groß"
ItemRetrieverException :  Unable to retrieve item from resource: NO Unable to add item part
Insert: payload file  "/home/ms/.local/share/akonadi/file_db_data//3026624_r0"  could not be open for writing!
Error:  "Die Datei ist zu groß"
ItemRetrieverException :  Unable to retrieve item from resource: NO Unable to add item part
ItemRetrieverException :  Unable to retrieve item from resource: NO PartHelperException: Could not open '/home/ms/.local/share/akonadi/file_db_data/3026607_r0' for writing, error was 'Die Datei ist zu groÃ'
akonadi_imap_resource_0(23478)/libakonadi Akonadi::SessionPrivate::dataReceived: Unable to login to Akonadi server: "BAD Unrecognized command:  cellspacing=3D
" 
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
ItemRetrieverException :  Unable to retrieve item from resource: <html>Ungültigen Eintrag erhalten</html>
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
QAbstractSocket::waitForBytesWritten() is not allowed in UnconnectedState
Insert: payload file  "/home/ms/.local/share/akonadi/file_db_data//3026625_r0"  could not be open for writing!
Error:  "Die Datei ist zu groß"
ItemRetrieverException :  Unable to retrieve item from resource: NO Unable to add item part
Insert: payload file  "/home/ms/.local/share/akonadi/file_db_data//3026626_r0"  could not be open for writing!
Error:  "Die Datei ist zu groß"
ItemRetrieverException :  Unable to retrieve item from resource: NO Unable to add item part
Insert: payload file  "/home/ms/.local/share/akonadi/file_db_data//3026627_r0"  could not be open for writing!
Error:  "Die Datei ist zu groß"
ItemRetrieverException :  Unable to retrieve item from resource: NO Unable to add item part
Insert: payload file  "/home/ms/.local/share/akonadi/file_db_data//3026628_r0"  could not be open for writing!
Error:  "Die Datei ist zu groß"
ItemRetrieverException :  Unable to retrieve item from resource: NO Unable to add item part
Insert: payload file  "/home/ms/.local/share/akonadi/file_db_data//3026629_r0"  could not be open for writing!
Error:  "Die Datei ist zu groß"
ItemRetrieverException :  Unable to retrieve item from resource: NO Unable to add item part
Insert: payload file  "/home/ms/.local/share/akonadi/file_db_data//3026630_r0"  could not be open for writing!
Error:  "Die Datei ist zu groß"
ItemRetrieverException :  Unable to retrieve item from resource: NO Unable to add item part
ItemRetrieverException :  Unable to retrieve item from resource: NO PartHelperException: Could not open '/home/ms/.local/share/akonadi/file_db_data/2919253_r2' for writing, error was 'Die Datei ist zu groÃ'
 list is empty 

After this the resource seems basically unresponsive, not doing anything at all anymore for minutes. But then from time to time it spilles out another ItemRetrieverException.

I am a bit confused about the message that its trying to write the message, but I bet in order to display a message Akonadi tries to download it from IMAP and store it in file_db_data temporarily.

File is to big is strange as error message cause there seems to be enough space left:

ms@mango:~/.local/share/akonadi/file_db_data> LANG=C df -hT .
Filesystem                                   Type  Size  Used Avail Use% Mounted on
somemount                                 nfs   1.2T  888G  313G  74% /home

But there seems to be some issue. file_db_date seems to contain a *ton* of files. A find . | wc -l takes long to complete. Hmmm, very long actually. I wonder what amount of files I will get.

Well, but I can definately write files there, not fast during the find still running, but still:

ms@mango:~/.local/share/akonadi/file_db_data> LANG=C dd if=/dev/urandom of=bigmail bs=1M count=100 conv=fsync
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 8.1205 s, 12.9 MB/s


Now the find completes and the value doesn't seem to be very sane to me:

ms@mango:~/.local/share/akonadi/file_db_data#130> find . | wc -l
277878

277000 files in file_db_data!? I know I have a kernel-ml folder with 400000 mails, but KMail is not even subscribed to it (unchecked folder in serverside subscriptions). There are for sure several hundred mails spread in different folder, but still, I am not accessing these, so even on building Akonadi database and synchronizing folders I'd be surprised on this amount of data stored there. Its

ms@mango:~/.local/share/akonadi> du -sh file_db_data
2,2G    file_db_data

The account has about 29 GB according to Outlook Web Access. I'd estimated about 10 GB back from Zimbra times, but well. Maybe Akonadi just doesn't scale to this amount of data?

Well as it happens now without doing anything to provoke it, it might be easy to debug. So if you have some hints / pointers on how to approach that, I am keen to read them. The above logs do not seem to relate to IMAP, thus I bet IMAP debugging won't find the cause.
Comment 4 Martin Steigerwald 2014-03-25 14:14:03 UTC
Forget my comment regarding file write speed. Its likely limited by speed of /dev/urandom, which as to my experience is about 15 MB on this box. Writing zeros may not be relastic either. Anyway, I can write a 100 MB file there and this is what counts.
Comment 5 Martin Steigerwald 2014-03-25 14:15:39 UTC
On stopping Akonadi I got:

ms@mango:~/.local/share/akonadi> akonadictl stop
ms@mango:~/.local/share/akonadi> akonadi_kabc_resource_0(23480): Saving failed:  "Keine Verbindung zu Rechner ldap://localhost:1389ou=people??sub
Weitere Informationen: ." 
Application '/usr/bin/akonadi_kabc_resource' exited normally...
Application '/usr/bin/akonadi_sendlater_agent' exited normally...
Application '/usr/bin/akonadi_agent_launcher' exited normally...
Application '/usr/bin/akonadi_agent_launcher' exited normally...
Application '/usr/bin/akonadi_maildispatcher_agent' exited normally...
Application '/usr/bin/akonadi_agent_launcher' exited normally...
ProcessControl: Application /usr/bin/akonadi_imap_resource stopped unexpectedly ( "Process crashed" )
Application '/usr/bin/akonadi_imap_resource' crashed. No restart!

So the IMAP resource basically appeared to have crashed on exit now.

ItemRetrieverException :  Unable to retrieve item from resource: Message did not receive a reply (timeout by message bus)
Application '/usr/bin/akonadi_agent_launcher' exited normally...
Application '/usr/bin/akonadi_imap_resource' exited normally...
Application '/usr/bin/akonadi_mailfilter_agent' exited normally...
Application '/usr/bin/akonadi_archivemail_agent' exited normally...
Application '/usr/bin/akonadi_migration_agent' exited normally...
Application '/usr/bin/akonadi_newmailnotifier_agent' exited normally...
Application '/usr/bin/akonadi_agent_launcher' exited normally...
Application '/usr/bin/akonadi_folderarchive_agent' exited normally...
Application '/usr/bin/akonadi_nepomuk_feeder' exited normally...
"Cannot connect to agent instance with identifier 'akonadi_imap_resource_0', error message: 'Could not get owner of name 'org.freedesktop.Akonadi.Resource.akonadi_imap_resource_0': no such name'"
ItemRetrieverException :  Unable to contact resource
QSocketNotifier: Invalid socket 162 and type 'Read', disabling...
ProcessControl: Application 'akonadiserver' returned with exit code 255 (Process operation timed out)
Comment 6 Martin Steigerwald 2014-03-25 14:36:17 UTC
Well, I straced akonadiserver during that happening and really got a file to big back on trying to write the file. Excerpt:

28443 <... poll resumed> )              = 1 ([{fd=96, revents=POLLIN}])
28443 read(97, 0x7f12ad7f9b60, 16)      = -1 EAGAIN (Resource temporarily unavailable)
28443 ioctl(96, FIONREAD, [10])         = 0
28443 read(96, "491 BEGIN\n", 10)       = 10
28443 poll([{fd=137, events=POLLIN|POLLPRI}], 1, 0) = 0 (Timeout)
28443 write(137, "\v\0\0\0\3BEGIN WORK", 15) = 15
28443 read(137, "\7\0\0\1\0\0\0\3\0\0\0", 16384) = 11
28443 write(97, "\1\0\0\0\0\0\0\0", 8)  = 8
28443 select(97, [96], [96], NULL, {30, 0}) = 1 (out [96], left {29, 999998})
28443 write(96, "491 OK Begin completed\r\n", 24) = 24
28443 write(97, "\1\0\0\0\0\0\0\0", 8)  = 8
28443 poll([{fd=97, events=POLLIN}, {fd=96, events=POLLIN}], 2, 21930) = 1 ([{fd=97, revents=POLLIN}])
28443 poll([{fd=97, events=POLLIN}, {fd=96, events=POLLIN}], 2, 21930) = 1 ([{fd=97, revents=POLLIN}])
28443 read(97, "\2\0\0\0\0\0\0\0", 16)  = 8
28443 poll([{fd=97, events=POLLIN}, {fd=96, events=POLLIN}], 2, 21930) = 1 ([{fd=96, revents=POLLIN}])
28443 read(97, 0x7f12ad7f9b60, 16)      = -1 EAGAIN (Resource temporarily unavailable)
28443 ioctl(96, FIONREAD, [123])        = 0
28443 read(96, "492 X-AKAPPEND 591 26707 (\\MimeT"..., 123) = 123
28443 write(97, "\1\0\0\0\0\0\0\0", 8)  = 8
28443 select(97, [96], [96], NULL, {30, 0}) = 1 (out [96], left {29, 999998})
28443 write(96, "+ Ready for literal data (expect"..., 49) = 49
28443 write(97, "\1\0\0\0\0\0\0\0", 8)  = 8
28443 select(97, [96], [], NULL, {30, 0}) = 1 (in [96], left {29, 999998})
28443 write(97, "\1\0\0\0\0\0\0\0", 8)  = 8
28443 ioctl(96, FIONREAD, [5153])       = 0
28443 read(96, "MIME-Version: 1.0\r\nReceived: fro"..., 5153) = 5153
28443 write(97, "\1\0\0\0\0\0\0\0", 8)  = 8
28443 stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=2309, ...}) = 0
28443 poll([{fd=137, events=POLLIN|POLLPRI}], 1, 0) = 0 (Timeout)
28443 write(137, "\5\0\0\0\32!\0\0\0", 9) = 9
28443 read(137, "\7\0\0\1\0\0\0\3\0\0\0", 16384) = 11
28443 poll([{fd=137, events=POLLIN|POLLPRI}], 1, 0) = 0 (Timeout)
28443 poll([{fd=137, events=POLLIN|POLLPRI}], 1, 0) = 0 (Timeout)
28443 write(137, "N\0\0\0\27!\0\0\0\0\1\0\0\0\f\0\1\3\0\376\0\376\0\376\0\10\0\10\0\f\0\1"..., 82) = 82
28443 read(137, "\n\0\0\1\0\1\375\247$\27\3\0\0\0", 16384) = 14
28443 poll([{fd=137, events=POLLIN|POLLPRI}], 1, 0) = 0 (Timeout)
28443 write(137, "\5\0\0\0\32\"\0\0\0", 9) = 9
28443 read(137, "\7\0\0\1\0\0\0\3\0\0\0", 16384) = 11
28443 poll([{fd=137, events=POLLIN|POLLPRI}], 1, 0) = 0 (Timeout)
28443 poll([{fd=137, events=POLLIN|POLLPRI}], 1, 0) = 0 (Timeout)
28443 write(137, "6\0\0\0\27\"\0\0\0\0\1\0\0\0\4\1\10\0\376\0\374\0\10\0\3\0\1\0\247$\27\0"..., 58) = 58
28443 read(137, "\n\0\0\1\0\1\375A/.\3\0\0\0", 16384) = 14
28443 stat("/home/ms/.local/share/akonadi/file_db_data", {st_mode=S_IFDIR|0775, st_size=21475328, ...}) = 0
28443 open("/home/ms/.local/share/akonadi/file_db_data//3026753_r0", O_WRONLY|O_CREAT|O_TRUNC|O_CLOEXEC, 0666) = -1 EFBIG (File too large)

I am not sure whether its possible to find out the exact size, Akonadi requested here. In any, cause I do not see any fallocate flags in there, which probably wouldn't work over NFS anyway.

28443 open("/home/ms/.local/share/akonadi/akonadiserver.error", O_WRONLY|O_CREAT|O_APPEND|O_CLOEXEC, 0666) = 8

There it writes 

Insert: payload file  "/home/ms/.local/share/akonadi/file_db_data//3026753_r0"  could not be open for writing! 
Error:  "Die Datei ist zu groß" 

into the log.

28443 fcntl(8, F_SETFD, FD_CLOEXEC)     = 0
28443 fstat(8, {st_mode=S_IFREG|0664, st_size=15365, ...}) = 0
28443 fstat(8, {st_mode=S_IFREG|0664, st_size=15365, ...}) = 0
28443 lseek(8, 15365, SEEK_SET)         = 15365
28443 lseek(8, 15365, SEEK_SET)         = 15365


I am wondering what is happening here. Does an IMAP debug protocol makes sense here? Maybe Exchange IMAP server reports wrong size of mail or something?

I stored the complete strace log, but I don't like to paste it here before checking whether it has any confidential data in it.

Ah, I mentioned "file too big" in subject of bug. And well on exit, that might not be a crash, but just a slow response of the IMAP resource?
Comment 7 Christian Mollekopf 2014-03-25 16:52:15 UTC
I think with your home on NFS you're unfortunately in an unsupported situation (at least as far as I'm concerned). I don't have any experience myself with it, but I already heard a couple of times that running this with home on nfs doesn't really work.
If this is indeed an NFS related issue, the best you can do is open a wishlist item for NFS support, but I doubt anyone will invest a lot of time in that, or fix it yourself.

Sorry about that, but we can only cater so many usecases. I'll leave it up to Daniel to close this as WONTFIX.

Let us know though if you think the issue is unrelated to NFS.
Comment 8 Christian Mollekopf 2014-03-25 17:01:02 UTC
Apparently there was some success in the past with mariadb over nfs at least: https://bugs.kde.org/show_bug.cgi?id=275261 (see the last comment)
Comment 9 Martin Steigerwald 2014-03-26 09:06:40 UTC
I kindly ask you to reconsider. Running home directories on NFS together with some LDAP as user directory in my oppinion is a rather common use case in any organizations from companies, to universities or schools. It is used to make user data available at different work places / work stations.

I read that Kontact is to be used in Munich. Heck, I don't know their setup, but I would be highly surprised if actual user data is stored on the workstations themselves. It just wouldn't make much sense to me to do that.

As to whether the issue is related to NFS: Frankly, I don't know. I never saw this with another application tough. But there is a reason I believe that at least the issue is not related to the MySQL database itself: Storing journal entries via Korganizer works just fine. This also needs MySQL access for storing metadata about journal entries. Also Kmail actually lists the folders and the contains of the folders and to what I know it could do this without a working MySQL database.

Hmmm, on the other hand on looking at MySQL more close, it indeed seems to have some issues. Even after akonadictl stop and akonadictl status telling that MySQL is not running, I have a mysqld process:

ms        4332  0.6  0.9 2304944 115580 ?      Sl   Mär24  17:03 /usr/sbin/mysqld --defaults-file=/home/ms/.local/share/akonadi/mysql.conf --datadir=/home/ms/.local/share/akonadi/db_data/ --socket=/tmp/akonadi-ms.LdcAOs/mysql.socket

I will check what this process is doing. Well it seems fine:

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| akonadi            |
| mysql              |
| performance_schema |
| test               |
+--------------------+
5 rows in set (0.02 sec)

I also checked all tables in akonadi database all with OK results, like this one:

mysql> check table parttable;
+-------------------+-------+----------+----------+
| Table             | Op    | Msg_type | Msg_text |
+-------------------+-------+----------+----------+
| akonadi.parttable | check | status   | OK       |
+-------------------+-------+----------+----------+
1 row in set (30.33 sec)

Okay, I will now stop this process by hand with a friendly TERM signal. It excited gracefully. Now lets restart Akonadi: Well mysql is fine:

akonadi.collectionattributetable                   OK
akonadi.collectionmimetyperelation                 OK
akonadi.collectionpimitemrelation                  OK
akonadi.collectiontable                            OK
akonadi.flagtable                                  OK
akonadi.mimetypetable                              OK
akonadi.parttable                                  OK
akonadi.pimitemflagrelation                        OK
akonadi.pimitemtable                               OK
akonadi.resourcetable                              OK
akonadi.schemaversiontable                         OK
mysql.columns_priv                                 OK
mysql.db                                           OK
mysql.event                                        OK
mysql.func                                         OK
mysql.general_log                                  OK
mysql.help_category                                OK
mysql.help_keyword                                 OK
mysql.help_relation                                OK
mysql.help_topic                                   OK
mysql.host                                         OK
mysql.ndb_binlog_index                             OK
mysql.plugin                                       OK
mysql.proc                                         OK
mysql.procs_priv                                   OK
mysql.proxies_priv                                 OK
mysql.servers                                      OK
mysql.slow_log                                     OK
mysql.tables_priv                                  OK
mysql.time_zone                                    OK
mysql.time_zone_leap_second                        OK
mysql.time_zone_name                               OK
mysql.time_zone_transition                         OK
mysql.time_zone_transition_type                    OK
mysql.user                                         OK

So seems that Akonadi for whatever reason didn't stop a left over MySQL process. Okay, I get the actual issue with "file to big again". Still that still running MySQL issue may be unrelated.


Okay, well, to confirm whether this is NFS related or not, I will do the following trick:

I do have local storage on my Workstation. I will stop Akonadi, make sure MySQL isn't running rsync -aAHXS and then symlink the ~/.local/share/akonadi onto a local Ext4 filesystem.


I will open a wish list bug about making sure that Akonadi by default works well on NFS. I bet that functionality is needed for people wanting to use KDEPIM in an organization without the overhead to configure it to use a central database and keep it in sync with any locally stored metadata. Heck, since the error is with writing files to file_db_data, a central database may not even help with that. I will link this and any other possibly NFS related bugs to this wish list bug.

Thanks,
Martin
Comment 10 Martin Steigerwald 2014-03-26 09:10:25 UTC
Created attachment 85756 [details]
mysql.err which hinted my at double MySQL database

It shows that locking via NFS worked correctly.
Comment 11 Daniel Vrátil 2014-03-26 10:16:58 UTC
The "The file is too large" error is not from the database, but is returned by the system when we try to save content of the item on disk.

Because a new file is being created (O_WRONLY|O_CREAT|O_TRUNC), I doubt that the problem is the actual size of the file. Instead the problem could be in the size of the directory, or rather in the amount of files in the directory. Some filesystems have a limit on maximum amount of files in a directory, in your case this can be either a limitation of NFS or the underlaying file system. 

I'd be interested in seeing output from ls -1 ~/.local/share/akonadi/file_db_data | wc -l

I still agree with what Christian said: NFS is currently not a supported use case at the moment (which can change in the future, however). People were successful running Akonadi on NFS (using either SQLite or external DB server), but the access to external payloads will still be slow.
Comment 12 Martin Steigerwald 2014-03-26 10:33:54 UTC
Thanks Daniel and Christian. I now have

ms@mango:~/.local/share/akonadi> LANG=C df -hT .
Filesystem                    Type  Size  Used Avail Use% Mounted on
/dev/mapper/mangoraid-akonadi ext4   50G  4.0G   43G   9% /mnt/akonadi

located in a Ext4 SoftRAID 1 on two 1,5 TB WD harddisks.

And it indeed makes a difference. I did not yet see the errors I pasted in this bug report so far and I can access and view mails just fine.

I confirm that it does not seem a database issue. As for the value you requested I had it already: 

ms@mango:~/.local/share/akonadi/file_db_data> find . | wc -l 277878

And I also confirmed that I can still write a 100 MiB file to the directory.

Current value with local-share-akonadi on local Ext4 is:

ms@mango:~/.local/share/akonadi#130> ls -1 file_db_data | wc -l
285336

I will open a wish list bug about NFS as it IMHO is an important use case in organizations. Icedove works on NFS, Evolution works on NFS.

I had Akonadi running against a central PostgreSQL database for a file, but I think, for ease of setup, I keep Akonadi running as:

ms@mango:~/.local/share> LANG=C ls -l akonadi
lrwxrwxrwx 1 ms teamix 23 Mar 26 11:13 akonadi -> /mnt/akonadi/ms/akonadi

I added this directory to our Crashplan Pro Backup Engine (knowing that the database backup won't be consistent without stopping Akonadi first). For IMAP I bet its able to reproduce most metadata anyway in case its lost.
Comment 13 Martin Steigerwald 2014-03-26 11:04:17 UTC
Meta bug: Bug 332624 - NFS support
Comment 14 Martin Steigerwald 2014-03-26 13:50:46 UTC
Daniel, I will ask around here a bit as to a file per directory limit WAFL filesystem on Ontap operating system on NetApp FAS machines, but I bet this can be the case:

http://serverfault.com/questions/76018/maximum-number-of-files-in-a-single-directory-for-netapp-nfs-mounts-on-linux

https://communities.netapp.com/thread/2564

Well if thats the case, I bet we can close this bug, still:

How about distributing those files into several sub directories? I think this may improve performance on Linux filesystems as well, as I think directory entry lookup time increase with size of directory?
Comment 15 Martin Steigerwald 2014-03-26 15:00:50 UTC
Daniel, a NetApp expert co-worker confirmed that your assumption that the error code refers to the directory is correct:

Wed Mar 26 09:12:13 CET [somevfiler@somefiler:wafl.dir.size.max:warning]: Directory /vol/nas_user_home/ms/.local/share/akonadi/file_db_data/ reached the maxdirsize limit. 

My co-worker raised the limit, yet still, I think its wise to split 280000 files into several sub directories. There may be other storage system or filesystems that may have a limit there. And even if not: I bet the lookup time of a directory increases with amount of files in it.

I renamed the bug report and set it to wishlist as its technical not really a bug.
Comment 16 Daniel Vrátil 2014-03-29 10:24:52 UTC
We access the files directly by it's name (we never list content of the directory), which makes use of internal hash/tree of the filesystem implementation, which is usually very fast and should cope with large folders just fine.
Comment 17 Martin Steigerwald 2014-04-02 09:49:30 UTC
Are you sure this is the case with NFS and other networking filesystems as well? Anyway, its a wish list item now.
Comment 18 Daniel Vrátil 2014-04-02 10:18:28 UTC
Sorry, my resposne was to "I bet the lookup time of a directory increases with amount of files in it."

We can re-evaluate this for networking filesystems of course.
Comment 19 Daniel Vrátil 2016-02-18 11:01:47 UTC
Multi-level file_dbdata has been implemented in 15.12, you can migrate your database by manually running "akonadictl fsck".