Bug 327812

Summary: Strange error message on scanning large collections
Product: [Applications] amarok Reporter: Evert Vorster <evorster>
Component: Collections/LocalAssignee: Amarok Developers <amarok-bugs-dist>
Status: RESOLVED FIXED    
Severity: normal CC: matej, ralf-engels
Priority: NOR    
Version: 2.8.0   
Target Milestone: 2.9   
Platform: Compiled Sources   
OS: Linux   
Latest Commit: http://commits.kde.org/amarok/6405e5cb7e2fa8a5c597a27034f75b873f00fe8e Version Fixed In: 2.9
Sentry Crash Report:
Attachments: 0001-collectionscanner-don-t-store-dirs-in-shared-memory-.patch

Description Evert Vorster 2013-11-19 13:04:38 UTC 
While running with amarok --debug in a terminal, when I scan a large music collection ( 400Gb, 80,000 files ) 

I get this error message for each file scanned:

void CollectionScanner::ScanningState::SetLastFile(const & QString&) QSharedMemory is too small to hold the data.
It is of size 1048576 bytes but we need more than 3839770 bytes




Reproducible: Always

Steps to Reproduce:
1. Delete collection database
2. re-scan collection
3.
Actual Results:  
Loads of error messages flooding the console

Expected Results:  
Just error messages one can use in the console

The system is an i7 in 64bit mode with 8Gb of RAM available. There is a maximum of 2Gb of RAM utilized at any point, so no memory leak. 

amarokcollectionscanner shows 168m virt, 40m res, 8868 shr in top while scanning, and this value stays constant. 

I looked for related bugs, but every bug report so far says the collectionscanner or amarok crashed. In my case, they don't crash, so there is no way to get a backtrace with gdb.
Comment 1 Myriam Schweingruber 2013-11-19 19:55:04 UTC 
You only tell what you go in the konsole, but what was in Amarok itself? Couldn't you use it, was it not scanning the collection?

Are you sure these were "ERROR"-messages and not rather "Warning"-messages?

A copy of around 30 lines of that output would have been helpful, ideally starting where the scanning starts
Comment 2 Evert Vorster 2013-11-19 20:12:06 UTC 
Thanks for the quick reply. 

Amarok was scanning the collection in this case, true. 
The reason it was running in debug mode in the console was that the collectionscanner would sometimes quit scanning before reaching the end of the collection. It would also fail randomly while importing music, and I am still trying to figure out why. 

The warning/error messages are obscuring all other messages being output to the console, making debugging / troubleshooting the actual cause of amarok's frailty with large music collections next to impossible. 

I have more than 80,000 files, each one of which produces two lines of message when read, ie: 160,000 lines of messages.
My console does not support 160,000 lines of scrollback, which is kind of the point of this bug report. 
So, to re-iterate, these messages are spamming out useful debug information. 

Surely the size of QSharedMemory can be allocated according to the size of the collection that is being scanned, and so avoid this spamming of the console?

Also, what is being stored in that variable, and what is the effect of it being truncated?
Comment 3 Evert Vorster 2013-11-19 20:27:55 UTC 
Here is some lines from the console, as requested. 

Firstly the collectionscanner finds a bunch of missing directories, so many that the start of scanning has scrolled off the top of the console's history, but that is another matter. 

-------------snip--------------Directory "/data/Sounds/Collection/mp3/Tlpopmusik/" does not exist.
Directory "/data/Sounds/Collection/mp3/Tnia Maria/" does not exist.
Directory "/data/Sounds/Collection/mp3/Toquinho e Vincius/" does not exist.
Directory "/data/Sounds/Collection/mp3/Toquinho e Vincius-Toquinho e Vincius/" does not exist.
Directory "/data/Sounds/Collection/mp3/Tr/" does not exist.
Directory "/data/Sounds/Collection/mp3/Trby Trio/" does not exist.
Directory "/data/Sounds/Collection/mp3/Trio Esperana/" does not exist.
Directory "/data/Sounds/Collection/mp3/Ultra Nat/" does not exist.
Directory "/data/Sounds/Collection/mp3/Vacher, mile/" does not exist.
Directory "/data/Sounds/Collection/mp3/Vittorio Mu/" does not exist.
Directory "/data/Sounds/Collection/mp3/Vrios/" does not exist.
Directory "/data/Sounds/Collection/mp3/Vronique Sanson/" does not exist.
Directory "/data/Sounds/Collection/mp3/zero nol/" does not exist.
Directory "/data/Sounds/Collection/mp3/~~_Prh Bst rds_~~, Shreya Ghoshal , Vishal Bhardwaj/" does not exist.
void CollectionScanner::ScanningState::writeFull() QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839236 bytes. 
amarok: [AbstractScanResultProcessor] got 32599 directories 
void CollectionScanner::ScanningState::writeFull() QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839292 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839374 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839394 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839380 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839390 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839380 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839402 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839384 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839386 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839442 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839400 bytes. 
TagLib: MPEG::Header::parse() -- Invalid sample rate.
TagLib: MPEG::Header::parse() -- Invalid sample rate.
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839402 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839372 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839472 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839452 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839418 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839408 bytes. 
TagLib: MPEG::Header::parse() -- Invalid sample rate.
TagLib: MPEG::Header::parse() -- Invalid sample rate.
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839374 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839374 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839374 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839374 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839374 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839374 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839374 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839374 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839374 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839374 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839374 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839374 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839374 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839374 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839374 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839374 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839374 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839374 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839358 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839376 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839410 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839402 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839384 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839410 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839400 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839398 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839412 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839384 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839376 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839406 bytes. 
TagLib: MPEG::Header::parse() -- Invalid sample rate.
TagLib: MPEG::Header::parse() -- Invalid sample rate.
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839424 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839372 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839390 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839418 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839416 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839390 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839386 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839398 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839426 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839414 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839424 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839390 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839446 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839402 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839382 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839378 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839360 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839394 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839422 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839412 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839420 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839386 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839420 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839382 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839390 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839412 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839380 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839382 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839384 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839374 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839392 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839408 bytes. 
TagLib: MPEG::Header::parse() -- Invalid sample rate.
TagLib: MPEG::Header::parse() -- Invalid sample rate.
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839372 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839398 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839376 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839408 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839406 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839434 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839388 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839410 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839376 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839392 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839442 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839378 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839360 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839406 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839416 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839384 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839412 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839464 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839404 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839384 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839378 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839398 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839410 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839384 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839386 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839394 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839382 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839422 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839388 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839396 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839394 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839400 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839410 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839378 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839410 bytes. 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839412 bytes. 
amarok: BEGIN: void App::quit() 
void CollectionScanner::ScanningState::setLastFile(const QString&) QSharedMemory is too small to hold the data. 
It is of size 1048576 bytes but we need more than 3839406 bytes. 
amarok: END__: void App::quit() [Took: 0.039s]

-------------------snip--------------------------
I had to quit amarok to stop the scrolling and be able to capture something for you to look at.
kind regards,
evert
Comment 4 Evert Vorster 2013-11-19 20:30:52 UTC 
Just one final comment...

The error message shows that the colectionscanner already knows how big the QSharedMemory should be...
It is just never initialized to a proper size. 

-Evert-
Comment 5 Myriam Schweingruber 2013-11-19 20:33:57 UTC 
Well, first of all: make sure the partition where it tries to find tracks is accessible and writable.

Then: you can run amarokcollectionscanner as a standalone app to find where the problem lies, check out 'amarokcollectionscanner --help' for more details, as well as have a look at this page: http://community.kde.org/Amarok/Development/BatchMode for more details.
Comment 6 Evert Vorster 2013-11-19 20:56:49 UTC 
Thanks for the quick reply. 

The directory is write-able. It was created by amarok itself, after importing all the music from other collections. I then tried to re-scan the collection, and the scanner did not want to work properly. 
Hence I first tried to start with a clean database by doing a full rescan from the settings menu, and as a last resort I went into .kde/share/apps and removed amarok subdirectory. 

By this time I was running amarok in debug mode to see what is happening in the background. 



Thank you for the info on running the collectionscanner in batch mode. 
This workaround does not stop the collectionscanner from spamming the console when amarok is run in debug mode. 

My collection is successfully scanned, so it's not a huge problem for me, but the difficulty in troubleshooting amarok in debug mode on large collections will remain until this bug is fixed. 

Kind regards,
Evert
Comment 7 Myriam Schweingruber 2013-11-20 08:41:46 UTC 
Well "spamming" is not the right word, as that is supposed to happen when you run it in debug mode, that is not a bug :)

One can easily redirect the output to a text file, which can then be examined in an editor, no need to look at it in the konsole itself, that is not the point of it.

So lets resume: what exactly is the bug you are experiencing when using Amarok? If you can't scan properly, then scanning with the amarokcollectionscanner in batch mode is advised, as it produces an XML file that ca be easily examined.
Comment 8 Matěj Laitl 2013-11-20 14:07:59 UTC 
This is a valid bug, a minor one.

The message is intentional and emitted on purpose - not allocating enough of shared memory could lead to problems later on, specifically it may prevent collection scanning from restarting in case of a crash, so thanks for reporting it.

I'll look into this later.
Comment 9 Evert Vorster 2013-11-22 17:00:26 UTC 
Hi there. 

Thanks for looking into this.

I am quite willing & capable of applying patches to source code to verify a fix, if you feel like creating and sending a patch when you have time to look at this one. 

-Evert-
Comment 10 Matěj Laitl 2013-11-24 15:06:30 UTC 
Created attachment 83732 [details]
0001-collectionscanner-don-t-store-dirs-in-shared-memory-.patch

Hi,
please apply the attached patch and retest. This changes the shared memory usage in order not to store data (list of all folders) that are dependent on collection size.
Comment 11 Evert Vorster 2013-12-06 19:09:14 UTC 
Patch applied, and recompiled. 
Hunk 1 failed to apply properly, but that was just to update a change log, maybe on the git code. 

Anyways, it compiled cleanly, and the error message is not being produced now when I initialized the scan. 

In debug mode I get a scanSucceeded() after a bunch of error messages about the questionable quality of my files, so I guess this bug is fixed. :)

Sorry for the long delay in testing, I was being side-tracked. Thank you for the quick and proper fix!

Regards,
Evert
Comment 12 Matěj Laitl 2013-12-13 14:49:58 UTC 
Git commit 6405e5cb7e2fa8a5c597a27034f75b873f00fe8e by Matěj Laitl.
Committed on 24/11/2013 at 15:03.
Pushed by laitl into branch 'master'.

collectionscanner: don't store dirs in shared memory, prevent size issues

We used to store all directories to scan in the shared memory, which
can exceed the 1M we reserve for it. Refactor the resume-on-crash
algorithm in order not to need this.

Thanks to Evert Vorster for noticing the problem and testing this patch.
FIXED-IN: 2.9

M  +1    -0    ChangeLog
M  +1    -18   shared/collectionscanner/ScanningState.cpp
M  +3    -5    shared/collectionscanner/ScanningState.h
M  +24   -25   utilities/collectionscanner/CollectionScanner.cpp

http://commits.kde.org/amarok/6405e5cb7e2fa8a5c597a27034f75b873f00fe8e