Bug 85372 - KSpread memory consumption balloons: was MS Excel document causes explosive memory consumption
Summary: KSpread memory consumption balloons: was MS Excel document causes explosive m...
Status: RESOLVED FIXED
Alias: None
Product: calligrasheets
Classification: Applications
Component: filters (show other bugs)
Version: 1.3.1
Platform: Compiled Sources Linux
: NOR normal with 20 votes (vote)
Target Milestone: ---
Assignee: Ariya Hidayat
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2004-07-17 12:41 UTC by Dik Takken
Modified: 2006-05-12 18:26 UTC (History)
0 users

See Also:
Latest Commit:
Version Fixed In:


Attachments
Killer Excel Document (114.00 KB, application/x-tbz)
2004-07-17 12:43 UTC, Dik Takken
Details
another killer xls file (316.72 KB, application/x-tgz)
2006-01-15 22:18 UTC, claudiu vlad
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Dik Takken 2004-07-17 12:41:14 UTC
Version:           1.3.1 (using KDE KDE 3.2.2)
Installed from:    Compiled From Sources
OS:                Linux

Enclosed Excel document took down two independent systems I tried to view it on. After clicking on the document, KSpread fired up, memory usage jumped to hundreds of megabytes within seconds and the system did not recover during the next few minutes (Tested on a Athlon 600, 128 MB RAM, and a Athlon 800, 256 MB RAM). 

The compression ratio suggests that the document contains huge amounts of bogus information. When you open this document in MS Office 2000, it loads in a split second, showing nothing but a few tables, nothing special.

Haven't tested it on Open Office.
Comment 1 Dik Takken 2004-07-17 12:43:48 UTC
Created attachment 6706 [details]
Killer Excel Document

WARNING: Brace for impact before clicking on this link!
Comment 2 Bram Schoenmakers 2005-04-18 11:57:00 UTC
OpenOffice Calc does the job well.
Comment 3 Inge Wallin 2005-05-10 21:59:40 UTC
KSpread 1.4beta cannot open the file.  Here is the relevant part of the debug output:

ole-lib: ERROR: KLaola::KLaola(): Invalid file size!
ole-lib: ERROR: OLEFilter::filter(): Unable to read input file correctly!
koffice (lib kofficecore): KoMainWindow::addRecentURL url=file:///usr/local/KDE/src/TESTCASE-85372.xls

Maybe a damaged file? Open Office (oocalc) 1.1.2 couldn't open it either.  What version were you using, Bram?
Comment 4 Ariya Hidayat 2005-05-21 21:45:12 UTC
(Inge: you are still with the old Excel filter).

The new Excel also had crash with this document. Backtrace follows.
I will have a look on it.

#0  0xffffe410 in ?? ()
#1  0xbffed9b4 in ?? ()
#2  0x00000000 in ?? ()
#3  0x00000000 in ?? ()
#4  0x415e6e63 in __waitpid_nocancel () from /lib/tls/libpthread.so.0
#5  0x40b03992 in KCrash::defaultCrashHandler ()
   from /opt/kde3/lib/libkdecore.so.4
#6  <signal handler called>
#7  0xffffe410 in ?? ()
#8  0xbffedcac in ?? ()
#9  0x00000006 in ?? ()
#10 0x000016ef in ?? ()
#11 0x417027a1 in raise () from /lib/tls/libc.so.6
#12 0x41703f79 in abort () from /lib/tls/libc.so.6
#13 0x41688db5 in __cxxabiv1::__terminate () from /usr/lib/libstdc++.so.5
#14 0x41688df2 in std::terminate () from /usr/lib/libstdc++.so.5
#15 0x41688f32 in __cxa_throw () from /usr/lib/libstdc++.so.5
#16 0x4163be40 in std::__throw_out_of_range () from /usr/lib/libstdc++.so.5
#17 0x423f2b51 in mergeTokens ()
   from /home/ariya/cvs/lib/kde3/libexcelimport.so
#18 0x423fb79c in Swinder::ExcelReader::decodeFormula ()
   from /home/ariya/cvs/lib/kde3/libexcelimport.so
#19 0x423fbd86 in Swinder::ExcelReader::handleFormula ()
   from /home/ariya/cvs/lib/kde3/libexcelimport.so
#20 0x423fc17c in Swinder::ExcelReader::handleRecord ()
   from /home/ariya/cvs/lib/kde3/libexcelimport.so
#21 0x423f9ed7 in Swinder::ExcelReader::load ()
   from /home/ariya/cvs/lib/kde3/libexcelimport.so
#22 0x42410857 in Swinder::Workbook::load ()
   from /home/ariya/cvs/lib/kde3/libexcelimport.so
#23 0x423e5f92 in ExcelImport::convert ()
   from /home/ariya/cvs/lib/kde3/libexcelimport.so
#24 0x4008f1d5 in KoFilterChain::ChainLink::invokeFilter ()
   from /home/ariya/cvs/lib/libkofficecore.so.2
#25 0x4008f435 in KoFilterChain::invokeChain ()
   from /home/ariya/cvs/lib/libkofficecore.so.2
#26 0x40074a3e in KoFilterManager::import ()
   from /home/ariya/cvs/lib/libkofficecore.so.2
#27 0x400618a7 in KoDocument::openFile ()
   from /home/ariya/cvs/lib/libkofficecore.so.2
#28 0x40226b96 in KParts::ReadOnlyPart::openURL ()
   from /opt/kde3/lib/libkparts.so.2
#29 0x40063fd0 in KoDocument::openURL ()
   from /home/ariya/cvs/lib/libkofficecore.so.2
#30 0x4007bc49 in KoMainWindow::openDocumentInternal ()
   from /home/ariya/cvs/lib/libkofficecore.so.2
#31 0x4007be42 in KoMainWindow::openDocument ()
   from /home/ariya/cvs/lib/libkofficecore.so.2
#32 0x400836c5 in KoApplication::start ()
   from /home/ariya/cvs/lib/libkofficecore.so.2
#33 0x400198bb in kdemain () from /home/ariya/cvs/lib/libkdeinit_kspread.so
#34 0x08048757 in main ()

Comment 5 Ariya Hidayat 2005-05-29 15:58:18 UTC
OK, I fixed the import filter so it won't crash anymore. But still it takes ages to load and eats too much memory.

Another investigation revealed that importing the Excel file is very fast, but constructing the corresponding KSpread document was extremely slow except for the first few sheets (this file has 81 sheets). This is due to the use of QDom which needs big amount of memory for its operation. An alternative would be to use KOffice's KoDom, so it'll be in my TODO.
Comment 6 Halla Rempt 2005-10-18 15:05:36 UTC
This bug is actually no longer in the filter code, which is fine.
Comment 7 Ariya Hidayat 2005-10-18 15:27:10 UTC
It is still in the filter code, namely the part which generates KSpread document.
Comment 8 claudiu vlad 2006-01-15 22:18:45 UTC
Created attachment 14264 [details]
another killer xls file

This document here also makes kspread eat one billion mb of ram. I hope someone
figures it out. I like koffice more than other similar office suites :).
Comment 9 Alfredo Beaumont 2006-01-29 11:18:50 UTC
I've another file if needed. The problem is with function KoOasisStore::loadAndParse. It seems that QtXmlSimpleReader is not good enough for the task.
Comment 10 Dik Takken 2006-03-21 15:49:22 UTC
KSpread 1.5-beta2 also fails to read both Excel documents: internal error.
Comment 11 Jo Øiongen 2006-04-17 14:12:30 UTC
Is this bug related to bug 59510?

Cheers Jo
Comment 12 Ariya Hidayat 2006-04-19 22:18:03 UTC
Jo: No. This bug is about the filter, as it consumes too much memory just to produce the output.
Comment 13 Ariya Hidayat 2006-04-19 22:30:14 UTC
I already improved the filter in the SVN version of KOffice. Hopefully it will be also included in KOffice 1.5.1 (still need to be scheduled).

If you can compile from source, could you check again?

Here what I got running koconverter on both files:

koconverter kde85372_a.xls kde85372_a.ods
ole-excel: File /home/ariya/kde85372_a.xls loaded. Time: 859 ms
ole-excel: Converted to /home/ariya/kde85372_a.ods. Time: 2394 ms

koconverter kde85372_b.xls kde85372_b.ods
ole-excel: File /home/ariya/kde85372_b.xls loaded. Time: 351 ms
ole-excel: Converted to /home/ariya/kde85372_b.ods. Time: 1344 ms

So both documents are converted fast enough, and without explosion in memory usage.

KSpread will still take time and memory (around 230 MB and 150 MB) when loading the resulting ODS files, or if you open both XLS files from KSpread directly. But that's bug 59510.


Comment 14 Ariya Hidayat 2006-05-12 18:26:33 UTC
I close this bug. See my previous explanation about it.