| Summary: | Handles large spreadsheets "extremely" poorly | ||
|---|---|---|---|
| Product: | [Applications] calligrasheets | Reporter: | Sean Clarke <sean.clarke> |
| Component: | general | Assignee: | Laurent Montel <montel> |
| Status: | RESOLVED FIXED | ||
| Severity: | normal | CC: | celer, esigra, prigault, sroberts, vivek.rai |
| Priority: | NOR | ||
| Version: | 1.2.90 | ||
| Target Milestone: | --- | ||
| Platform: | openSUSE | ||
| OS: | Linux | ||
| Latest Commit: | Version Fixed In: | ||
| Sentry Crash Report: | |||
| Attachments: |
CSV testcase
Mortgage amortization |
||
|
Description
Sean Clarke
2003-06-08 13:33:05 UTC
I can't attach the Kspread document as it is 1.3MB in size (there is an upload limit of 1MB). If you require the file, send me an email or an ftp location (I can put it on a server and you can download it via HTTP if you wish?). I also have the file in CSV, StarOffice and Excel formats if that is any use. Halo Sean, KSpread is known to have problem with large files. We are working on that, but not for the current release. About the test file, yes I can download the file given the URL. The other solution would be to cut part of the file (until it fits in 1 MB or less) and attach it here. I guess 1 MB or 1.3 MB would not make any big difference. The test file can be found at : http://homepage.ntlworld.com/sean.e.clarke/Master_Database_9256.ksp Just bear in mind that when you open it Kspread "may" want up to 310MB of RAM. Just tested this again with KOffice 1.3 b3 (1.2.92), problem still present. Problem still present in KOffice 1.3 release - now uses 336MB. Don't quite understand how a ~1.3MB file can cause an application to use >330MB of RAM. Application doesn't crash, once loaded it works as normal albeit slower. *** Bug 55872 has been marked as a duplicate of this bug. *** Still present in 1.3.5 Cannot find the file when I wanted to test it. Shall I close the bug now? :-) *** Bug 113530 has been marked as a duplicate of this bug. *** Created attachment 14666 [details]
CSV testcase
Simple testcase: CSV file with 10,000 rows x 10 columns, all cells set to '1'
Loads instantly (<1 sec) in OpenOffice.org
Hogs CPU for ages (23 min !) in kspread. It does not eat up much RAM though, so
this is clearly not system-limited in this case.
Timings one on a Dual Opteron 2.0 GHz with 4GB RAM.
This is not a 'Normal' bug. It is a grave one that makes kspread unusable for
most academic and scientific users, who routinely exchange thousands of lines
of data in text/CSV.
Note following comment#10: Once the spreadsheet is loaded, exporting to CSV is instantaneous. Is this bug related to bug 85372? kspread 1.6.0 (KDE 3.5.5, Debian Package 4:3.5.5a.dfsg.1-3 (4.0)) Linux (x86_64) release 2.6.18.3 Simple testcase described above does not load instantly, but does within 3 seconds. That makes it acceptable, no? Timing is on AMD64 3000 with 1GB Hmm, probably so. I wonder if the bug got fixed or if the hardware got faster. Thanks, celer +--[celer]--+ On Sun, 26 Nov 2006, Ernest ter Kuile wrote: [bugs.kde.org quoted mail] Indeed, there is an improvement with regard to the problem described in comment #10 (tested with kspread 1.6.1). Here are my measurements for loading the CSV testcase on a Sun Ultra20 (DualCore AMD Opteron 2.4GHz, 2GB RAM): 2 sec: Time from opening the file to the "Import" menu 8 sec: Time from starting import to fully loaded document Note that this is still much (5x) slower than OpenOffice.org-calc (respective timings are 0 sec and 2 sec). There is still a _big_ problem with regard to memory usage in kspread. Look at pmap measurements (http://web.hexapodia.org/~adi/pmap.c) Kspread, when started with an empty sheet, uses the following memory: mapped: 269708 KB writable/private: 9412 KB shared: 1928 KB and when the CSV testcase is loaded shows: mapped: 621012 KB writable/private: 350912 KB shared: 1928 KB which is an increase of 340 MB in private memory. In comparison: OpenOffice empty sheet (look at this hog, 90MB for an empty sheet) mapped: 534084 KB writable/private: 90768 KB shared: 12644 KB OpenOffice loaded with CSV testcase mapped: 659616 KB writable/private: 109000 KB shared: 12940 KB which shows an increase of only 19 MB. If I load a 30,000 row x 30 columns CSV document, the numbers are: Kspread loads in 36 sec (8 sec + 28 sec) memory increases by 970 MB mapped: 1244076 KB writable/private: 973976 KB shared: 1928 KB OpenOffice loads in 5 sec (0 sec + 5 sec) memory increases by 50 MB mapped: 703300 KB writable/private: 141048 KB shared: 12940 KB Created attachment 18950 [details]
Mortgage amortization
Mortgage amortization
I can beat that. See 'Mortgage amortization', open it in kspread, and just use the arrow keys to move around. It's got about 200 rows, and moving to the bottom and then back around caused kspread to hang and eat 500M of memory on my system before I killed it. This .ksp was generated using kspread 1.2 or 1.3, but it loads and displays data ok under my version of kspread. But moving around it or changing the numbers at the top (initial amount, interest rate, monthly payment) seems to cause it to reproducibly hang. The .ksp is all of 27kb, about 2Mb uncompressed. Kubuntu 6.06 running kde 3.5.5 and koffice 1.6 as backports: Qt: 3.3.6 KDE: 3.5.5 KSpread: 1.6.0 You seem to describe a different bug, since 'Mortgage amortization' loads quite quickly. It looks like this one has trouble with rendering its (many) formulas. *** Bug 142572 has been marked as a duplicate of this bug. *** Closing. The state for the current development version: The file in #10 loads fast enough and the memory consumption got reduced. The file in #17 loads okay and memory consumption is okay. Can't say anything about the reporter's file, because it's not accessible anymore. |