Created attachment 150174 [details] I'm attaching a relative simple txt. Its the solution of the heat equation with retention with 301 space nodes and 1000 time nodes. SUMMARY *** NOTE: If you are reporting a crash, please try to attach a backtrace with debug symbols. See https://community.kde.org/Guidelines_and_HOWTOs/Debugging/How_to_create_useful_crash_reports *** STEPS TO REPRODUCE 1. Create a new spreadsheet 2. Load a big txt file with data on it OBSERVED RESULT LabPlot becomes temporarily unresponsive with no indication of progress and then it unfreezes but everything becomes laggy (ploting, spreadsheet resizing) EXPECTED RESULT SOFTWARE/OS VERSIONS Linux/KDE Plasma: ArchLinux kernel 5.18.6-arch1-1 KDE Frameworks Version: 5.95.0 Qt Version: 5.15.5 (compiled with 5.15.4) ADDITIONAL INFORMATION
(In reply to Douglas from comment #0) > STEPS TO REPRODUCE > 1. Create a new spreadsheet > 2. Load a big txt file with data on it This is because of the high number of columns in this file. When importing into a spreadsheet, the data is organized internally in "columns" and we need to create a column object for every column in the file. This, the overhead around this creation and also the high memory consumption after the import is what is causing issues here and this is definitely something that we need to improve. P.S.: on a side node, the import of this data into a matrix container (not spreadsheet) is significantly faster and doesn't cause any issues. Though, the plotting and analysis capabilities for matrix-like data are very poor at the moment in LabPlot...
(In reply to Douglas from comment #0) > STEPS TO REPRODUCE > 1. Create a new spreadsheet > 2. Load a big txt file with data on it > > OBSERVED RESULT > LabPlot becomes temporarily unresponsive with no indication of progress and > then it unfreezes but everything becomes laggy (ploting, spreadsheet > resizing) I did a fix now that improved the performance. On my notebook the time for the import of this file went down from ca. 50s to 22s. It's still very slow of course and we'll improve more soon. But in case you need more performance now and working with our nightly builds is an option for you, you can give it a try.
Git commit da77b31e4dcd6ff2c638ff05290fbc0add0fad90 by Alexander Semke. Committed on 06/07/2022 at 16:09. Pushed by asemke into branch 'master'. [import] speed-up the resize of the spreadsheet columns after the import. The current logic is very expensive for very high number of columns. In such situations we use a different logic where we fit to the header sections only, no parsing of the cell values is done. FIXED-IN: 2.10 M +24 -4 src/commonfrontend/spreadsheet/SpreadsheetView.cpp https://invent.kde.org/education/labplot/commit/da77b31e4dcd6ff2c638ff05290fbc0add0fad90
(In reply to Alexander Semke from comment #2) > (In reply to Douglas from comment #0) > > STEPS TO REPRODUCE > > 1. Create a new spreadsheet > > 2. Load a big txt file with data on it > > > > OBSERVED RESULT > > LabPlot becomes temporarily unresponsive with no indication of progress and > > then it unfreezes but everything becomes laggy (ploting, spreadsheet > > resizing) > I did a fix now that improved the performance. On my notebook the time for > the import of this file went down from ca. 50s to 22s. It's still very slow > of course and we'll improve more soon. But in case you need more performance > now and working with our nightly builds is an option for you, you can give > it a try. This is fixed now. The import of your file takes ca. 1s which is acceptable for this amount of data I think. The improvement will be part of the next release 2.10 and we'll consider to include this change also in the next patch release for 2.9, if we're going to produce a patch release at all without going directly to 2.10. In case you need this improvement sooner, you can consider using our nightly builds for the dev versions of labplot.
(In reply to Alexander Semke from comment #4) > (In reply to Alexander Semke from comment #2) > > (In reply to Douglas from comment #0) > > > STEPS TO REPRODUCE > > > 1. Create a new spreadsheet > > > 2. Load a big txt file with data on it > > > > > > OBSERVED RESULT > > > LabPlot becomes temporarily unresponsive with no indication of progress and > > > then it unfreezes but everything becomes laggy (ploting, spreadsheet > > > resizing) > > I did a fix now that improved the performance. On my notebook the time for > > the import of this file went down from ca. 50s to 22s. It's still very slow > > of course and we'll improve more soon. But in case you need more performance > > now and working with our nightly builds is an option for you, you can give > > it a try. > This is fixed now. The import of your file takes ca. 1s which is acceptable > for this amount of data I think. The improvement will be part of the next > release 2.10 and we'll consider to include this change also in the next > patch release for 2.9, if we're going to produce a patch release at all > without going directly to 2.10. > > In case you need this improvement sooner, you can consider using our nightly > builds for the dev versions of labplot. Thank you so much for your exceptional work! I'll try the nightly version!