| Summary: | example netcdf file complains "You cannot load this much data" | ||
|---|---|---|---|
| Product: | [Applications] kst | Reporter: | Timo Lindfors <timo.lindfors> |
| Component: | datasources | Assignee: | kst |
| Status: | RESOLVED FIXED | ||
| Severity: | normal | CC: | arwalker |
| Priority: | NOR | ||
| Version: | 1.7.0 | ||
| Target Milestone: | --- | ||
| Platform: | Debian unstable | ||
| OS: | Linux | ||
| Latest Commit: | Version Fixed In: | ||
| Attachments: | netcdf testcase to reproduce the bug | ||
Created attachment 47611 [details]
netcdf testcase to reproduce the bug
Hi Timo, just my first idea: in kst1, the netcdf datasource does not properly check for dimensions. It can load only vectors, but shows all types of variables, and it is possible that the computation of the number of available samples gives a completely wrong number. But it needs to be investigated a bit more to make sure. You could also try to use a recent kst2 snapshot to see how it behaves there. A couple of days ago the netcdf datasource for kst2 has been substantially improved. It is not 100% done, but may work better. You can find the links and info on the kst mailing list if you need support for that. From what I understand of netcdf only the lon and lat fields actually have data, all the rest are empty. The correct solution would seem to be to remove the empty vectors from the field list - so that the user cannot ask for them to be plotted. I have tried a patch for this in 1.10.0-devel which appears to achieve the desired effect. Let me know if there is any reason not to take this approach. SVN commit 1133924 by arwalker: BUG:240475 return 0 for the samplesForFrame for any empty field. This effectively creates a vector with a single element with a NaN value. This fixes the problem described in this bug report for the kst 1.10 development branch. M +10 -0 netcdf.cpp WebSVN link: http://websvn.kde.org/?view=rev&revision=1133924 Thank you! I can confirm that after I apply http://websvn.kde.org/branches/extragear/kde3/graphics/kst/src/datasources/netcdf/netcdf.cpp?r1=826410&r2=1133924&pathrev=1133924&view=patch I can successfully read testcase1.netcdf. The vectors that are not in use just show a single nan value. |
Version: 1.7.0 (using KDE 4.4.3) OS: Linux Steps to reproduce: 1) cat > testcase1.cdl <<EOF netcdf foo { // an example netCDF specification in CDL dimensions: lat = 10, lon = 5, time = unlimited ; variables: long lat(lat), lon(lon), time(time); float Z(time,lat,lon), t(time,lat,lon); double p(time,lat,lon); long rh(time,lat,lon); // variable attributes lat:long_name = "latitude"; lat:units = "degrees_north"; lon:long_name = "longitude"; lon:units = "degrees_east"; time:units = "seconds since 1992-1-1 00:00:00"; Z:units = "geopotential meters"; Z:valid_range = 0., 5000.; p:_FillValue = -9999.; rh:_FillValue = -1; data: lat = 0, 10, 20, 30, 40, 50, 60, 70, 80, 90; lon = -140, -118, -96, -84, -52; } EOF 2) ncgen -b -o testcase1.netcdf testcase1.cdl 3) kst 4) Data Wizard 5) select testcase1.netcdf 6) select all data columns 7) Next, Next, Finish Expected results: 7) kst opens testcase1.netcdf Actual results: 7) kst prints "You requested to read in over 16777215 TB of data but it seems that you have approximately only 0 TB of usable memory available. You cannot load this much data." More info: 1) The example is from the manual page of the ncgen command that I use to generate netcdf files. 2) If I select only lat and lon columns then kst does not crash and opens the data successfully. Maybe the crash is related to the fact that other columns do not have data points in this file? 3) I am not quite sure which version of KDE is in debian unstable: $ COLUMNS=100 dpkg -l | grep kde ii kde-icons-oxygen 4:4.4.4-1 transitional package for oxygen icon theme ii kdebase-data 4:4.4.4-1 shared data files for the KDE 4 base module rc kdebase-runtime 4:4.3.4-2 runtime components from the official KDE 4 release ii kdebase-runtime-dat 4:4.4.4-1 shared data files for the KDE 4 base runtime module ii kdelibs-data 4:3.5.10.dfsg.1-3 core shared data for all KDE applications ii kdelibs4-dev 4:3.5.10.dfsg.1-3 development files for the KDE core libraries ii kdelibs4c2a 4:3.5.10.dfsg.1-3 core libraries and binaries for all KDE applications rc kdelibs5 4:4.3.4-3 core libraries for all KDE 4 applications rc kdelibs5-data 4:4.3.4-3 core shared data for all KDE 4 applications rc libkdeedu4 4:4.3.4-1+b1 libraries for KDE 4 educational applications rc liblockdev1 1.0.3-1.2+b1 Run-time shared library for locking devices suggests 4.4.4 but that option is not available in bugs.kde.org. Reproducible: Didn't try