Bug 301203 - In CSV files "empty" values are grouped and lead to column offsets
Summary: In CSV files "empty" values are grouped and lead to column offsets
Status: RESOLVED FIXED
Alias: None
Product: kst
Classification: Applications
Component: datasources (show other bugs)
Version: 2.0.5
Platform: unspecified All
: NOR normal
Target Milestone: ---
Assignee: kst
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-06-05 10:33 UTC by repaul_schrott-kst
Modified: 2014-10-17 07:55 UTC (History)
2 users (show)

See Also:
Latest Commit:
Version Fixed In:
Sentry Crash Report:


Attachments
Zip archive which contains 3 input CSV files, 1 .kst file and the resulting PNG export file. (35.09 KB, application/octet-stream)
2012-06-11 11:48 UTC, repaul_schrott-kst
Details
Example csv file there columns are shifted on import (1.38 KB, text/plain)
2014-10-17 07:55 UTC, daniel
Details

Note You need to log in before you can comment on or make changes to this bug.
Description repaul_schrott-kst 2012-06-05 10:33:55 UTC
In CSV files the current handling of "empty" values leads to data shifting between columns. Also introducing whitespace changes the behavior.

Examples are:

1,2,3,4,5,6
1,2,3,4,,6
1,2,,4,,
1,,,,,6

vs.

1,2,3,4,5,6
1,2,3,4, ,6
1,2, ,4, ,
1, , , , , 6


Reproducible: Always

Steps to Reproduce:
1.Create an ASCII
2. Load Data
3. Plot results


Expected Results:  
It would like to the data to stay in correct curve/line.
In the plot the lines should stay horizontal.

If desired I can provide test data and bitmaps of the resulting plots.
Comment 1 Nicolas Brisset 2012-06-10 22:25:40 UTC
This sounds pretty serious and should be fixed IMHO.
It would indeed be nice if you could attach some test data to this report.
Comment 2 repaul_schrott-kst 2012-06-11 11:48:38 UTC
Created attachment 71734 [details]
Zip archive which contains 3 input CSV files, 1 .kst file and the resulting PNG export file.

Example was made on Mac. However behavior is the same on Windows and Linux.
Comment 3 Netterfield 2012-06-13 14:32:26 UTC
SVN commit 1300414 by netterfield:


Interpret empty custom-delimited columns as NaN.

NOTE: this is different than non-numeric columns, which are interpreted
by atof() as zero.  I think NaN is the 'correct' interpretation, but
this would slow the interpretation of all ascii files.


 M  +8 -0      asciisource.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=1300414
Comment 4 daniel 2014-10-17 07:50:39 UTC
I  see that this bug is NOT completely solved in Kst 2.0.7 (Ubuntu). With the example above, all seems fine. However, with real data I have (see attachment), I get the same behaviour as described above, empty values do not lead to empty (NaN) vector members. Instead, the value of the next column is taken.
For example, try importing the columns V1 to V4 (the four first columns after the time column). The first member of V1 should be empty, but shows 40.7 which is the first value in the 5th column (CH41).
Comment 5 daniel 2014-10-17 07:55:25 UTC
Created attachment 89170 [details]
Example csv file there columns are shifted on import

Import the ASCII file and look at vectors called V1 to V4. They should be empty but show values from later columns.