Bug 301203

Summary:	In CSV files "empty" values are grouped and lead to column offsets
Product:	[Applications] kst	Reporter:	repaul_schrott-kst
Component:	datasources	Assignee:	kst
Status:	RESOLVED FIXED
Severity:	normal	CC:	daniel, nicolas.brisset
Priority:	NOR
Version:	2.0.5
Target Milestone:	---
Platform:	unspecified
OS:	All
Latest Commit:		Version Fixed In:
Sentry Crash Report:
Attachments:	Zip archive which contains 3 input CSV files, 1 .kst file and the resulting PNG export file. Example csv file there columns are shifted on import

Description repaul_schrott-kst 2012-06-05 10:33:55 UTC

In CSV files the current handling of "empty" values leads to data shifting between columns. Also introducing whitespace changes the behavior.

Examples are:

1,2,3,4,5,6
1,2,3,4,,6
1,2,,4,,
1,,,,,6

vs.

1,2,3,4,5,6
1,2,3,4, ,6
1,2, ,4, ,
1, , , , , 6


Reproducible: Always

Steps to Reproduce:
1.Create an ASCII
2. Load Data
3. Plot results


Expected Results:  
It would like to the data to stay in correct curve/line.
In the plot the lines should stay horizontal.

If desired I can provide test data and bitmaps of the resulting plots.

Comment 1 Nicolas Brisset 2012-06-10 22:25:40 UTC

This sounds pretty serious and should be fixed IMHO.
It would indeed be nice if you could attach some test data to this report.

Comment 2 repaul_schrott-kst 2012-06-11 11:48:38 UTC

Created attachment 71734 [details]
Zip archive which contains 3 input CSV files, 1 .kst file and the resulting PNG export file.

Example was made on Mac. However behavior is the same on Windows and Linux.

Comment 3 Netterfield 2012-06-13 14:32:26 UTC

SVN commit 1300414 by netterfield:


Interpret empty custom-delimited columns as NaN.

NOTE: this is different than non-numeric columns, which are interpreted
by atof() as zero.  I think NaN is the 'correct' interpretation, but
this would slow the interpretation of all ascii files.


 M  +8 -0      asciisource.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=1300414

Comment 4 daniel 2014-10-17 07:50:39 UTC

I  see that this bug is NOT completely solved in Kst 2.0.7 (Ubuntu). With the example above, all seems fine. However, with real data I have (see attachment), I get the same behaviour as described above, empty values do not lead to empty (NaN) vector members. Instead, the value of the next column is taken.
For example, try importing the columns V1 to V4 (the four first columns after the time column). The first member of V1 should be empty, but shows 40.7 which is the first value in the 5th column (CH41).

Comment 5 daniel 2014-10-17 07:55:25 UTC

Created attachment 89170 [details]
Example csv file there columns are shifted on import

Import the ASCII file and look at vectors called V1 to V4. They should be empty but show values from later columns.