Bug 352367

Summary: vectors too short. last frame not plotted
Product: [Applications] kst Reporter: Michael Brewer <brewer>
Component: plottingAssignee: kst
Status: RESOLVED WORKSFORME    
Severity: major CC: brewer, netterfield
Priority: NOR    
Version: 2.0.8   
Target Milestone: ---   
Platform: Ubuntu   
OS: Linux   
Latest Commit: Version Fixed In:
Sentry Crash Report:

Description Michael Brewer 2015-09-06 21:51:32 UTC
I have a set of dirfiles containing 600 frames with 200 samples per frame 120000 samples total.
I open the files with Update Type set to No Update. When I create a plot with Read to End set the INDEX field goes from 0 to 599 instead of 0 to 600 as it should. Checking View Vectors confirms that the vectors are being cut off at 119801 samples. This is 599 frames plus one sample.


Reproducible: Always

Steps to Reproduce:
1. Create a dirfile with an integer number of frames
2.Make a plot and check the last INDEX value
3. Check View Vectors to confirm that crated vector is truncated

Actual Results:  
Vectors contain one less frame than what is present in dirfile

Expected Results:  
Create vectors using all data or at least all complete frames present in dirfile
Comment 1 Netterfield 2016-02-24 22:47:58 UTC
This is, surprisingly, by design.

We address data in a file by its index.  We often plot data in a file against a field called INDEX, which is always 1 sample per frame.

kst assumes that INDEX is coincident in time with the first sample in a frame.
This clearly makes sense for the first sample of a file, but what to do at the end of a file is less clear.

The only alternative to this that I can think of is to produce new INDEX-like pseudo-fields which are at the same rate as any given field.  This is complicated by the fact that different fields may have data at different rates, so there would need to be one for every SPF in the file.  Perhaps fully automatic?  

We can think about this some more.
Comment 2 Michael Brewer 2016-02-25 13:06:06 UTC
Following this logic, if my data set consisted of less than 201 samples, KST would display nothing. In my opinion, the INDEX vector should always extend to one point beyond the number of frames (or partial frames) present in the data set to capture all of the data that the user may be interested in. It is like the X axis on a plot which should extend slightly past the end of the data. Another way to look at this is that each frame represents a unit of time (one second in my case). If I have 10 seconds of data, then the INDEX field shoold extend from 0 seconds to 10 seconds, not from 0 seconds to 9 seconds.

In fact, my data set contains fields with different sample rates. Most of the data is sampled at 200 Hz. But some of it is sampled at 10 Hz and some at 1 Hz. Extending the INDEX vector by 1 would capture all of this data.
Comment 3 Netterfield 2016-02-26 00:37:59 UTC
On February 25, 2016 01:06:06 PM you wrote:
> https://bugs.kde.org/show_bug.cgi?id=352367
> 
> --- Comment #2 from Michael Brewer <brewer@astro.umass.edu> ---
> Following this logic, if my data set consisted of less than 201 samples, KST
> would display nothing.

Right: Imagine you want to plot a field which is 100 samples per frame vs 
something that is 1 samples per frame.  And you want to plot one frame.  Maybe 
because your datasource is only one frame long.  What should you do?

There is clearly no general solution to this problem.

> In my opinion, the INDEX vector should always extend
> to one point beyond the number of frames (or partial frames) present in the
> data set to capture all of the data that the user may be interested in. 

INDEX is nothing special: it is something provided by some data sources.  It 
would be just the same if we were talking about any field sampled once per 
frame; say.... "Time".  So all vectors from a data source would need to do 
this.  But then we would need to know to ignore the last extrapolated sample 
for everything other than interpolating.

> It
> is like the X axis on a plot which should extend slightly past the end of
> the data. Another way to look at this is that each frame represents a unit
> of time (one second in my case). If I have 10 seconds of data, then the
> INDEX field shoold extend from 0 seconds to 10 seconds, not from 0 seconds
> to 9 seconds.

Formally, Index goes from 0 to 10 - (1/sample rate), but it needs to be an 
integer.

> 
> In fact, my data set contains fields with different sample rates. Most of
> the data is sampled at 200 Hz. But some of it is sampled at 10 Hz and some
> at 1 Hz. Extending the INDEX vector by 1 would capture all of this data.

It is a tricky problem to get right everywhere, but I will mull upon it 
further (and not close the bug...)
Comment 4 Michael Brewer 2016-02-26 15:26:34 UTC
From your previous comment, I got the impression that INDEX was something specific to the plot. If INDEX is in fact just another 1 sample per frame vector, then I agree that it should be the same as all the other 1 sample per frame vectors.

*******
Right: Imagine you want to plot a field which is 100 samples per frame vs something that is 1 samples per frame. And you want to plot one frame. Maybe because your datasource is only one frame long. What should you do? 
*******

Currently, this question is moot since it is not possible to plot one frame of data in KST.
But that is my point. If the user has only one frame of data, there should be a way for them to look at that data. In the example that you give, there is only one X-Y pair to plot. In order to plot this, the plot itself needs to extend from 0 to 1. In general, the plot should always extend from 0 to nframes. This is one more than the INDEX array that extends from 0 to nframes - 1. Since this is just a matter of setting the boundaries of the X axis of the plot, it seems straight forward to me. Perhaps I am missing something though.
Comment 5 Michael Brewer 2016-02-26 15:49:04 UTC
(In reply to Michael Brewer from comment #4)
> From your previous comment, I got the impression that INDEX was something
> specific to the plot. If INDEX is in fact just another 1 sample per frame
> vector, then I agree that it should be the same as all the other 1 sample
> per frame vectors.
> 
> *******
> Right: Imagine you want to plot a field which is 100 samples per frame vs
> something that is 1 samples per frame. And you want to plot one frame. Maybe
> because your datasource is only one frame long. What should you do? 
> *******
> 
> Currently, this question is moot since it is not possible to plot one frame
> of data in KST.
> But that is my point. If the user has only one frame of data, there should
> be a way for them to look at that data. In the example that you give, there
> is only one X-Y pair to plot. In order to plot this, the plot itself needs
> to extend from 0 to 1. In general, the plot should always extend from 0 to
> nframes. This is one more than the INDEX array that extends from 0 to
> nframes - 1. Since this is just a matter of setting the boundaries of the X
> axis of the plot, it seems straight forward to me. Perhaps I am missing
> something though.

BTW,  in your example if the user has two fields of data at 100 samples per frame that they want to plot using one field as X and the other as Y, then there are 100 X-Y pairs to plot. Does it make sense that KST should do nothing?
Comment 6 Justin Zobel 2022-10-15 10:59:49 UTC
Thank you for reporting this bug in KDE software. As it has been a while since this issue was reported and confirmed, can we please ask you to see if you can reproduce the issue with a recent software version?

If you can reproduce the issue, please change the status to "CONFIRMED" when replying. Thank you!
Comment 7 Bug Janitor Service 2022-10-30 05:03:47 UTC
Dear Bug Submitter,

This bug has been in NEEDSINFO status with no change for at least
15 days. Please provide the requested information as soon as
possible and set the bug status as REPORTED. Due to regular bug
tracker maintenance, if the bug is still in NEEDSINFO status with
no change in 30 days the bug will be closed as RESOLVED > WORKSFORME
due to lack of needed information.

For more information about our bug triaging procedures please read the
wiki located here:
https://community.kde.org/Guidelines_and_HOWTOs/Bug_triaging

If you have already provided the requested information, please
mark the bug as REPORTED so that the KDE team knows that the bug is
ready to be confirmed.

Thank you for helping us make KDE software even better for everyone!
Comment 8 Bug Janitor Service 2022-11-14 05:16:42 UTC
This bug has been in NEEDSINFO status with no change for at least
30 days. The bug is now closed as RESOLVED > WORKSFORME
due to lack of needed information.

For more information about our bug triaging procedures please read the
wiki located here:
https://community.kde.org/Guidelines_and_HOWTOs/Bug_triaging

Thank you for helping us make KDE software even better for everyone!