Bug 108025 - Matrix and image reworking
Summary: Matrix and image reworking
Status: RESOLVED FIXED
Alias: None
Product: kst
Classification: Applications
Component: general (show other bugs)
Version: 1.x
Platform: Compiled Sources Linux
: NOR normal
Target Milestone: ---
Assignee: Rick Chern
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2005-06-24 00:20 UTC by Andrew Walker
Modified: 2005-08-19 21:19 UTC (History)
1 user (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments
Mockup of proposed new matrix dialog (27.27 KB, image/png)
2005-06-30 20:40 UTC, Rick Chern
Details
Test ASCII file for testing matrices (29.86 KB, application/octet-stream)
2005-07-14 01:46 UTC, Rick Chern
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Andrew Walker 2005-06-24 00:20:18 UTC
Version:           HEAD (using KDE KDE 3.3.0)
Installed from:    Compiled From Sources
OS:                Linux

Some matrix and image reworking needs to be done in the code.

George, please append a design plan asap so that Rick can
get to work on this.
Comment 1 George Staikos 2005-06-24 02:09:10 UTC
I'll provide details ASAP but it will take some time with so many documents to write.  Meanwhile if you want to have discussions about design and about every feature inside bugzilla, you'll need to add kst@kde.org as a cc: when you reassign.  I use the email interface for bugzilla primarily and I (as well as others) won't be seeing the conversation.
Comment 2 George Staikos 2005-06-28 00:34:01 UTC
Kst contains types, just as a typical computer application.  In C++, there
are primitive types (POD) and derived types (classes).  In Kst, we have the
following categorization:

Primitive Types         Derived Types           Interpretations
-------------------------------------------------------------------------
KstScalar               KstPSD                  KstVCurve
KstString               KstEquation             KstImage*
KstVector               KstHistogram
KstRVector              KstPlugin
KstAVector              KstCSD
KstSVector
KstMatrix*
KstRMatrix
KstAMatrix
KstSMatrix



KstMatrix and KstImage are presently wrong in Kst.  They are both derived
types, and yet one represents primitive underived data (though it is not
implemented that way), and the other is used directly to interpret that
data as a visualization.


Primitive types can be created by the user, or loaded from storage.  The
present data loading mechanism is flawed in that it only supports loading
one type of data from disk: vectors.  KstDataSource needs to be expanded
with a keyed plugin mechanism and needs to drop binary compatibility
issues [-internally- -only-] so that it can be extended to support these
other primitive types.  This key is implemented in the development code.

The next step is to create Kst?Matrix classes which behave as close as
possible to the same way Kst?Vector does, with an interface as close as
possible to that of Kst?Vector.  Once this is done, the KstDataSource
interface can be extended to support loading them (as well as all other
primitive types).  The Matrix dialog will become an equivalent to the
Vector dialog.

Note that Matrix needs to be a primitive type internally too.  It should
not use KstVector.

After matrices are renovated, KstImage needs to be converted into a
KstBaseCurve and adjusted somewhat to be a generic curve with a matrix
for storing data.  It is desired at some point to be able to render all
sort of different image formats easily with this class.  This change,
however, depends on changes to Kst2DPlot to an extent, and should wait
for those changes.  (they are minor)
Comment 3 Rick Chern 2005-06-30 20:40:38 UTC
Created attachment 11638 [details]
Mockup of proposed new matrix dialog

Mockup of proposed new matrix dialog.  
- users can no longer specify X and Y dimensions of the matrix 
- the datasource provides functions for reading a sub-range of the matrix
- the user can still specify scaling (x step size, y step size) and translation
(x minimum, y minimum) for the matrix, as these are just interpretation
parameters for the raw "blocks" provided by the datasource
Comment 4 Rick Chern 2005-07-06 02:05:30 UTC
Proposed new functions for KstDataSource:

// returns the list of matrices this datasource contains
virtual QStringList matrixList();

// whether or not the matrix name is a valid name of a matrix
virtual bool isValidMatrix(const QString& matrix);

// read the specified sub-range of the matrix, flat-packed in z
// the suggested scaling and translation is returned in 
// xMin, yMin, xStepSize, and yStepSize
virtual int readMatrix(double *z, const QString& matrix, int xStart, int yStart, int xNumSteps, int yNumSteps, double& xMin, double& yMin, double& xStepSize, double& yStepSize);

The list of matrices is separate from the list of field names, but the datasource may choose to support all, no, or some vector fields as matrices.

There is no concept of frames for matrices.

For live data in conjunction with the "read to end" option, the matrix dimensions xMin, yMin, may change on every read.

Comment 5 George Staikos 2005-07-06 15:56:33 UTC
On Thursday 30 June 2005 14:40, Rick Chern wrote:
> 20:40 ------- Created an attachment (id=11638)
>  --> (http://bugs.kde.org/attachment.cgi?id=11638&action=view)
> Mockup of proposed new matrix dialog
>
> Mockup of proposed new matrix dialog.
> - users can no longer specify X and Y dimensions of the matrix
> - the datasource provides functions for reading a sub-range of the matrix
> - the user can still specify scaling (x step size, y step size) and
> translation (x minimum, y minimum) for the matrix, as these are just
> interpretation parameters for the raw "blocks" provided by the datasource


  Would this really apply (the vector part) since the matrix is provided by 
the data source?
Comment 6 George Staikos 2005-07-06 16:09:55 UTC
On Tuesday 05 July 2005 20:05, Rick Chern wrote:
> // returns the list of matrices this datasource contains
> virtual QStringList matrixList();
>
> // whether or not the matrix name is a valid name of a matrix
> virtual bool isValidMatrix(const QString& matrix);
>
> // read the specified sub-range of the matrix, flat-packed in z
> // the suggested scaling and translation is returned in
> // xMin, yMin, xStepSize, and yStepSize
> virtual int readMatrix(double *z, const QString& matrix, int xStart, int
> yStart, int xNumSteps, int yNumSteps, double& xMin, double& yMin, double&
> xStepSize, double& yStepSize);


  I really don't like passing data out in references.  It's very unclear 
what's happening, and Kst2DPlot is turning into spaghetti (partially) because 
of it.  I think it makes more sense either to use a context handle, or use 
pointers so that we see the & and know that it can be modified.

  What is the return code mechanism for readMatrix()?

  Ted, what do you think of this interface?

> The list of matrices is separate from the list of field names, but the
> datasource may choose to support all, no, or some vector fields as
> matrices.


  Seems reasonable to me.  Maybe it makes sense to split it up like this:

fieldList(); // all fields of all types
vectorList();
matrixList();
scalarList();
stringList();

  And perhaps have a method to query the capabilities of a given field name.

> There is no concept of frames for matrices.


  Why not?  Does it never make sense?
Comment 7 Ted Kisner 2005-07-06 22:36:35 UTC
Sorry for the late reply, I'll be in poor email contact for the next
couple weeks...  I don't have too strong a preference on the
datasource interface.  I like the idea of providing a different set of
fields that are available for each type of data.  This is much better
than having a global field list and then just returning zero samples
if reading the field as that type is not supported...

sorry I can't be more helpful with this effort in the short term...

-Ted

On 6 Jul 2005 14:09:57 -0000, George Staikos <staikos@kde.org> wrote:
[bugs.kde.org quoted mail]
Comment 8 Rick Chern 2005-07-11 19:29:53 UTC
On Wed, 2005-07-06 at 13:56 +0000, George Staikos wrote:
[bugs.kde.org quoted mail]

Sorry, the Data Vector Parameters section should be Data Source
Parameters (it works similarly to the section in the Edit/New vector
dialog).
Comment 9 Rick Chern 2005-07-11 20:00:47 UTC
Suggested functions version 2:

isValidMatrix as before, and:
fieldList(); // all fields of all types
vectorList();
matrixList();
scalarList();
stringList(); 

and:

// read the specified sub-range of the matrix, flat-packed in z
// xStart - starting x *frame*
// yStart - starting y *frame*
// xNumSteps - number of *frames* to read in x direction; 
               -1 to read 1 *sample* from xStart
// yNumSteps - number of *frames* to read in y direction;
               -1 to read 1 *sample* from yStart 
// Will skip according to the parameter, but it may not
// be implemented.  If it returns -9999, use the non-skip version instead. 
// The suggested scaling and translation is returned in
// xMin, yMin, xStepSize, and yStepSize
// Returns the number of *samples* read
virtual int readMatrix(double *z, const QString& matrix, int xStart, int yStart, int xNumSteps, int yNumSteps, int skip, double* xMin, double* yMin, double* xStepSize, double* yStepSize); 

// read the specified sub-range of the matrix, flat-packed in z
// xStart - starting x *frame*
// yStart - starting y *frame*
// xNumSteps - number of *frames* to read in x direction; 
               -1 to read 1 *sample* from xStart
// yNumSteps - number of *frames* to read in y direction;
               -1 to read 1 *sample* from yStart 
// The suggested scaling and translation is returned in
// xMin, yMin, xStepSize, and yStepSize
// Returns the number of *samples* read
virtual int readMatrix(double *z, const QString& matrix, int xStart, int yStart, int xNumSteps, int yNumSteps, double* xMin, double* yMin, double* xStepSize, double* yStepSize); 
Comment 10 George Staikos 2005-07-12 04:06:41 UTC
> 20:00 ------- Suggested functions version 2:
>
> isValidMatrix as before, and:
> fieldList(); // all fields of all types
> vectorList();
> matrixList();
> scalarList();
> stringList();


  I agree.

> and:
>
> // read the specified sub-range of the matrix, flat-packed in z

[...]

  I think we should use a context variable/struct of sorts to manage this 
stuff.  These functions have too many arguments and look really confusing 
(including the vector ones we had before).
Comment 11 Rick Chern 2005-07-14 01:46:28 UTC
Created attachment 11796 [details]
Test ASCII file for testing matrices

Use this for testing KstRMatrix and Images - the 3rd column should be
recognized by the test AsciiSource as a matrix.
Comment 12 Rick Chern 2005-07-19 01:06:11 UTC
Matrix reworking should be done.  KstDataSource has been modified to support matrices from data sources.  AsciiSource has been modified with test functions to support matrices in ASCII files.  KstMatrix, KstRMatrix, KstSMatrix, and KstAMatrix classes have been implemented and appropriate changes to the rest of Kst to work with the new classes have been made.  

Waiting on changes to Kst2DPlot so that the Image reworking can be done.
Comment 13 George Staikos 2005-07-25 19:32:04 UTC
Which changes for Kst2DPlot do you need to wait for besides factoring out the drawing code?
Comment 14 Rick Chern 2005-07-25 19:54:45 UTC
I am waiting for the changes mentioned previously: 

After matrices are renovated, KstImage needs to be converted into a
KstBaseCurve and adjusted somewhat to be a generic curve with a matrix
for storing data.  It is desired at some point to be able to render all
sort of different image formats easily with this class.  This change,
however, depends on changes to Kst2DPlot to an extent, and should wait
for those changes.  (they are minor) 

For now, should KstImage just be changed into a subclass of KstBaseCurve (and most of the plotting code would stay the same)? 

Comment 15 George Staikos 2005-07-25 20:43:48 UTC
On Monday 25 July 2005 13:54, Rick Chern wrote:
> 19:54 ------- I am waiting for the changes mentioned previously:
> After matrices are renovated, KstImage needs to be converted into a
> KstBaseCurve and adjusted somewhat to be a generic curve with a matrix
> for storing data.  It is desired at some point to be able to render all
> sort of different image formats easily with this class.  This change,
> however, depends on changes to Kst2DPlot to an extent, and should wait
> for those changes.  (they are minor)
>
> For now, should KstImage just be changed into a subclass of KstBaseCurve
> (and most of the plotting code would stay the same)?


   I actually did that (KstImage into a BaseCurve) once before but it was too 
close to the release so I had to junk it.  It's quite diverged now, so the 
patch is useless.  It did seem to work, though, so I think it's the right 
approach.  If you want you can pull out the Kst2DPlot code too.  The first 
step should look roughly like this:

  1) BaseCurve properties that really belong in VCurve get demoted.
  2) KstCurveRenderContext is created which contains the long list of 
parameters (and the painter) that a curve needs in order to draw itself into 
that painter.
  3) BaseCurve gets a paint()/draw()/whatever virtual function that takes this 
context and the painting code moves from Kst2DPlot to the curves.
  4) KstImage becomes a KstBaseCurve

  I'll take care of updating the script bindings as the changes happen.  Once 
this is done, I should be able to bind matrices and images completely.

  Is there anything else you know of that we should deal with while we're 
fixing it?  The label code is also a big issue, but it's fairly separate.
Comment 16 Rick Chern 2005-07-30 01:53:24 UTC
SVN commit 440144 by rchern:

Image reworking:
- Many KstBaseCurve properties moved to KstVCurve
- KstImage turned into KstBaseCurve
- Much code referring to KstImage removed from 2dplot, data manager, dialogs, updatethread etc..
- functions provided by KstBaseCurve removed from KstImage
- KstBaseCurve gets paint() function, curve/image rendering code removed from 2dplot, and KstCurveRenderContext struct created
- the following virtual functions added to KstBaseCurve:

virtual void paint(KstCurveRenderContext context) = 0;
virtual KstDataObjectPtr providerDataObject() const = 0;
virtual double distanceToPoint(double xpos, double dx, double ypos) const = 0;
virtual void yRange(double xFrom, double xTo, double* yMin, double* yMax) = 0;

CCMAIL: 108025@bugs.kde.org



 M  +6 -4      datawizard.ui.h  
 M  +136 -1298 kst2dplot.cpp  
 M  +5 -22     kst2dplot.h  
 M  +1 -178    kstbasecurve.cpp  
 M  +49 -94    kstbasecurve.h  
 M  +1 -5      kstchangefiledialog_i.cpp  
 M  +1 -1      kstcolorsequence.cpp  
 M  +2 -2      kstcolorsequence.h  
 M  +3 -3      kstcsddialog_i.cpp  
 M  +1 -59     kstdatamanager_i.cpp  
 M  +0 -2      kstdatamanager_i.h  
 M  +1 -17     kstdoc.cpp  
 M  +3 -3      kstfilterdialog_i.cpp  
 M  +3 -3      kstfitdialog_i.cpp  
 M  +7 -2      kstiface_impl.cpp  
 M  +301 -3    kstimage.cpp  
 M  +18 -12    kstimage.h  
 M  +2 -2      kstimagedialog_i.cpp  
 M  +27 -23    kstlegend.cpp  
 A             kstmathutils.cpp   [License: GPL (v2+)]
 A             kstmathutils.h   [License: GPL (v2+)]
 M  +2 -34     kstplotdialog_i.cpp  
 M  +942 -1    kstvcurve.cpp  
 M  +77 -9     kstvcurve.h  
 M  +8 -6      main.cpp  
 M  +0 -1      threadevents.h  
 M  +0 -37     updatethread.cpp  
 M  +0 -2      updatethread.h  
Comment 17 Rick Chern 2005-08-19 21:19:54 UTC
Matrix and image reworking should be complete.