Bug 386800 - Custom fitting often fails
Summary: Custom fitting often fails
Status: RESOLVED FIXED
Alias: None
Product: LabPlot2
Classification: Applications
Component: general (show other bugs)
Version: latest
Platform: Microsoft Windows Microsoft Windows
: NOR major
Target Milestone: ---
Assignee: Stefan Gerlach
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-11-12 23:18 UTC by Uwe Stöhr
Modified: 2018-05-13 22:43 UTC (History)
2 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments
fit parameter b is red (230.72 KB, image/jpeg)
2017-11-12 23:18 UTC, Uwe Stöhr
Details
no fit result and senseless fit parameters (244.85 KB, image/jpeg)
2017-11-12 23:19 UTC, Uwe Stöhr
Details
Fit result from MagicPlot (183.46 KB, image/jpeg)
2017-11-12 23:27 UTC, Uwe Stöhr
Details
MagicPlot good fit result with a=0.01 (178.86 KB, image/jpeg)
2017-11-12 23:36 UTC, Uwe Stöhr
Details
screenshot of test case (156.47 KB, image/png)
2017-11-18 09:56 UTC, Stefan Gerlach
Details
lml testfile (3.16 KB, application/gzip)
2017-12-18 01:51 UTC, Uwe Stöhr
Details
another test file (3.69 KB, application/gzip)
2017-12-18 01:54 UTC, Uwe Stöhr
Details
.lml test file (3.69 KB, application/gzip)
2018-02-12 00:34 UTC, Uwe Stöhr
Details
test as described by OP (5.17 KB, application/x-labplot2)
2018-05-08 15:49 UTC, urcindalo
Details
Test works OK in Linux (5.14 KB, application/x-labplot2)
2018-05-09 11:32 UTC, urcindalo
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Uwe Stöhr 2017-11-12 23:18:22 UTC
Created attachment 108818 [details]
fit parameter b is red

- create a new table with 2 columns
- fill the table columns with row numbers
- create a new custom fit
- use as fit function this one
exp(b*sin(a*x))+c
- set fit interval x:10 - 25
- set as start value for all parameters "1"

result: 
1. the fit parameter b gets red, see attached
2. one cannot see the fit functions using the given parameters. Therefore one doesn't know is the parameter "1" is sensible.

- Now press the recalculate button

result: one doesn't get a fit curve nor sensible values for the fit parameters, see attached.

But this fit is possible without problems, see the attached screenshot from MagicPlot.
Comment 1 Uwe Stöhr 2017-11-12 23:19:35 UTC
Created attachment 108819 [details]
no fit result and senseless fit parameters
Comment 2 Uwe Stöhr 2017-11-12 23:27:38 UTC
Created attachment 108821 [details]
Fit result from MagicPlot

The fit is bad but at least is exists and one can see it. One gets the idea that setting parameter a to a low value should result in a good fit.
Comment 3 Uwe Stöhr 2017-11-12 23:33:55 UTC
OK, let's assume the user got the idea that the parameter a has to be a low value despite LP doesn't show him the fit function.

- Therefore let's fit with the parameters
a: "0.01", b and c with "1"

result: still not fit. So the fit routine in LP seems to have an error because doing the same with MagicPlot leads to a nice fit result, see attached.

now it gets even worse:

- now use again
a: "0.01", b and c with "1"
but this time set a to be fixed.

result: one gets for the parameter a the value 1.79769e+308 although it was set fixed and must not be changed during the fit.

I played a  bit with fitting and because of this bug LP is not usable for me regarding custom fits.
Comment 4 Uwe Stöhr 2017-11-12 23:36:02 UTC
Created attachment 108823 [details]
MagicPlot good fit result with a=0.01

I attached is the fit result of MagicPlot using the start parameters a=0.01, b=1 and c=1.
Comment 5 Stefan Gerlach 2017-11-18 09:56:43 UTC
Created attachment 108942 [details]
screenshot of test case

I could not reproduce the problem. Attached my screenshot of your test case. Is there anything i may have missed?
Comment 6 Uwe Stöhr 2017-11-19 22:21:34 UTC
> I could not reproduce the problem. Attached my screenshot of your test case. Is there anything i may have missed?

Then this bug was fixed in the meantime. I use the same settings as you and "0.01" as start value for the parameter a, 1 for b and c. As result i get what I posted:
https://bugsfiles.kde.org/attachment.cgi?id=108819

To be able to check if this bug is fixed, I need a new build from you.
Comment 7 Stefan Gerlach 2017-12-17 22:08:34 UTC
I uploaded a new package at
https://theorie.physik.uni-konstanz.de/gerlach/labplot/labplot-2.5.0rc4-64bit-setup.exe
Comment 8 Uwe Stöhr 2017-12-18 01:50:13 UTC
> I uploaded a new package

The bug persists for me, with your build and also with mine. Attached is my test file.

The situation is now worse: The recalculate button is always disabled as you can see in my test file. In order to get a fit I have to create a new fit plot.

-> major regression
Comment 9 Uwe Stöhr 2017-12-18 01:51:23 UTC
Created attachment 109437 [details]
lml testfile

test project showing the bug
Comment 10 Uwe Stöhr 2017-12-18 01:54:32 UTC
Created attachment 109438 [details]
another test file

with this test file the Recalculate button is always disabled
Comment 11 Stefan Gerlach 2018-02-11 17:03:23 UTC
Do this problem still exist in the latest version 2.5rc8?
Comment 12 Uwe Stöhr 2018-02-12 00:32:19 UTC
(In reply to Stefan Gerlach from comment #11)
> Do this problem still exist in the latest version 2.5rc8?

Yes. When opening the attached .lml file I still get both errors:

> 1. the fit parameter b gets red, see attached
> 2. one cannot see the fit functions using the given parameters. Therefore one doesn't know is the parameter "1" is sensible.

Additionally:

3. The fit parameter window can still not be moved.
4. I cannot press the recalculate button because it is permanently grayed-out.

If I therefore create a new fit to be able to calculate I still get senseless results.
Comment 13 Uwe Stöhr 2018-02-12 00:34:01 UTC
Created attachment 110551 [details]
.lml test file

Test file for comment 12
Comment 14 Stefan Gerlach 2018-03-10 12:11:41 UTC
the data columns are not restored when opening your project. What happens when  you select the x- and y-column correctly?
The fit parameter are using lower and upper limits for the values. Which values for a,b,c do you expect for best fitting you data?
Comment 15 Christoph Feck 2018-03-30 18:29:10 UTC
If you can provide the information requested in comment #14, please add it.
Comment 16 Christoph Feck 2018-04-26 22:13:28 UTC
To further investigate this issue, LabPlot developers need the information requested in comment #14. If you can provide it, or need help with finding that information, please add a comment.
Comment 17 urcindalo 2018-05-08 15:49:40 UTC
Created attachment 112508 [details]
test as described by OP

I'm not the original poster, but I tried what he suggests in the opening message with a recent compilation, a few minutes ago:
===
GIT update -->
   repository:               https://anongit.kde.org/labplot
   updating from commit:     9ba9b2636a94ec4c53dba369c922cb5b482c8333
   to commit:                ed4021f2e459ab4cbc8ff4ff5a9c7b2d5c908924
===

My findings:
1) When entering the parameter initial values, b fields turn red when entering 1. However, it is accepted: after pressing "Apply" and entering into Options again, b=1 is there and b fields are no longer red.

2) The reason for it to turn red is both b and a have default lower and upper limits equal to 0, whilst you are telling labplot their value is 1.

3) If you then perform the fitting, c=1 and both a and b get the same huge value: ~1.8e308

4) However, if you change the default 0 for both lower and upper limits (bug?) to none, set the starting value to 1 for the three parameters and perform the fitting, then you get distinct values for them. The fitting is not good, as you can see in the attached file, because probably you would need to set more realistic starting values, but I would say this is a normal behavior for such a function.
Comment 18 Stefan Gerlach 2018-05-08 19:38:06 UTC
I see at least two things that can be improved. The default ranges should be not 0..0  and "Apply" should not be possible with start values outside the given range. I will fix these soon.
Comment 19 Stefan Gerlach 2018-05-08 21:26:41 UTC
Both problems from comment #18 are fixed now. Please try again to check if there is still anything left to be improved.
Comment 20 Uwe Stöhr 2018-05-08 22:16:23 UTC
This bug is not fixed in the latest available LabPlot version.

> If you can provide the information requested in comment #14, please add it.
OK:

> Which values for a,b,c do you expect for best fitting you data?
See the screen shot I once attached with a correct fit:
https://bugsfiles.kde.org/attachment.cgi?id=108823

> the data columns are not restored when opening your project. What happens when  you select the x- and y-column correctly?
I don't know what you mean. I created the file using LabPlot so there must be another bug that columns are not restored.

Is there a new LabPlot build available to test?
Comment 21 Stefan Gerlach 2018-05-08 23:04:32 UTC
I uploaded a new Windows packages including the latest fixes at
https://theorie.physik.uni-konstanz.de/gerlach/labplot/labplot-2.5.0rc17-64bit-setup.exe
The parameter a is very sensitive. The fit works for me using start values closer to the result (b=2, a=.05, c=2).
Comment 22 Uwe Stöhr 2018-05-08 23:31:53 UTC
> The parameter a is very sensitive. The fit works for me using start values closer to the result (b=2, a=.05, c=2).

;-) Well, people use LabPlot because they don't yet the solution. using 1 for all 3 parameters is sensible because one can expect that the solution will have positive values and that b and c won't be very high.

However, I restarted now from scratch using RC17 and this recipe:

- create a new table with 2 columns
- fill the table columns with row numbers
- create a new custom fit
- use as fit function this one
exp(b*sin(a*x))+c
- set fit interval x:10 - 25
- set as start value for all parameters "1"

result: one doesn't get a fit curve

What is missing is that LabPlot does not draw the cureve for A=b=c=1 immediately, meaning even before starting the calculation of the fit. Other programs like MagicPlot do this since it helps a lot to see if the start parameters are sensible.

With RC17 the fitting result are now correct but as long as LabPlot does not draw the fit curve with the given start parameters, the problem is not fixed in my opinion.

Btw. with RC17 LabPlot does no longer show any fit curve. I'll open new bug reports now.
Comment 23 urcindalo 2018-05-09 11:32:19 UTC
Created attachment 112528 [details]
Test works OK in Linux

Well, I just sync'ed and compiled labplot again (Gentoo Linux). My results are identical to those shown by Uwe Stöhr in his MagicPlot Pro graph after knowing the parameters value range, as I show in the new attached file. So, I think there's really no issue here and labplot is now perfectly able to perform the suggested fitting.

I got a question for Uwe: does MagicPlot Pro find the "correct" parameter values when the starting values are 1,1,1? If it is, then it's almost magic, imho. The function is complexed and, in these cases, it is common for whatever fitting algorithm that, the further from a straight line the function is, the more prone to fail is the fit when the initial guess values are far away from the true ones.
Comment 24 Uwe Stöhr 2018-05-09 22:53:35 UTC
As I wrote in comment 22: "the fitting result are now correct but as long as LabPlot does not draw the fit curve with the given start parameters, the problem is not fixed in my opinion."

> does MagicPlot Pro find the "correct" parameter values when the starting values are 1,1,1? If it is, then it's almost magic

Well, it is called MagicPlot ;-) No, it does not find the optimal fit result with these parameters. it finds this:
https://bugsfiles.kde.org/attachment.cgi?id=108821

The reason this bug is now still open is that LabPlot does not give feedback about the fit curve. So if I use 1,1,1 as start parameter it is very important that LabPlot drawn immediately the function
exp(sin(x))+1
into the plot. Then the user gets a feeling if his start parameters are sensible.
Comment 25 Stefan Gerlach 2018-05-13 19:50:34 UTC
It looks like custom fitting in LabPlot now works. Supporting a preview of the fitting curve using the start values is a new feature which currently is breaking too many things (just before the release of 2.5). But i  will put this on the TODO list for the next release and hope to make it possible soon. Is it ok to contact you when we think this new feature is working?
Comment 26 Uwe Stöhr 2018-05-13 22:43:39 UTC
> I will put this on the TODO list for the next release

Many thanks. I opened bug 394223 as feature request.

> Is it ok to contact you when we think this new feature is working?

Sure. You have already my private mail address.