Bug 103256 - Calculator gives non-integer answer when operating on integer values, precision wrong
Summary: Calculator gives non-integer answer when operating on integer values, precisi...
Status: RESOLVED DUPLICATE of bug 97561
Alias: None
Product: kcalc
Classification: Applications
Component: general (show other bugs)
Version: 1.8
Platform: unspecified Solaris
: NOR normal
Target Milestone: ---
Assignee: Klaus Niederkrüger
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2005-04-05 02:16 UTC by Aaron Williams
Modified: 2005-07-18 21:31 UTC (History)
0 users

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Aaron Williams 2005-04-05 02:16:22 UTC
Version:           1.8 (using KDE 3.4.0, compiled sources)
Compiler:          gcc version 3.3.5
OS:                SunOS (sun4u) release 5.8

Input the following
16*16+4

Answer should be 260 but I get 260.0000000000000568434188608080148696899 which displays as
260.0000000000001

Note that this is on Solaris running on an Ultrasparc processor.  This is sort of like the old Windows 3.1 calculator bug.
Comment 1 Aaron Williams 2005-04-05 02:18:02 UTC
This problem does not show up on Linux running on x86 (SuSE 9.2, KDE 3.4.0)
Comment 2 Adrien Cordonnier 2005-04-25 10:33:13 UTC
This problem does not show on my system but I have a similar one:

849 + 2 + 4 = 855 mod 19 = 2.220446049250313080847263336181640625e-16
Should be 0.

Running on x86 Pentium M, Mdk 10.2 Cooker, KDE 3.4.0.
Comment 3 Aaron Williams 2005-06-20 22:23:30 UTC
I might add that KCalc in KDE 3.3.2 gives the correct answer.
Comment 4 Aaron Williams 2005-06-20 22:32:43 UTC
One difference I might add on the Ultrasparc Solaris platform is that there is no support for long doubles, so the supported precision is less.  Perhaps compiling on x86 without long double support could recreate the problem I am seeing?
Comment 5 Aaron Williams 2005-06-20 22:59:08 UTC
The bug is actually worse... i.e. 2+2=4.000000000000000888178419700125232338905 but it displays as 2+2=4.000000000000001


At least 1+1 displays properly as 2, but if I copy and paste it I get 2.000000000000000444089209850062616169453

-Aaron
Comment 6 Thiago Macieira 2005-07-14 05:55:49 UTC

*** This bug has been marked as a duplicate of 34765 ***
Comment 7 Aaron Williams 2005-07-14 21:58:58 UTC
This is NOT a duplicate of 34765.  I don't want more precision, I want the correct result to be displayed.  Simple integer math should yield a simple integer answer, especially if all I'm doing is simple addition and/or multiplication.  KCalc is giving the wrong answer.  This might be due to the fact that there is a bug in the rounding code since Solaris does not support long doubles but only doubles.  This is not a wish list but a bug.
Comment 8 Thiago Macieira 2005-07-15 03:24:03 UTC
No, it means you just don't understand how floating point in computers work.  There is no exact calculation, unless you use an arbitrary-precision library like libgmp or bc.

The fact is the value KCalc is showing is correct, given the limitations of the internal representation.

*** This bug has been marked as a duplicate of 34765 ***
Comment 9 Thiago Macieira 2005-07-15 04:28:45 UTC
Reopening to mark as duplicate of another.
Comment 10 Thiago Macieira 2005-07-15 04:29:46 UTC

*** This bug has been marked as a duplicate of 97561 ***
Comment 11 Aaron Williams 2005-07-18 21:08:56 UTC
IEEE Floating point should be able to handle pure integer math without any roundoff errors.

I wrote some extremely simple floating point C code using the lowly printf to print the result to 20 digits of precision and I always get the correct result unlike kcalc.

I.e., in my sample I use (16*16) + 4 as an example, but in the following C program I get the correct result.

#include <stdio.h>
#include <math.h>

int main(int argc, char *argv[])
{
        double a=16.0;
        double b=16.0;
        double c=4.0;
        double d;

        d = (a * b) + c;

        printf("%.20g\n", d);
    return 0;
}

Prints out 260

kcalc gives 260.0000000000000568434188608080148696899 or 260.00000000000001 is displayed.
Comment 12 Thiago Macieira 2005-07-18 21:31:47 UTC
Stop reopening.

I've already marked this bug as a duplicate of another dealing with the same kind of rounding/non-rounding issues. We don't need three bugs open for the same reason (you guys don't agree with me with the arbitrary-precision library).

*** This bug has been marked as a duplicate of 97561 ***