Bug 131482 - Wrong result with large number in hexa on amd64
Summary: Wrong result with large number in hexa on amd64
Status: RESOLVED FIXED
Alias: None
Product: kcalc
Classification: Applications
Component: general (show other bugs)
Version: unspecified
Platform: unspecified Linux
: NOR normal
Target Milestone: ---
Assignee: Klaus Niederkrüger
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2006-07-28 19:40 UTC by Thomas Capricelli
Modified: 2006-11-10 03:05 UTC (History)
0 users

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Thomas Capricelli 2006-07-28 19:40:51 UTC
Version:           2.0.3 (using KDE 3.5.3, compiled sources)
Compiler:          Target: x86_64-pc-linux-gnu
OS:                Linux (x86_64) release 2.6.16.14

All in hexa mode.
* do "FF * FF" (or FF and then 'x^2'). You get FE01, which is ok
* FFF^2 gives : FFE001, which is good as well
* now do FFFFFFFF^2 (this is a 32bit with all bit sets). You get the wrong answer FFFFFFFC00000001, it should be FFFFFFFE00000001.

proof 
0xffffffff is (2^32 - 1)
so 0xffffffff ^ 2 is 2^64 - 2 * 2^32 + 1

moreover, 'bc' gives the same result.

i was using kcalc to check unit tests for another project. took me some times to figure out the unit test was ok, and kcalc was wrong :)

Could be related to bug #128684, but not sure.
Comment 1 Klaus Niederkrüger 2006-09-17 13:25:34 UTC
>i was using kcalc to check unit tests for another project. took me some times >to figure out the unit test was ok, and kcalc was wrong :) 

Sorry to hear this. Kcalc has many problems with Hex-numbers, since I changed to GMP and use arbitrary precision numbers (because in Hex I still use the old standard C-type numbers). I believe I fixed now this bug. Maybe you could run the unit-test again, but now against KCalc? ;-)

Thanks for reporting the bug.

Klaus
Comment 2 Klaus Niederkrüger 2006-09-19 11:45:57 UTC
Seems to be fixed.
Comment 3 Thomas Capricelli 2006-09-19 13:17:01 UTC
not here. I've updated/installed/tested from my kde-3.5 branch checkout, and 
the bug is still here... but maybe you've fixed it somewhere else ? 

URL: https://orzel@svn.kde.org/home/kde/branches/KDE/3.5/kdeutils/kcalc


Thomas

On Tuesday 19 September 2006 11:45, Klaus NiederkrÃŒger wrote:
[bugs.kde.org quoted mail]
Comment 4 Josh Metzler 2006-11-10 03:05:35 UTC
Works for me in Debian, kde 3.5.5