Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Can this be true? Does the calculator have any irrational number? What if I enter PI * 2? Infinite precision, wouldn't that mean running until your RAM is full? Or would it simply work, because no irrational number is calculated to infinite (as many as the device could) digits and is therefore finite and can easily be multiplied?

Or another even simpler case: If division is infinite precision, and I enter (2/3) * X, does the calculator internally work with fractions? Otherwise it would have infinite digits to compute.



For PI, calc.exe, in scientific mode, returns 3.1415926535897932384626433832795 which is 32 digits of precision (33 chars if you include the decimal pt).


For 2/3, it does indeed work with fractions internally. For pi, it uses some floating point representation I think, you get about 50 digits.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: