Why Computers are Bad at Algebra | Infinite Series
Why Stack Overflow recommends it
Stack Overflow users link it when answering or commenting on questions about floating-point rounding, machine epsilon, and numerical inconsistencies.
Recommended by 3 people across 3 different months, 2017 to 2021.
What it is
A PBS Infinite Series episode, written and hosted by Kelsey Houston-Edwards, explaining how computers represent the real number line with floating-point numbers and why that leads to rounding errors. It uses the 1994 Intel Pentium recall as a motivating example.
What people said
Recommends loganjones16 · 2017-10 · Stack Overflow · on “as3 Number type - Logic issues with large numbers”
To be safe with integer math when using Numbers, I checked the AS3 documentation [LINK] for Number and it states: The Number class can be used to represent integer values well beyond the valid range of the int and uint data types. The Number data type can use up to 53 bits to represent integer values, compared to the 32 bits available to int and uint. A 53 bit integer gets you to 2^53 - 1, if I'm not mistaken which is 9007199254740991, or about 9 quadrillion. The other 11 bits that help make up the 64 bit Number are used in the exponent. The number used in the question is about 64.3 quadrillion. Going past that point (9 quadrillion) requires more bits for the significant number portion (the
Recommends S0AndS0 · 2019-04 · Stack Overflow · on “Matrix multiplication instability with float32 numbers in numpy on python3”
`1`: might be a good idea to review how `computers do math` [this video] ... TLDW it's all in binary scientific notation; there will be rounding. `2`: I don't believe you can with Python alone, and there are so many processor types (all with _little qwerks_), that it may not be worth your time to sort out in a language that does provide that level of control... Eg. ARM has had soft and hard floating point CPUs, each generation has been slightly different, and that's just one architecture _family_; which is _messy_ without changing precision from 64 to 32 too.
Recommends DwightFromTheOffice · 2021-06 · Stack Overflow · on “Numerical inconsistency between loop and builtin function”
machine epsilon is only a function of the bit representation, the difference comes from a 'random' factor (for which there is only an upper bound). It is a field on its own, and there are entire books written about it, so it is hard to give just one article for it. I recommend googling 'machine precision', 'machine epsilon', 'numerical analysis' and searching for articles that can help you. You can even browse youtube for information, for ex: [this video]
Cites void · 2017-07 · Stack Overflow · on “I can't understand the float output”
Consider this fraction. 1/2 What will be it's output? 0.5 simple right. Now for 1/3 what will be the output? 0.3 seems okay because 3 x 0.3 = 0.9 More accurately, 3 x 0.33 = 0.99 or even more 3 x 0.333 = 0.999 Hell if you do the same for say 10 decimal points you still get, 3 x 0.33333333333 = 0.99999999999 As you see we are getting more and more nearer to the value 1 but never are we exactly equal . So that means we are getting more and more accurate with such high precisions. However you won't normally require such precisions for your simple programs. See what Python docs [LINK] say about it, 0.999999999 , That is more digits than most people find useful, so Python keeps the number of digi