TechNote (Archived)

Flash returns erroneous results for certain mathematical calculations

Issue

---

Flash returns erroneous results for certain mathematical calculations. For example, the action script Trace (1.01 - 1) will output the value 0.0100000000000002.

Reason

---

This is not a bug in Flash. This is an example of roundoff error, a fundamental issue in floating-point arithmetic. The book "Numerical Recipes in C: The Art of Scientific Computing" (Cambridge University Press) contains an excellent discussion of roundoff error: "Arithmetic among numbers in floating-point representation is not exact, even if the operands happen to be exactly represented ... The smallest (in magnitude) floating-point number which, when added to the floating-point number 1.0, produces a floating-point result different from 1.0 is termed the machine accuracy e(m). A typical computer with B = 2 and a 32-bit wordlength has e(m) around 3 x 10^-8 ... Pretty much any arithmetic operation among floating numbers should be thought of as introducing an additional fractional error of at least e(m). This type of error is called roundoff error ... Roundoff error is a characteristic of computer hardware ... Roundoff errors accumulate with increasing amounts of calculation." (p. 28-29) If you are interested in reading further, this book is available online at http://www.ulib.org/html/index.html.

Roundoff error is a fact of life in any computer program that uses floating-point arithmetic. Consider these results taken from some tools often used for Web and E-Commerce applications: Perl, VBScript and JavaScript.

JavaScript:

 <script language = "JavaScript">  alert(1.01-1)
		

	</script> 

The output of the HTML file is 0.010000000000000009 (Internet Explorer 4.0 SP 1, Win NT 4.0), 0.010000000000000009 (Netscape Communicator 4.5, Win NT 4.0)

VBScript:

 <script language = "VBScript">  alert(1.001-1)
		

	</script> 

The output of the HTML file is 9.9999999999989E-04. (Internet Explorer4.0 SP 1, Win NT 4.0)

Perl: The output of the command line Perl -e "print 1.001 - 1" is 0.00099999999999989 (ActivePerl Build 518, Win NT 4.0)

Note: Due to differences in algorithms, roundoff error may manifest itself in different ways.

Solution

---

This is a fact of life in applications that use floating point arithmetic.

Search Support

---