Lecture 2: Errors

Decimal Machine Numbers

Definition 1. The machine numbers represented in the normalized decimal floating-point form
0.d1 d2 . . . dk 10n ,

1 d1 9,

0 di 9 for i = 2, . . . , k,

are called k-digit decimal machine numbers. Consider a number of the normalized decimal form
y = s(0.d1 d2 . . . dk dk+1 . . .) 10n

d1 6= 0.

The floating-point form of y, denoted by f l(y), can be obtained by

1. k-digit chopping: f lc (y) = s(0.d1 d2 . . . dk ) 10n ;
2. k-digit rounding: f lr (y) = s(0.1 2 . . . k ) 10n . If dk+1 5, we round up by adding 1 to dk .
If dk+1 < 5, we round down by chopping off all but the first k digits.
Example 1. Determine the 5-digit chopping and rounding values of = 3.14159265 . . ..
Solution.
f lc () = 0.31415 101 = 3.1415,

f lr () = 0.31416 101 = 3.1416.

Absolute and relative errors

Definition 2. Let p be an approximation to p R. The absolute error is ea (p, p ) = |p p |, and

|
the relative error is er (p, p ) = |pp
provided that p 6= 0.
|p|
Remark: In practice, if p is not known and |p p | is known, we use er (p, p )

|pp |
|p | .

Definition 3. Let a (p, p ) > 0 such that |ea (p, p )| a (p, p ) is the (upper) bound for the absolute
error. We write p = p e for |e| a (p, p ) or simply p = p a (p, p ). Likewise, r (p, p ) > 0
such that |er (p, p )| r (p, p ) is the (upper) bound for the relative error, written as p = p (1 ) for
|| r (p, p ) or simply p = p (1 r (p, p )).
Remark: To simplify the notation without confusion of the context, we simply write a , r , ea , er .
Definition 4. The number p is said to approximate p to t significant digits (or figure) if t is the
largest nonnegative integer for which er (p, p ) 5 10t .
Remark: this definition is consistent with the number of decimal digits to be accurate.
Example 2. Let p = 0.12345 . . . and p = 0.123. Since er (p, p ) =
significant digits.

0.45...103
0.12345...

5 103 , p has three

Definition 5. (Error bound)

1. Bound of absolute error |ea (p, p )| a (p, p ), or p = p + e where |e| a (p, p ), or p = p + a .
2. Bound of relative error |er (p, p )| r (p, p ), or p = p (1 + ) where || r (p, p ), or p =
p (1 + r ).
1

Example 3. Find an error bound for k-digit chopping.

Solution. Let y = (0.d1 d2 . . . dk dk+1 . . .) 10n be normalized decimal representation. Then
f lc (y) = (0.d1 d2 . . . dk ) 10n .
Therefore



 y f lc (y)  (0.dk+1 . . .) 10nk
=

er = 

y
(0.d1 . . .) 10n

If dk+1 < 5, then

0.5
10k = 5 10k
0.1
which implies that k-digit chopping will keep k significant digits. If dk+1 5, then
er

er

1
10k = 10k+1 < 5 10k+1
0.1

which implies that k-digit chopping will keep k 1 significant digits.

Remark: Similarly, we have er (y, f lr (y)) 0.5 10k+1 which implies that the k-digit rounding
keeps k significant digits.

Finite-digit arithmetic

Let , , and
represent machine addition, subtraction, multiplication and division.
xy = f l(f l(x)+f l(y)) x y = f l(f l(x)f l(y)) xy = f l(f l(x)f l(y)) x
y = f l(f l(x)f l(y)).
Example 4. Let x =

2
3

and y = 13 , find 3-digit chopping and rounding values of x y.

Solution. 3-digit chopping:

2
1
x y = f lc (f lc ( ) + f lc ( )) = f lc (0.666 + 0.333) = f lc (0.999) = 0.999.
3
3
In this case, ea = 0.100 102 and er = 0.100 102 .
3-digit rounding:
x y = f lr (0.667 + 0.333) = f lr (1.000) = 1.00 = 0.100 101 .
But ea = er = 0.

Ways to reduce errors

I. Avoid subtraction of 2 nearly equal numbers. Reason: it causes cancelation of significant

digits. Given 2 nearly equal numbers x > y of k-digit representation:
f l(x) = 0.d1 d2 . . . dp p+1 . . . k 10n ,

f l(y) = 0.d1 d2 . . . dp p+1 . . . k 10n .

Then
f l(f l(x) f l(y)) = 0.p+1 p+2 . . . k 10np = (0.p+1 . . . k 0.p+1 . . . k ) 10np
x y has at most k p significant digits. In most machines, x y will be assigned k-significant digits
with last p digits either 0 or randomly assigned.
2