1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
|
# Errors in Numerical Computations
^d57531
In any numerical method, **error** is inevitable. Understanding **what kinds of errors occur** and **why** is essential to building reliable and accurate computations.
We mainly classify errors into two major types:
- Truncation Error
- Round-off Error
## What is Error?
Let's remind ourselves what error is:
$$
\text{Error} = \text{True Value} - \text{Approximate Value}
$$
However, often the **true value** is unknown, so we focus on **reducing** and **analyzing** different types of errors instead of eliminating them completely. This can be done by using relative error when using iterative methods and is calculated as follows:
$$
\text{Relative Error} = \frac{\text{Best} - \text{Second to best}}{Best}
$$
## Truncation Error
Truncation error occurs **when an infinite process is approximated by a finite process**.
In simple terms, it happens **when you cut off or "truncate" part of the computation**. An example of this could be using a finite number of terms from a Taylor Series expansion to approximate a function.
Approximating $e^x$ by the first few terms of its Taylor series:
$$e^x \approx 1 + x + \frac{x^2}{2!} + \frac{x^3}{3!}$$
The error comes from **neglecting** all the higher order terms ($\frac{x^4}{4!}, \frac{x^5}{5!}$, ...).
Truncation error occurs when using numerical methods such as approximating and calculating derivatives and integrals. A representation of the truncation error is show in the figure below. Using our numerical methods we are left if some degree of error.
In order to reduce truncation error there are a few things we can do:
- Include more terms (higher-order methods)
- Decrease step sizes (e.g., smaller $\Delta x$ in approximations)
- Use better approximation algorithms.
## Round-off Error
Round-off error is caused by **the limited precision** with which computers represent numbers. Since computers cannot store an infinite number of digits, **they round off** after a certain number of decimal or binary places. For example, instead of representing π with infinite decimal places it may be rounded off to approximately 16 digits depending on number of bits and the representation of the bits.
In other words, round-off error happens because of how computers store numbers. For a double-floating point, the number is stored using 64-bits. The more bits we use, the more precise of a number we can store. However, it makes it costs us more memory making it more computational expensive.
While individual round-off errors may seem negligible, their effects can **accumulate over repeated computations**, leading to significant inaccuracies. This is particularly problematic in operations such as **subtracting two nearly equal numbers**, where **loss of significance** can occur, severely reducing numerical precision and amplifying the impact of round-off error.
### How to Reduce Round-off Error:
To reduce round-off error, use higher-precision data types when storing numerical values. Additionally, code and algorithms should be structured to **avoid subtracting nearly equal numbers**, a common source of significant error. Finally, employing **numerically stable algorithms** is essential for minimizing the accumulation of round-off errors during computation.
## Total Error
Truncation and round-off error are inversely proportional, meaning that if we decrease one, the other increases. If we want to minimize total error we must find the optimal point between step size and error.
|
