1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
|
\section{Errors in Numerical
Computations}\label{errors-in-numerical-computations}
In any numerical method, \textbf{error} is inevitable. Understanding
\textbf{what kinds of errors occur} and \textbf{why} is essential to
building reliable and accurate computations.
We mainly classify errors into two major types: - Truncation Error -
Round-off Error
\subsection{What is Error?}\label{what-is-error}
Let's remind ourselves what error is: \[
\text{Error} = \text{True Value} - \text{Approximate Value}
\] However, often the \textbf{true value} is unknown, so we focus on
\textbf{reducing} and \textbf{analyzing} different types of errors
instead of eliminating them completely. This can be done by using
relative error when using iterative methods and is calculated as
follows: \[
\text{Relative Error} = \frac{\text{Best} - \text{Second to best}}{Best}
\]
\subsection{Truncation Error}\label{truncation-error}
Truncation error occurs \textbf{when an infinite process is approximated
by a finite process}.\\
In simple terms, it happens \textbf{when you cut off or ``truncate''
part of the computation}. An example of this could be using a finite
number of terms from a Taylor Series expansion to approximate a
function.
Approximating $e^x$ by the first few terms of its Taylor series:
$e^x \approx 1 + x + \frac{x^2}{2!} + \frac{x^3}{3!}\$
The error comes from \textbf{neglecting} all the higher order terms
(\(\frac{x^4}{4!}, \frac{x^5}{5!}\), \ldots).
Truncation error occurs when using numerical methods such as
approximating and calculating derivatives and integrals. A
representation of the truncation error is show in the figure below.
Using our numerical methods we are left if some degree of error.
\begin{figure}
\centering
\includegraphics{figures/truncationError.png}
\caption{Representation of truncation error under a curve}
\end{figure}
In order to reduce truncation error there are a few things we can do: -
Include more terms (higher-order methods) - Decrease step sizes (e.g.,
smaller \(\Delta x\) in approximations) - Use better approximation
algorithms.
\subsection{Round-off Error}\label{round-off-error}
Round-off error is caused by \textbf{the limited precision} with which
computers represent numbers. Since computers cannot store an infinite
number of digits, \textbf{they round off} after a certain number of
decimal or binary places. For example, instead of representing π with
infinite decimal places it may be rounded off to approximately 16 digits
depending on number of bits and the representation of the bits.
In other words, round-off error happens because of how computers store
numbers. For a double-floating point, the number is stored using
64-bits. The more bits we use, the more precise of a number we can
store. However, it makes it costs us more memory making it more
computational expensive.
While individual round-off errors may seem negligible, their effects can
\textbf{accumulate over repeated computations}, leading to significant
inaccuracies. This is particularly problematic in operations such as
\textbf{subtracting two nearly equal numbers}, where \textbf{loss of
significance} can occur, severely reducing numerical precision and
amplifying the impact of round-off error.
\subsubsection{How to Reduce Round-off
Error:}\label{how-to-reduce-round-off-error}
To reduce round-off error, use higher-precision data types when storing
numerical values. Additionally, code and algorithms should be structured
to \textbf{avoid subtracting nearly equal numbers}, a common source of
significant error. Finally, employing \textbf{numerically stable
algorithms} is essential for minimizing the accumulation of round-off
errors during computation.
\subsection{Total Error}\label{total-error}
Truncation and round-off error are inversely proportional, meaning that
if we decrease one, the other increases. If we want to minimize total
error we must find the optimal point between step size and error.
\begin{figure}
\centering
\includegraphics{figures/totalError.png}
\caption{Total Error}
\end{figure}
|
