Documente Academic
Documente Profesional
Documente Cultură
Numerical Analysis/Methods
What is numerical analysis/method?
Analysis and design of algorithms for numerically solving mathematical problems in science and engineering
Numerically Solving x =2
2
Other Examples
Science and Engineering convert a physical phenomenon into a mathematical model. Usually this leads to problems with no analytical solution:
Course Overview
Approximation and errors Solving nonlinear equations Solving sets of equations
Simulation for prototypingODE solver, PDE solver, optimization, numerical integration, interpolation, finite element method
Circuit design Mechanical design CAD/CAM
TYPES OF ERRORS
Truncation error (finite speed and time) - An example:
Round-off error (finite word length): All computing devices represent numbers with some imprecision, except for integers. Human errors:
Mathematical equation/model. Computing tools/machines. Error in original data. Propagated error.
MEASURE OF ERRORS
The accuracy of any computation is always of great importance. Two ways to express the size of error in a computed result:
Absolute error Relative error
Floating-Point Arithmetic
A computer stores numbers as floating-point quantities that resemble scientific notation.
For example 13.524 is often displayed as .13524E2 Also -0.0442=-.442E-1 normalized (that is leading digit is made nonzero)
For storing floating-point numbers the IEEE standard is the most common.
There are three level of precision and these are the number of bits used for Length Sign Mantissa Exponent Range mantissa and exponent.
Single Double Extended 32 64 80 1 1 1 23 52 64 8 11 15 1038 10308 104931
Consequence of normalization is that the absolute value of m (mantissa) is limited. That is (1/b)m<1 where b is the
A Hypothetical Example
MATLAB
Matlab is a software package that allows you to program the mathematics of an algorithm without getting too bogged down in the details of data structures, pointers, and reinventing the wheel. It also includes graphing capabilities, and there are numerous packages available for all kinds of functions, enabling relatively high-level
MATLAB
The basic Matlab data structure is a matrix; a scalar is a 1x1 matrix, a vector is an nx1 matrix. Vectors and matrices in Matlab are indexed starting from 1; and not from 0; as is more common in modern programming languages. Moreover, the last index of a vector is denoted by the special symbol end.
Programming in Matlab
If you are going to do any serious programming in Matlab, you should keep your commands in a file. Matlab loads commands from `.m' files. If you have the following in a file called myfunc.m:
function [y1,y2] = myfunc(x1,x2) % comments start with a `%' % this function is useless, except as an example of functions. % input: % x1 a number % x2 another number % output: % y1 some output % y2 some outpu y1 = cos(x1) .* sin(x2); y2 = norm(y1);
Programming in Matlab
then you can call this function from Matlab, as follows:
> myfunc(2,3) ans = -0.058727 > [a,b] = myfunc(2,3) a = -0.058727 b = 0.058727 > [a,b] = myfunc([1 2 3 4],[1 2 3 4]) a= 0.45465 -0.37840 -0.13971 0.49468 b = 0.78366
%an `irregular' for loop for i=[1 2 3 5 8 13 21 34] fsm = fsm + i; end
Disadvantages:
The estimate of root may be better at an earlier iteration than at later ones. It requires the values of a and b. The convergence of interval halving is very slow. Multiple zeros between a and b can cause
Example (Bisection)
Let f(x)=x2-1. Find its zero in [0, 1.5] Of course, we know f (x) has a root at x0=1. Let us find it using the Bisection Method: xa= 0, xb=1.5, n =1 x1=(0+1.5)/2=0.75 x2=(0.75+1.5)/2=1.125 x3=(0.75+1.125)/2=0.9375 x4=(0.9375+1.125)/2=1.03125 ...
Example (Bisection)
fx='3*x+sin(x)-exp(x)' fx = 3*x+sin(x)-exp(x) bisec(fx,0,1,13) 1.0000 0 1.0000 0.5000 0.3307 2.0000 0 0.5000 0.2500 -0.2866 3.0000 0.2500 0.5000 0.3750 0.0363 4.0000 0.2500 0.3750 0.3125 -0.1219 5.0000 0.3125 0.3750 0.3438 -0.0420 6.0000 0.3438 0.3750 0.3594 -0.0026 7.0000 0.3594 0.3750 0.3672 0.0169 8.0000 0.3594 0.3672 0.3633 0.0071 9.0000 0.3594 0.3633 0.3613 0.0023
function rtn=bisec(fx,xa,xb,n) x=xa;fa=eval(fx); x=xb;fb=eval(fx); for i=1:n xc=(xa+xb)/2;x=xc;fc=e val(fx); X=[i,xa,xb,xc,fc]; disp(X) if fc*fa<0 xb=xc; else xa=xc; end end
Disadvantage:
The method may fail if the function is far from linear and near the root the successive iterates can fly off to points far from the root.
r x0 x1 x2
False Position
To avoid the mentioned pathology is to ensure that the root is bracketed between the successive pairs. When this is done, this method is known as false position. This method is similar to bisection except that the next iterate is taken at the intersection of a line between the pair of x-values and x-axis rather than at the midpoint. This yields faster convergence, but with a more complicated algorithms.
Newtons Method
Like the previous ones, this method is also based on a linear approximation of the function, but does so using a tangent to the curve. tan =f(b)=f(b)/(b-x1)x1=b-f(b)/f(b)
Disadvantage
Needs f(x).
nd
Example
x=g3(x)=(x2-3)/2
Start with x0=4 x1= 6.5 X2=19.625 x3=191.070 Seems diverging
Remark: If the above conditions are not satisfied, the iteration scheme might still converge as the above theorem only gives sufficient conditions
Converge?Diverge
Converge Diverge
Example
Solve f(x) = 0 by rewriting as x = g(x) and iterating xn+1 = g(xn) f(x) = x3 sin(x) = 0 (has 3 roots)
Example
Find x which satisfies x = ex. Can re-write this as log x = x
Summary of Methods
Bisection Method
Condition: Continuous function f (x) in [a, b] satisfying f (a) f (b) < 0. Convergence: Slow but sure. Linear.
Newton Method
Condition: Existence of nonzero f '(x) Convergence: Fast (quadratic).
Secant Method
Condition: Existence of nonzero f (xn+1) f (xn) Convergence: Fast (quadratic).
Fixed-point Method
Muellers Method
Multiple Roots
A multiple root corresponds to a point where a function is tangent to the x axis. For example
f(x)=(x-3)(x-1)(x-1)
1 3
f(x)=(x-3)(x-1)(x-1)(x-1)
1 3
xi+1=xi- (f(xi).f(xi))/([f(xi)]2-
Example
f(x)=(x-3)(x-1)(x-1)
Two matrices of the same size may be added or subtracted. Let A=[aij] and B=[bij], then C=A+B=[aij+ bij] = [cij].
5 12 1 C = A+ B = 2 4 15
3 2 9 D = A B = 6 8 9
Augmented Form
4 2 1 x1 15 4x1 - 2x 2 + x 3 = 15 3 1 4 x = 8 - 3x - x + 4x = 8 1 2 3 2 1 1 3 x3 13 x1 - x 2 + 3x 3 = 13
4 2 1 15 3 1 4 8 Augmented Form : 1 1 3 13
Elimination Methods
Aim: try to convert an augmented form matrix to an upper triangular form by using elementary row operations. Elementary row operations
We may multiply any row of the augmented coefficient matrix by a constant. We can add a multiple of one row to a multiple of any other row.
Example
1 15 4 2 1 15 4 2 3 1 4 8 3R + 4 R 0 10 19 77 1 2 1 1 3 13 R1 4 R3 0 2 11 37
1 15 4 2 0 10 19 77 R2 + 5 R3 0 0 36 108
Gaussian Elimination
Gaussian elimination is an elimination method that cares for round off errors and ill-conditioning or division by zero. A useful strategy to avoid zero divisors is to rearrange the equations so as to put the coefficient of largest magnitude on the diagonal at each step. This is called pivoting.
Example
Example
Gauss-Jordan Method
It is a variation of Gauss elimination. In it, the elements above the diagonal are made zero at the same time that zeros are created below the diagonal. The diagonal elements are normalized (made ones) at the same time that the reduction is performed. Thus, the elimination step results in an identity matrix rather than a triangular matrix. Consequently, it is not necessary to employ back substitution to obtain the solution. Pivoting is used to preserve accuracy.
Example
Pathological Systems
An arbitrary set of equations may not have such a guaranteed solution. There are several such possible situations, which is called pathological. In each case there is no unique solution to the set of equations.
Example
Singular Matrices
A matrix that does not have an inverse is defined as singular. Properties of a singular matrix: A singular matrix has
A determinant of zero Has rows that are linearly dependent Has columns that are linearly dependent Has no unique solution The rank is less than n, number of rows
Ill-Conditioned Systems
A system whose coefficient matrix is nearly singular is ill-conditioned. When a system is ill conditioned, the solution is very sensitive to changes in the right-hand side vector. It is also sensitive to small changes in the coefficients. For an ill-conditioned system, small changes in the input make large changes in the output.
Example
1.01x+0.99y=2 0.99x+1.01y=2 solution : x=1, y=1 If b is [2.02 1.98]T solution becomes x=2, y=0. If b is [1.98 2.02]T we would have x=0, y=2. It must be apparent that while solving an ill-conditioned system, the
Interpolation
Let the cubic polynomial passing through these be such that ax3+bx2+cx+d and
a(3.2)3+b(3.2)2+c(3.2)+d=22.0 a(2.7)3+b(2.7)2+c(2.7)+d=17.8 a(1.0)3+b(1.0)2+c(1.0)+d=14.2 a(4.8)3+b(4.8)2+c(4.8)+d=38.3
Interpolation
These equations lead us to a set of linear equations with four unknowns and four equations. But we know how to solve it by: Gaussian elimination. This gives us the solution as
a=0.5275 b=6.4952 c=-16.1177 d=24.3499 and f(3.0)=20.212 with these values.
Interpolation
Now, assume that fifth data pair came out as above. For this new set we should form a quadratic (fourth degree) polynomial and solve for f(3.0) once more.
x 3.2 2.7 1.0 4.8 5.6 f(x) 22.0 17.8 14.2 38.3 51.7
Lagrangian Polynomials
The simplest way to exhibit the existence of a polynomial for interpolation with unevenly spaced data. Suppose at hand we have:
i 0 1 2 3 x x0 x1 x2 x3 f(x) f0 f1 f2 f3
Lagrangian Polynomials
( x x1 )( x x2 )( x x3 ) ( x x0 )( x x2 )( x x3 ) P3 ( x) = f0 + f1 ( x0 x1 )( x0 x2 )( x0 x3 ) ( x1 x0 )( x1 x2 )( x1 x3 ) ( x x0 )( x x1 )( x x3 ) ( x x0 )( x x1 )( x x2 ) + f2 + f3 ( x2 x0 )( x2 x1 )( x2 x3 ) ( x3 x0 )( x3 x1 )( x3 x2 )
Lagrangian Polynomials
The lagrangian for other degrees of interpolating polynomials employs this same pattern of forming a sum of polynomials all of the desired degree; it will have n-1 terms when the degree is n. Try to see that this Lagrangian polynomial passes through each point. Example: Solve the previous example by Lagrangian and compute f(3.0)
See that it is the same result.
Lagrangian PolynomialsExample
Lagrangian Polynomials
Extrapolation (applying the interpolating outside the range of x values employed to construct it) will have larger errors than for interpolation. Caution: never fit a polynomial of a degree higher than 4 or 5 to a set of points. If you need to fit to a set of more than 6 points, be sure to break up the set into subsets and fit separate polynomials to these. Better way may be using spline curves.