EEE 305 Lecture 12: Initial Value Problems

Initial value problems

Methods described here are for solving differential equations of the form:
𝑑𝑦
= 𝑓(𝑥, 𝑦)
𝑑𝑥
The methods in this lecture are all one-step methods and have the general format:
𝑦𝑖+1 = 𝑦𝑖 + 𝜙ℎ
where ϕ is called an increment function, and is used to extrapolate from an old value yi to a new
value yi+1.
This is in a general function of
𝑁𝑒𝑤 𝑣𝑎𝑙𝑢𝑒 = 𝑂𝑙𝑑 𝑣𝑎𝑙𝑢𝑒 + 𝑠𝑙𝑜𝑝𝑒 × 𝑠𝑡𝑒𝑝 𝑠𝑖𝑧𝑒
According to this equation, the slope estimate of φ is used to extrapolate from an old value y i to a
new value yi+1 over a distance h (Fig. 1). This formula can be applied step by step to compute out into
the future and, hence, trace out the trajectory of the solution.

Figure 1 Graphical depiction of a one-step method.

All one-step methods can be expressed in the general form, with the only difference being the
manner in which the slope is estimated. As in the falling parachutist problem, the simplest approach
is to use the differential equation to estimate the slope in the form of the first derivative at x i. In
other words, the slope at the beginning of the interval is taken as an approximation of the average
slope over the whole interval. This approach, called Euler’s method, is discussed in the next section.
This is followed by other one-step methods that employ alternative slope estimates that result in
more accurate predictions. All these techniques are generally called Runge-Kutta (RK) methods.
Euler’s Method
The first derivative provides a direct estimate of the slope at xi:

 f  xi , y i 
dy
dx xi

and the Euler method uses that estimate as the increment function:
𝜙 = 𝑓(𝑥𝑖 , 𝑦𝑖 )
where f (xi, yi) is the differential equation evaluated at xi and yi. This estimate can be substituted to
1

get
Page

𝑦𝑖+1 = 𝑦𝑖 + 𝑓 𝑥𝑖 , 𝑦𝑖 ℎ

Prepared BY
Shahadat Hussain Parvez
 EEE 305 Lecture 12: Initial Value Problems

This formula is referred to as Euler’s (or the Euler-Cauchy or the point-slope) method. A new value of
y is predicted using the slope (equal to the first derivative at the original value of x) to extrapolate
linearly over the step size h (Fig. 2).

Figure 2 Euler's method

Error analysis of Euler’s method
The numerical solution of ODEs involves two types of error:
– Truncation errors, caused by the nature of the techniques employed
– Roundoff errors, caused by the limited numbers of significant digits that can be retained
The total, or global truncation error can be further split into:
– local truncation error that results from an application method in question over a single
step, and
– propagated truncation error that results from the approximations produced during
previous steps.
The local truncation error for Euler’s method is O(h2) and proportional to the derivative of f(t,y)
while the global truncation error is O(h).
This means:
– The global error can be reduced by decreasing the step size, and
– Euler’s method will provide error-free predictions if the underlying function is linear.
Euler’s method is conditionally stable, depending on the size of h.
Higher order Taylor series method
One way to reduce the error of Euler’s method would be to include higher-order terms of the Taylor
series expansion in the solution. For example, including the second-order term

With a local truncation error of

Although the incorporation of higher-order terms is simple enough to implement for polynomials,
their inclusion is not so trivial when the ODE is more complicated. In particular, ODEs that are a
function of both the dependent and independent variable require chain-rule differentiation. For
example, the first derivative of f (x, y) is

The second derivative is

2
Page

Higher-order derivatives become increasingly more complicated.

Prepared BY
Shahadat Hussain Parvez
 EEE 305 Lecture 12: Initial Value Problems

Heun’s Method
One method to improve Euler’s method is to determine derivatives at the beginning and predicted
ending of the interval and average them:

Figure 3 Graphical depiction of Heun’s method. (a) Predictor and (b) corrector.

This process relies on making a prediction of the new value of y, then correcting it based on the
slope calculated at that new value.
From Eulers method
𝑦𝑖′ = 𝑓(𝑥𝑖 , 𝑦𝑖 )
Is used to extrapolate linearly to 𝑦𝑖+1 :
0
𝑦𝑖+1 = 𝑦𝑖 + 𝑓 𝑥𝑖 , 𝑦𝑖 ℎ
For the standard Euler method we would stop at this point. However, in Heun’s method the y 0i +1
calculated in last equation is not the final answer, but an intermediate prediction. This is why we
have distinguished it with a superscript 0. This equation is called a predictor equation. It provides an
estimate of yi+1 that allows the calculation of an estimated slope at the end of the interval:
′ 0
𝑦𝑖+1 = 𝑓(𝑥𝑖+1 , 𝑦𝑖+1 )
Thus, the two slopes can be combined to obtain an average slope for the interval:

This average slope is then used to extrapolate linearly from yi to yi+l using Euler’s method:

This is called corrector equation

This predictor-corrector approach can be iterated to convergence:

Error can be calculated using

3

𝑗 −1 𝑗
where 𝑦𝑖+1 𝑎𝑛𝑑 𝑦𝑖+1 are the result from the prior and the present iteration of the corrector,
Page

respectively.

Prepared BY
Shahadat Hussain Parvez
 EEE 305 Lecture 12: Initial Value Problems

Midpoint Method
Another improvement to Euler’s method is similar to Heun’s method, but predicts the slope at the
midpoint of an interval rather than at the end:

This method has a local truncation error of O(h3) and global error of O(h2)

Runge-Kutta (RK) Methods

Runge-Kutta (RK) methods achieve the accuracy of a Taylor series approach without requiring the
calculation of higher derivatives.
𝑦𝑖+1 = 𝑦𝑖 + 𝜙 𝑥𝑖 , 𝑦𝑖 , ℎ ℎ
For RK methods, the increment function ϕ can be generally written as:
  a1k1  a2 k2    an kn
where the a’s are constants and the k’s are
k1  f  x i , y i 
k 2  f xi  p1 h, y i  q11k1 h 
k 3  f xi  p 2 h, y i  q 21k1 h  q 22 k 2 h 

k n  f x i  p n 1 h, y i  q n 1,1 k1 h  q n 1, 2 k 2 h    q n 1, n 1 k n 1 h 
where the p’s and q’s are constants.
Various types of Runge-Kutta methods can be devised by employing different numbers of terms in
the increment function as specified by n. Note that the first-order RK method with n = 1 is, in fact,
Euler’s method. Once n is chosen, values for the a’s, p’s,and q’s are evaluated

Classical Fourth-Order Runge-Kutta Method

The most popular RK methods are fourth-order, and the most commonly used form is:

yi 1  yi 
1
k1  2k2  2k3  k4 h
6
where:
k1  f  x i , y i 
 1 1 
k 2  f  x i  h, y i  k1 h 
 2 2 
 1 1 
k 3  f  x i  h, y i  k 2 h 
 
4

2 2
k 4  f x i  h, y i  k 3 h 
Page

Prepared BY
Shahadat Hussain Parvez
 EEE 305 Lecture 12: Initial Value Problems

Figure 4 Graphical depiction of the slope estimates comprising the fourth-order RK method.
Example 25.7

5
Page

Prepared BY
Shahadat Hussain Parvez
 EEE 305 Lecture 12: Initial Value Problems

The graph below shows

Figure 5 Comparison of percent relative error versus computational effort for first- through fifth-order RK methods
Systems of Equations
Many practical problems require the solution of a system of equations:
dy1
 f 1  x , y1 , y 2 ,  , y n 
dt
dy 2
 f 2  x , y1 , y 2 ,  , y n 
dt

dy n
 f n  x , y1 , y 2 ,  , y n 
dt
The solution of such a system requires that n initial conditions be known at the starting value of x.
Solution Methods
Single-equation methods can be used to solve systems of ODE’s as well; for example, Euler’s method
can be used on systems of equations - the one-step method is applied for every equation at each
step before proceeding to the next step.
Fourth-order Runge-Kutta methods can also be used, but care must be taken in calculating the k’s.

1. Chapra examples 25.1, 25.5, 25.7

2. Chapra exercise 25.1- 25.5
Self study
 25.1.2 Algorithm for Euler’s Method [LAB]
 25.2.3 Computer Algorithms for Heun and Midpoint Methods [LAB]
 25.3.5 Computer Algorithms for Runge-Kutta Methods [LAB]
 25.3.1 Second-Order Runge-Kutta Methods
 25.3.2 Third-Order Runge-Kutta Methods
6

 25.3.4 Higher-Order Runge-Kutta Methods

Page

 25.4 SYSTEMS OF EQUATIONS

Prepared BY
Shahadat Hussain Parvez