Numerical Simulation of Stochastic

Differential Equations: Lecture 2, Part 1

Des Higham
Department of Mathematics
University of Strathclyde
Montreal, Feb. 2006 p.1/25
Lecture 2, Part 1: EulerMaruyama
Denition of EulerMaruyama Method
Weak Convergence
Strong Convergence
Linear Stability
Montreal, Feb. 2006 p.2/25
Recap: SDE
Given functions f and g, the stochastic process X(t) is a
soluton of the SDE
dX(t) = f(X(t))dt + g(X(t))dW(t)
if X(t) solves the integral equation
X(t) X(0) =
_
t
0
f(X(s)) ds +
_
t
0
g(X(s)) dW(s)
Discretize the interval [0, T]: let t = T/N and t
n
= nt
Compute X
n
X(t
n
)
Initial value X
0
is given
Montreal, Feb. 2006 p.3/25
EulerMaruyama
Exact solution:
X(t
n+1
) = X(t
n
) +
_
t
n+1
t
n
f(X(s)) ds +
_
t
n+1
t
n
g(X(s)) dW(s)
EulerMaruyama:
X
n+1
= X
n
+ tf(X
n
) + W
n
g(X
n
)
where W
n
= W(t
n+1
) W(t
n
)
(Left endpoint Riemann sums)
In MATLAB, W
n
becomes sqrt(Dt)*randn
Montreal, Feb. 2006 p.4/25
f(x) = x and g(x) = x, = 2, = 0.1, X(0) = 1
Solution: X(t) = X(0)e
(
1
2

2
)t+W(t)
Disc. Brownian path with t = 2
8
, E-M with t = 4t:
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
0
1
2
3
4
5
6
t
X
|X
N
X(T)| = 0.69
Reducing to t = 2t gives |X
N
X(T)| = 0.16
Reducing to t = t gives |X
N
X(T)| = 0.08
Montreal, Feb. 2006 p.5/25
Convergence?
X
n
and X(t
n
) are random variables at each t
n
In what sense does |X
n
X(t
n
)| 0 as t 0?
There are many, non-equivalent, denitions of convergence
for sequences of random variables
The two most common and useful concepts in numerical
SDEs are
Weak convergence: error of the mean
Strong convergence: mean of the error
Montreal, Feb. 2006 p.6/25
Weak Convergence
Weak convergence: capture the average behaviour
Given a function , the weak error is
e
weak
t
:= sup
0t
n
T
|E[(X
n
)] E[(X(t
n
))]|
from e.g. set of polynomials of degree at most k
Converges weakly if e
weak
t
0, as t 0
Weak order p if e
weak
t
Kt
p
, for all 0 < t t

In practice we estimate E[(X

n
)] by Monte Carlo simulation
over many paths 1/

M sampling error
Montreal, Feb. 2006 p.7/25
f(x) = x and g(x) = x, = 2, = 0.1, X(0) = 1
Solution has E[X(t)] = e
t
Measure weak endpoint error |a
M
e
T
| over M = 10
5
discretized Brownian paths. Try t = 2
5
, 2
6
, 2
7
, 2
8
, 2
9
10
3
10
2
10
1
10
2
10
1
10
0
t
|

E
[
X
(
T
)
]

S
a
m
p
l
e

a
v
e
r
a
g
e

o
f

X
N

|
Least squares t: power is 1.011
(Condence intervals smaller than graphics symbols)
Suggests weak order p = 1
Montreal, Feb. 2006 p.8/25
Weak EulerMaruyama
X
n+1
= X
n
+ tf(X
n
) +

W
n
g(X
n
)
where P
_

W
n
=

t
_
=
1
2
= P
_

W
n
=

t
_
E.g. use sqrt(Dt)*sign(randn)
or sqrt(Dt)*sign(rand-0.5)
10
3
10
2
10
1
10
2
10
1
10
0
t
|

E
[
X
(
T
)
]

S
a
m
p
l
e

a
v
e
r
a
g
e

o
f

X
N

|
Least squares t: power is 1.03
Montreal, Feb. 2006 p.9/25
Weak EulerMaruyama
Generally, EM and weak EM have weak order p = 1 on
appropriate SDEs for () with polynomial growth
Can prove via Feynman-Kac formula that relates SDEs to
PDEs
Montreal, Feb. 2006 p.10/25
Strong Convergence
Strong convergence: follow paths accurately
Strong error is
e
strong
t
:= sup
0t
n
T
E[|X
n
X(t
n
)|]
Converges strongly if e
strong
t
0, as t 0
Strong order p if e
strong
t
Kt
p
, for all 0 < t t

Montreal, Feb. 2006 p.11/25

f(x) = x and g(x) = x, = 2, = 1, X(0) = 1
Solution: X(t) = X(0)e
(
1
2

2
)t+W(t)
M = 5, 000 disc. Brownian paths over [0, 1] with t = 2
11
For each path apply EM with t = t, 2t, 4t, 16t, 32t, 64t
Record E[|X
N
X(1)|] for each t
10
4
10
3
10
2
10
1
10
2
10
1
10
0
t
S
a
m
p
l
e

a
v
e
r
a
g
e

o
f

|

X
N

X
(
T
)

|
Least squares t: power is 0.51
Montreal, Feb. 2006 p.12/25
Strong Convergence
Generally EM has strong order p =
1
2
on appropriate SDEs
Can prove using Itos Lemma, Ito isometry and Gronwall
Note: strong convergence weak convergence,
but this doesnt recover the optimal weak order
Montreal, Feb. 2006 p.13/25
Strong Convergence
EulerMaruyama has
E[|X
n
X(t
n
)|] Kt
1
2
Markov inequality says
P(|X| > a)
E[|X|]
a
, for any a > 0
Taking a = t
1
4
gives P
_
|X
n
X(t
n
)| t
1
4
_
Kt
1
4
, i.e.
P
_
|X
n
X(t
n
)| < t
1
4
_
1 Kt
1
4
Along any path error is small with high prob.
Montreal, Feb. 2006 p.14/25
Higher Strong Order
If g(x) is constant, then EM has strong order p = 1
More generally, strong order p = 1 is achieved by the
Milstein method
X
n+1
= X
n
+ tf(X
n
) + W
n
g(X
n
)
+
1
2
g(X
n
)g

(X
n
)
_
W
2
n
t
_
(More complicated for SDE systems.)
Montreal, Feb. 2006 p.15/25
Even Higher Strong Order: Warning!
Numerical methods for stochastic differential
equations
Joshua Wilkie
Physical Review E, 2004
Claims to derive arbitrarily high (strong?) order methods,
with a RungeKutta approach.
But using only Brownian increments, W
n
, rather than
more general integrals like
_
t
n+1
t
n
_
t
n+1
t
n
dW
1
(s)dW
2
(t)
there is an order barrier of p = 1 (Rmelin, 1982).
Montreal, Feb. 2006 p.16/25
Beyond Convergence . . .
Numerical methods approximate the continuous by the
discrete:
X
n
X(t
n
), with t
n+1
t
n
=: t
Convergence:
How small is X
n
X(t
n
) at some nite t
n
?
Stability (Dynamics):
Does lim
n
X
n
look like lim
t
X(t)?
Study stability by applying the method to a class of test
problems, where information about X(t) is known.
Hope to show good behavior either for all t > 0, or at least
for sufciently small t.
Montreal, Feb. 2006 p.17/25
Stochastic Theta Method
X
n+1
= X
n
+ (1 )tf(X
n
) + tf(X
n+1
) + g(X
n
)W
n
where we recall that W
n
= W(t
n+1
) W(t
n
),
so W
n
=

tV
n
, with V
n
Normal(0, 1) i.i.d.
X
n
X(t
n
) in the SDE (It)
dX(t) = f(X(t))dt + g(X(t))dW(t), X(0) = X
0
Montreal, Feb. 2006 p.18/25
Stochastic Test Equation
dX(t) = X(t)dt + X(t)dW(t)
(Asset model in math-nance)
Mean-square stability
lim
t
E(X(t)
2
) = 0 2 +
2
< 0
STM gives X
n+1
= (a + bV
n
)X
n
, with
a :=
1 + (1 )t
1 t
, b :=

t
1 t
Montreal, Feb. 2006 p.19/25
Mean-square stability
Saito & Mitsui, SIAM J Num Anal 1996
0 <
1
2
: SDE stable method stable iff
t <
|2 +
2
|

2
(1 2)
=
1
2
: SDE stable method stable t > 0
1
2
< 1: SDE stable method stable t > 0
Montreal, Feb. 2006 p.20/25
Stability Regions
Let x := t and y := t
2
SDE stable y < 2x
Method stable y < (2 1)x
2
2x
5 0 5
0
2
4
6
8
10
theta = 0
x
y
5 0 5
0
2
4
6
8
10
theta = 0.25
x
y
5 0 5
0
2
4
6
8
10
theta = 0.75
x
y
5 0 5
0
2
4
6
8
10
theta = 1
x
y
Montreal, Feb. 2006 p.21/25
Stochastic Test Equation
dX(t) = X(t)dt + X(t)dW(t)
Asymptotic stability
lim
t
|X(t)| = 0, with prob. 1 2
2
< 0
Recall that STM gives X
n+1
= (a + bV
n
)X
n
, with
a :=
1 + (1 )t
1 t
, b :=

t
1 t
Montreal, Feb. 2006 p.22/25
Asymptotic Stability: lim
n
|X
n
| = 0, w.p. 1
|X
n
| =
_
n1

i=0
|a + bV
i
|
_
|X
0
|
SLLN: lim
n
|X
n
| = 0 E(log |a + bV
i
|) < 0
Can be expressed in terms of Meijers G-function
Difcult to deal with analytically
No simple expression for stability region boundary
Montreal, Feb. 2006 p.23/25
Asymptotic Stability for Backward Euler ( = 1)
1.5 1 0.5 0 0.5 1 1.5 2 2.5 3 3.5
0
5
10
15
20
25
theta = 1
x
y
Montreal, Feb. 2006 p.24/25
Many open equestions regarding asymptotic stability
E.g. is there an A-stable method?
Generalizations to nonlinear SDEs are also possible
Montreal, Feb. 2006 p.25/25