Applied Intertemporal Optimization

These lecture notes are incomplete. Various sections still need polishing.
This
should be taken into account when using and working with them.
Applied Intertemporal Optimization
- draft -
Klaus Wlde
University of Wrzburg, CESifo, UCL Louvain la Neuve
www.waelde.com
Toolbox2006_8b.tex - January 19, 2007
These lecture notes are permanently being updated. The most recent version
is available at www.waelde.com/aio
@BOOK{Waelde06,
author = {Klaus Wlde},
year = 2006,
title = {Applied Intertemporal Optimization},
publisher = {Lecture Notes, University of Wrzburg},
address = {Available at www.waelde.com/aio}
}
Acknowledgments
This (soon-to-be) book is the result of many lectures given in various places,
including the Bavarian Graduate Program in Economics, the Universities of Dort-
mund, Dresden, Frankfurt, Louvain-la-Neuve, Munich and Wrzburg and the Euro-
pean Commission in Brussels. I would therefore rst like to thank students for their
questions and discussions. These conversations showed what the most important as-
pects in understanding model building and in formulating maximization problems
are. I also proted from many discussions with economists and mathematicians from
many places. I especially would like to thank Ken Sennewald for many insights into
the more subtle issues of stochastic continuous time processes. I hope I succeeded in
incorporating these insights into this book.
The basic structure of this book is replicated here:
1 Introduction
Part I Deterministic models in discrete time
2 OLG models and dierence equations
3 Innite horizon models
Part II Deterministic models in continuous time
4 Dierential equations
5 Optimal control theory - Hamiltonians
6 Dynamic programming in continuous time
Part III Stochastic models in discrete time
7 Stochastic dierence equations and applications
8 Multi-period models
Part IV Stochastic models in continuous time
9 Stochastic dierential equations and rules for dierentiating
10 Dynamic programming
11 On notation and variables
This book can be used for a rst year master class, a PhD course or for studying
for oneself. In fact, given the detailed step-by-step approach to problems, it should be
easier to understand solutions with this text than with other usually more condensed
presentations.
A rst year master class on economic methods (or third year undergraduate stud-
ies, i.e. rst semester Hauptstudium) would cover part I and part II, i.e. maxi-
mization and applications in discrete and continuous time under certainty. A lecture
series of 14 times 90 minutes (corresponding to one semester course in the German
1
system) is usually enough. It is useful to complement the lectures, however, by ex-
ercise classes which can use the same amount of time. Exercises are included at the
end of each chapter. A PhD class would review parts of part I and part II and cover
in full part III and part IV. It also requires around 14 times 90 minutes and exercise
classes help even more given the more complex material.
2
Contents
1 Introduction 1
I Deterministic models in discrete time 5
2 OLG models and dierence equations 5
2.1 The general setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.2 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.2.1 Optimal consumption . . . . . . . . . . . . . . . . . . . . . . . 7
2.2.2 Optimal consumption with prices . . . . . . . . . . . . . . . . 8
2.2.3 Some useful denitions . . . . . . . . . . . . . . . . . . . . . . 9
2.3 The idea behind the Lagrangian . . . . . . . . . . . . . . . . . . . . . 12
2.3.1 Where the Lagrangian comes from I . . . . . . . . . . . . . . . 12
2.3.2 How to understand shadow prices . . . . . . . . . . . . . . . . 14
2.4 A general equilibrium analysis . . . . . . . . . . . . . . . . . . . . . . 15
2.4.1 Technologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.4.2 Households . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.4.3 Goods market equilibrium and accumulation identity . . . . . 17
2.4.4 The reduced form . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.4.5 Properties of the reduced form . . . . . . . . . . . . . . . . . . 19
2.5 More on dierence equations . . . . . . . . . . . . . . . . . . . . . . . 20
2.5.1 Two useful proofs . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.5.2 The simplest dierence equation . . . . . . . . . . . . . . . . . 21
2.5.3 A slightly less but still simple dierence equation . . . . . . . 23
2.5.4 Fix points and stability . . . . . . . . . . . . . . . . . . . . . . 24
2.5.5 An example: Deriving a budget constraint . . . . . . . . . . . 25
2.6 Exercises chapter 2 and further reading . . . . . . . . . . . . . . . . . 27
3 Innite horizon models 31
3.1 The problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.2 Solving by Lagrange . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.3 The envelope theorem . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.3.1 The theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.3.2 Illustration . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.3.3 An example . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.4 Solving by dynamic programming . . . . . . . . . . . . . . . . . . . . 34
3.5 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
3.5.1 Individual utility maximization . . . . . . . . . . . . . . . . . 37
3.5.2 Optimal R&D eort . . . . . . . . . . . . . . . . . . . . . . . 39
3.6 A general equilibrium analysis . . . . . . . . . . . . . . . . . . . . . . 42
3.6.1 Technologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
3.6.2 Firms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
3.6.3 Households . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
3
3.6.4 Aggregation and reduced form . . . . . . . . . . . . . . . . . . 43
3.6.5 Steady state and transitional dynamics . . . . . . . . . . . . . 44
3.6.6 The intertemporal budget constraint . . . . . . . . . . . . . . 45
3.7 A central planner . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
3.7.1 Optimal factor allocation . . . . . . . . . . . . . . . . . . . . . 46
3.7.2 Where the Lagrangian comes from II . . . . . . . . . . . . . . 47
3.8 Exercises Chapter 3 and further reading . . . . . . . . . . . . . . . . 48
II Deterministic models in continuous time 52
4 Dierential equations 52
4.1 Some denitions and theorems . . . . . . . . . . . . . . . . . . . . . . 52
4.1.1 Denitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
4.1.2 Two theorems . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
4.2 Analysing ODEs through phase diagrams . . . . . . . . . . . . . . . . 53
4.2.1 One dimensional systems . . . . . . . . . . . . . . . . . . . . . 53
4.2.2 Two-dimensional systems I - An example . . . . . . . . . . . . 56
4.2.3 Two-dimensional systems II - The general case . . . . . . . . . 58
4.2.4 Types of phase diagrams and xpoints . . . . . . . . . . . . . 62
4.2.5 Multidimensional systems . . . . . . . . . . . . . . . . . . . . 64
4.3 Linear dierential equations . . . . . . . . . . . . . . . . . . . . . . . 64
4.3.1 Rules on derivatives . . . . . . . . . . . . . . . . . . . . . . . 64
4.3.2 Forward and backward solutions of a linear dierential equation 66
4.3.3 Dierential equations as integral equations . . . . . . . . . . . 67
4.3.4 Example 1: Budget constraints . . . . . . . . . . . . . . . . . 68
4.3.5 Example 2: Forward solutions . . . . . . . . . . . . . . . . . . 70
4.3.6 Example 3: A growth model . . . . . . . . . . . . . . . . . . . 71
4.4 Linear dierential equation systems . . . . . . . . . . . . . . . . . . . 72
5 Optimal control theory - Hamiltonians 76
5.1 An introductory example . . . . . . . . . . . . . . . . . . . . . . . . . 76
5.2 Deriving laws of motion . . . . . . . . . . . . . . . . . . . . . . . . . 77
5.2.1 Solving by using the Lagrangian . . . . . . . . . . . . . . . . . 77
5.2.2 Hamiltonians as a shortcut . . . . . . . . . . . . . . . . . . . . 79
5.3 Boundary conditions and sucient conditions . . . . . . . . . . . . . 79
5.3.1 Free value of state variable at endpoint . . . . . . . . . . . . . 79
5.3.2 Fixed value of state variable at endpoint . . . . . . . . . . . . 80
5.3.3 Sucient conditions . . . . . . . . . . . . . . . . . . . . . . . 80
5.4 The innite horizon . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
5.4.1 The maximization problem . . . . . . . . . . . . . . . . . . . . 81
5.4.2 The transversality condition . . . . . . . . . . . . . . . . . . . 81
5.4.3 The boundedness condition . . . . . . . . . . . . . . . . . . . 82
5.5 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
4
5.5.1 Free value at end point - Adjustment costs . . . . . . . . . . . 83
5.5.2 Fixed value at end point - Adjustment costs II . . . . . . . . . 85
5.5.3 Innite horizon - Optimal consumption paths . . . . . . . . . 86
5.5.4 The central planner and capital accumulation . . . . . . . . . 89
5.5.5 The matching approach to unemployment . . . . . . . . . . . 92
5.6 The present value Hamiltonian . . . . . . . . . . . . . . . . . . . . . . 95
5.6.1 Problems without (or with implicit) discounting . . . . . . . . 95
5.6.2 Deriving laws of motion . . . . . . . . . . . . . . . . . . . . . 96
5.6.3 The link between CV and PV . . . . . . . . . . . . . . . . . . 97
6 Dynamic programming in continuous time 103
6.1 Household utility maxmization . . . . . . . . . . . . . . . . . . . . . . 103
6.2 Comparing dynamic programming to Hamiltonians . . . . . . . . . . 106
6.3 Dynamic programming with two state variables . . . . . . . . . . . . 106
6.4 An example: Nominal and real interest rates and ination . . . . . . 109
6.4.1 Firms, the central bank and the government . . . . . . . . . . 109
6.4.2 Households . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
6.4.3 Equilibrium . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
6.6 Looking back . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
III Stochastic models in discrete time 117
7 Stochastic dierence equations and applications 117
7.1 Basics on random variables . . . . . . . . . . . . . . . . . . . . . . . . 117
7.1.1 Some concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
7.1.2 An illustration . . . . . . . . . . . . . . . . . . . . . . . . . . 118
7.2 Examples for random variables . . . . . . . . . . . . . . . . . . . . . 118
7.2.1 Discrete random variables . . . . . . . . . . . . . . . . . . . . 119
7.2.2 Continuous random variables . . . . . . . . . . . . . . . . . . 119
7.2.3 Higher-dimensional random variables . . . . . . . . . . . . . . 119
7.3 Expected values, variances, covariances and all that . . . . . . . . . . 120
7.3.1 Denitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
7.3.2 Lemmas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
7.3.3 Functions on random variables . . . . . . . . . . . . . . . . . . 122
7.4 Examples of stochastic dierence equations . . . . . . . . . . . . . . . 123
7.4.1 A rst example . . . . . . . . . . . . . . . . . . . . . . . . . . 123
7.4.2 A more general case . . . . . . . . . . . . . . . . . . . . . . . 127
7.5 A simple OLG model . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
7.5.1 Technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
7.5.2 Timing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
7.5.3 Firms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
7.5.4 Individual decision problem . . . . . . . . . . . . . . . . . . . 129
5
7.5.5 Aggregation and the reduced form for the CD case . . . . . . 131
7.5.6 Some analytical results . . . . . . . . . . . . . . . . . . . . . . 132
7.6 Risk-averse and risk-neutral households . . . . . . . . . . . . . . . . . 134
7.7 Pricing of contingent claims and assets . . . . . . . . . . . . . . . . . 138
7.7.1 The value of an asset . . . . . . . . . . . . . . . . . . . . . . . 138
7.7.2 The value of a contingent claim . . . . . . . . . . . . . . . . . 139
7.7.3 Risk-neutral valuation . . . . . . . . . . . . . . . . . . . . . . 140
7.8 Natural volatility I (*) . . . . . . . . . . . . . . . . . . . . . . . . . . 140
7.8.1 The basic idea . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
7.8.2 A simple stochastic model . . . . . . . . . . . . . . . . . . . . 142
7.8.3 Equilibrium . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
8 Multi-period models 148
8.1 Dynamic programming . . . . . . . . . . . . . . . . . . . . . . . . . . 148
8.2 Household utility maximization . . . . . . . . . . . . . . . . . . . . . 150
8.3 A central planner in an oil-importing economy . . . . . . . . . . . . . 151
8.4 Asset pricing in a one-asset economy . . . . . . . . . . . . . . . . . . 152
8.4.1 The model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
8.4.2 Optimal behaviour . . . . . . . . . . . . . . . . . . . . . . . . 153
8.4.3 The pricing relationship . . . . . . . . . . . . . . . . . . . . . 154
8.4.4 More real results . . . . . . . . . . . . . . . . . . . . . . . . . 155
8.5 Asset pricing with many assets (*) . . . . . . . . . . . . . . . . . . . 157
8.5.1 A many-sector economy . . . . . . . . . . . . . . . . . . . . . 157
8.5.2 The households maximization problem (*) . . . . . . . . . . 157
8.5.3 The fundamental pricing rule . . . . . . . . . . . . . . . . . . 159
8.5.4 The CAP formula . . . . . . . . . . . . . . . . . . . . . . . . . 160
8.6 Matching and unemployment . . . . . . . . . . . . . . . . . . . . . . 160
8.7 How to do without dynamic programming . . . . . . . . . . . . . . . 160
8.7.1 Individual utility maximization . . . . . . . . . . . . . . . . . 160
8.7.2 Contingent claims pricing . . . . . . . . . . . . . . . . . . . . 161
8.7.3 Sticky prices (*) . . . . . . . . . . . . . . . . . . . . . . . . . . 162
8.7.4 Optimal employment with adjustment cost (*) . . . . . . . . . 164
8.8 An explicit time path for a boundary condition . . . . . . . . . . . . 165
IV Stochastic models in continuous time 170
9 Stochastic dierential equations and rules for dierentials 170
9.1 Some basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
9.1.1 Stochastic processes . . . . . . . . . . . . . . . . . . . . . . . 170
9.1.2 Stochastic dierential equations . . . . . . . . . . . . . . . . . 173
9.1.3 The integral representation of stochastic dierential equations 176
9.2 Functions of stochastic processes . . . . . . . . . . . . . . . . . . . . . 177
6
9.2.1 Why all this? . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
9.2.2 Computing dierentials for Brownian motion . . . . . . . . . . 178
9.2.3 Computing dierentials for Poisson processes . . . . . . . . . . 180
9.2.4 Brownian motion and a Poisson process . . . . . . . . . . . . . 182
9.2.5 Application - Option pricing . . . . . . . . . . . . . . . . . . . 183
9.2.6 Application - Deriving a budget constraint . . . . . . . . . . . 185
9.3 Solving stochastic dierential equations . . . . . . . . . . . . . . . . . 186
9.3.1 Some examples for Brownian motion . . . . . . . . . . . . . . 186
9.3.2 A general solution for Brownian motions . . . . . . . . . . . . 189
9.3.3 Dierential equations with Poisson processes . . . . . . . . . . 191
9.4 Expectation values . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194
9.4.1 The idea . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194
9.4.2 Simple results . . . . . . . . . . . . . . . . . . . . . . . . . . . 195
9.4.3 Martingales . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198
9.4.4 A more general approach to computing moments . . . . . . . 199
9.5 Exercise to chapter 9 and further reading . . . . . . . . . . . . . . . . 203
10 Dynamic programming 209
10.1 Optimal saving under Poisson uncertainty . . . . . . . . . . . . . . . 209
10.1.1 The setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209
10.1.2 Three dynamic programming steps . . . . . . . . . . . . . . . 210
10.1.3 The solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212
10.1.4 Optimal consumption and portfolio choice . . . . . . . . . . . 213
10.1.5 Other ways to determine ~ c . . . . . . . . . . . . . . . . . . . . 217
10.1.6 Expected growth . . . . . . . . . . . . . . . . . . . . . . . . . 219
10.2 Matching on the labour market: where value functions come from . . 220
10.2.1 A household . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220
10.2.2 The Bellman equation and value functions . . . . . . . . . . . 221
10.3 Optimal saving under Brownian motion . . . . . . . . . . . . . . . . . 221
10.3.1 The setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
10.3.2 Three dynamic programming steps . . . . . . . . . . . . . . . 222
10.3.3 The solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223
10.4 Capital asset pricing . . . . . . . . . . . . . . . . . . . . . . . . . . . 224
10.4.1 The setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224
10.4.2 Optimal behaviour . . . . . . . . . . . . . . . . . . . . . . . . 225
10.4.3 Capital asset pricing . . . . . . . . . . . . . . . . . . . . . . . 227
10.5 Natural volatility II (*) . . . . . . . . . . . . . . . . . . . . . . . . . . 228
10.5.1 An RBC model . . . . . . . . . . . . . . . . . . . . . . . . . . 228
10.5.2 A natural volatility model . . . . . . . . . . . . . . . . . . . . 230
10.5.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230
10.6 Exercises chapter 10 and further reading . . . . . . . . . . . . . . . . 230
11 Notation and variables, references and index 236
7
12 Exams 247
(*) These sections are still work in progress. See the note on page 2.
8
1 Introduction
The objective of this course is to provide a toolbox for solving maximization problems
and for working with their solutions in economic models. Maximization problems can
be formulated in discrete or continuous time, under certainty or uncertainty. Various
maximization methods will be used, ranging from solving by inserting, via La-
grangian and Hamiltonian approaches to dynamic programming (Bellman equation).
Dynamic programming will be used for all environments, discrete, continuous, certain
and uncertain, the Lagrangian for most of them. Solving by inserting is also very use-
ful in discrete time setups. The Hamiltonian approach is used only for deterministic
continuous time setups. An overview is given on the next page.
Part I deals with discrete time models under certainty, Part II covers continuous
time models under certainty. Part III deals with discrete time models under uncer-
tainty and Part IV, logically, analyzes continuous time models under uncertainty.
The course is called applied intertemporal optimization, where the emphasis is
on applied. This means that each type of maximization problem will be illustrated
by some example from the literature. These will come from micro- and macroeco-
nomics, from nance, environmental economics and game theory. This also means,
maybe more importantly, that there is little emphasis on formal scrutiny. This course
is about computing solutions. A more formal treatment (especially of the stochastic
part) can be found elsewhere in the literature and many references will be given.
Anybody who has a nice maximization problem as part of a published
paper and who wants it to be presented in this book, is very wel-
come to send it as both a .tex and .pdf le to klaus@waelde.com. If
it complements or updates existing examples in an interesting way, I
would be very happy to include it. Given the idea of this book, how-
ever, the solution of maximization problems must be presented step
by step such that an understanding requires only short reading for an
economist with experience in solving maximization problems. Clearly,
the original author of the maximization problem will be stated in the
corresponding chapter.
Given that certain maximization problems are solved many times - e.g. utility
maximization of a household rst under certainty in discrete and continuous time and
then under uncertainty in discrete and continuous time - and using many methods,
the rules how to compute solutions can be easily understood: First, the discrete
deterministic two-period approach provides the basic intuition or feeling for a solution.
Next, innite horizon problems add one dimension of complexity by taking away
the simple boundary condition of nite horizon models. In a third step, uncertainty
adds expectations operators and so on. By gradually working through increasing steps
of sophistication and by linking back to simple but structurally identical examples,
intuition for the complex setups is built up as much as possible. This approach then
allows to nally understand the beauty of e.g. Keynes-Ramsey rules in continuous
time under Poisson uncertainty, Brownian motion or with Levy processes.
1
Even though I would like this to be a nice, clean and nished textbook,
this is unfortunately not yet the case. Some sections are incomplete
or contain only sketches of ideas. This is also due to the fact that
these notes are permanently expanding in an attempt to cover further
interesting material. The sections which are still work in progress are
marked by an asterisk (*).
An overview of maximization methods
In order to get quick access to various model building blocks, the following tables
can be used as guides. The tables show the numbers of the sections where the methods
are presented. Columns indicate whether the environment is of a deterministic or
stochastic nature, rows distinguish between setups in discrete and continuous time.
Lagrangian deterministic stochastic
discrete 2.3, 3.2, 3.7 -
continuous 5.2.1, 5.5.3 -
Hamiltonian deterministic stochastic
discrete - -
continuous 5 -
Solution by
inserting
deterministic stochastic
discrete 2.2.1 8.7
continuous - -
Dynamic
programming
discrete 3.4 8
continuous 6 10
Figure 1 Methods for solving maximization problems
An overview of model building tools
The simplest approach to dynamic modelling is denitely a two-period setup.
Surprisingly often, this yields many useful insights despite - or probably because of
- its simple structure. This simple structure allows very often to obtain analytical
solutions, in deterministic and stochastic setups, even when utility functions are not
Cobb-Douglas. Some examples are therefore provided also here, both for their own
sake, but also as introduction to more complex dynamic systems.
2
2 period GE
OLG model
discrete 2.4 7.5
continuous - -
Figure 2 The simplest approach to dynamic models
In addition to solving maximization problems, this book also shows (i) how the
structure of maximization problems is determined by the economic environment an
agent nds itself in and (ii) how to go from solutions of maximization problems
to properties of the resulting dynamic system. The former point simply states that
budget constraints of households or rms are endogenous and depend on technological
fundamentals of the model.
endogenous
budget constraints
discrete 2.5.5, 3.6.3 -
continuous 4.3.4 9.2.6
how to nd the
reduced form
discrete 2.4.4, 3.6.4 7.5.5
continuous - -
Figure 3 Things to be taken care of in model building
An overview of economic contents
After having gone through part I and part II, students will be familiar with the
two-period overlapping-generations (OLG) model, the optimal saving central planner
model in discrete and continuous time, the matching approach to unemployment,
the decentralized optimal growth model and (coming soon) an optimal growth model
with money, allowing to understand the dierence between nominal and real interest
rates in general equilibrium.
In part III and IV, economic insights will be extended and cover portfolio choices,
growth under uncertainty ...
Even more motivation for this book
Why teach a course based on this book? Is it not boring to go through all these
methods? In a way, the answer is yes. We all want to understand certain empirical
regularities or understand potential fundamental relationships and make exciting new
empirically testable predictions. In doing so, we also all need to understand existing
work and eventually present our own ideas. It is probably much more boring to
be hindered in understanding existing work and be almost certainly excluded from
3
presenting own ideas if we always spend long time trying to understand how certain
results were derived. How did this author get from equation (1) and (2) to equation
(3)? The major advantage of economic analysis over other social sciences is its strong
foundation in formal models. These models allow to discuss questions much more
precisely as expressions like marginal utility, time preference rate or Keynes-
Ramsey rule reveal a lot of information in very short time. It is therefore extremely
useful to rst spend some time in getting to know these methods and then try to do
what economics is really about: understand the real world.
But, before we really start, there is also a second reason - at least for some
economists - to go through all these methods: They contain a certain type of truth.
A proof is true or false. The derivation of some optimal behaviour is true or false. A
prediction of general equilibrium behaviour of an economy is truth. Unfortunately,
it is only truth in an analytical sense, but this is at least some type of truth. Better
than none.
4
Part I
Deterministic models in discrete
time
2 OLG models and dierence equations
Given the idea of this book to start from simple structures and extending them to the
more complex ones, this chapter starts with the simplest intertemporal problem, a
two-period decision framework. This simple setup already allows to illustrate the basic
dynamic trade-os. Aggregating over individual behaviour, assuming an overlapping-
generations (OLG) structure, and putting individuals in general equilibrium provides
an understanding of the issues involved in these steps and in identical steps in more
general settings. Some revision of properties of dierence equations concludes this
chapter.
2.1 The general setup
Let there be an individual living for two periods, in the rst it is young, in the second
it is old. Let her utility function be given by
l
t
= l
_
c
j
t
. c
c
t1
_
= l (c
t
. c
t1
) . (2.1)
where consumption when young and old are denoted by c
j
t
and c
c
t1
or c
t
and c
t1
.
respectively, when no ambiguity arises. The individual earns labour income n
t
in
both periods. It could also be assumed that she earns labour income only in the rst
period (e.g. when retiring in the second) or only in the second period (e.g. when
going to school in the rst). Here, with :
t
denoting savings, her budget constraint in
the rst period is
c
t
+:
t
= n
t
(2.2)
and in the second it reads
c
t1
= n
t1
+ (1 + :
t1
) :
t
.
Interests paid on savings in the second period are given by :
t1
. All quantities are
expressed in units of the consumption good (i.e. the price of the consumption good
is set equal to one. See ch. 2.2.2 for an extension where the price of the consumption
good is j
t
.).
This budget constraint says something about the assumptions made on the timing
of wage payments and savings. What an individual can spend in period two is princi-
pal and interest of her savings :
t
of the rst period. There are no interest payments in
period one. This means that wages are paid and consumption takes place at the end
of period 1 and savings are used for some productive purposes (e.g. rms use it in the
form of capital for production) in period 2. Therefore, returns :
t1
are determined
5
by economic conditions in period 2 and have the index t + 1. Timing is illustrated in
the following gure.
:
t
= n
t
c
t
n
t
c
t
t
(1 +:
t1
):
t
c
t1
n
t1
t + 1
Figure 4 Timing in OLG models
These two budget constraints can be merged into one intertemporal budget con-
straint by substituting out savings,
n
t
+ (1 + :
t1
)
1
n
t1
= c
t
+ (1 + :
t1
)
1
c
t1
. (2.3)
It should be noted that by not restricting savings to be non-negative in (2.2) or by
equating the present value of income on the left-hand side with the present value
of consumption on the right-hand side in (2.3), we assume perfect capital markets:
individuals can save and borrow any amount they desire. Equation (2.13) provides a
condition under which individuals save.
Adding the behavioural assumption that individuals maximize utility, the eco-
nomic behaviour of an individual is completely described and one can derive her
consumption and saving decisions. The problem can be solved by a Lagrange ap-
proach, by simply inserting (as e.g. in ch. 7.5.4), or in other ways, to be presented
shortly.
The maximization problem reads max
c
I
, c
I+1
(2.1) subject to the intertemporal bud-
get constraint (2.3). The households control variables are c
t
and c
t1
. As they need
to be chosen such that they satisfy the budget constraint, they can not be chosen
independently of each other. Wages and interest rates are exogenously given to the
household. When choosing consumption levels, the reaction of these quantities to the
decision of our household is assumed to be zero - simply because the household is too
small to have an eect on economy-wide variables. The corresponding Lagrangian
reads
/ = l (c
t
. c
t1
) +`
_
n
t
+ (1 + :
t1
)
1
n
t1
c
t
(1 +:
t1
)
1
c
t1
,
where ` is a constant called the Lagrange multiplier.
The rst-order conditions are
/
c
I
= l
c
I
(c
t
. c
t1
) ` = 0.
/
c
I+1
= l
c
I+1
(c
t
. c
t1
) `[1 +:
t1
]
1
= 0.
/
A
= n
t
+ (1 + :
t1
)
1
n
t1
c
t
+ (1 + :
t1
)
1
c
t1
= 0.
6
Clearly, the last rst-order condition is identical to the budget constraint. Note that
there are three variables to be determined, consumption for both periods and the
Lagrange multiplier `.
1
Having at least three conditions is a necessary, though not
sucient (they might, generally speaking, be linearly dependent) condition for this
to be possible.
The rst two rst-order conditions can be combined to give
l
c
I
(c
t
. c
t1
) = (1 +:
t1
) l
c
I+1
(c
t
. c
t1
) = `. (2.4)
Marginal utility from consumption today on the left-hand side must equal marginal
utility tomorrow, corrected by the interest rate, on the right-hand side. The economic
meaning of this correction can best be understood when looking at a version with
nominal budget constraints (see ch. 2.2.2).
What we learn from this maximization here is that its solution (2.4) gives us
an equation, that needs to hold when behaving optimally, which links consumption
c
t
today to consumption c
t1
tomorrow. This equation together with the budget
constraint (2.3) provides a two-dimensional system: two equations in two unknowns,
c
t
and c
t1
. These equations therefore allow in principle to compute these endogenous
variables as a function of exogenously given wages and interest rates. The next section
provides an example where this is indeed the case.
2.2 Examples
2.2.1 Optimal consumption
This rst example allows us to indeed solve explicitly for consumption levels in both
periods. Let households have a Cobb-Douglas utility function represented by
l
t
= ln c
t
+ (1 ) ln c
t1
. (2.5)
Utility from consumption in each period, instantaneous utility, is given by the log-
arithm of consumption. As ln c has a positive rst and negative second derivative,
higher consumption increases instantaneous utility but at a decreasing rate. The
parameter captures the importance of instantaneous utility in the rst relative to
instantaneous utility in the second. Overall utility l
t
is maximized subject to the
constraint we know from above,
\
t
= c
t
+ (1 + :
t1
)
1
c
t1
. (2.6)
where we denote the present value of labour income by \
t
= n
t
+ (1 +:
t1
)
1
n
t1
.
Again, the consumption good is chosen as numeraire good and its price equals unity.
Wages are therefore expressed in units of the consumption good.
The Lagrangian for this problem reads
/ = ln c
t
+ (1 ) ln c
t1
+`
_
\
t
c
t
(1 +:
t1
)
1
c
t1
.
1
Note that equals the marginal utility of one extra unit of income (cf. section 2.3).
7
The rst-order conditions are
/
c
I
= (c
t
)
1
` = 0.
/
c
I+1
= (1 ) (c
t1
)
1
`[1 +:
t1
]
1
= 0.
and the budget constraint (2.6) following from /
A
= 0. Dividing rst-order conditions
gives

1
c
I+1
c
I
= 1 +:
t1
and therefore
c
t
=

1
(1 +:
t1
)
1
c
t1
.
This equation corresponds to our optimality rule (2.4) derived above for the more
general case. Inserting into the budget constraint (2.6) yields
\
t
=
_

1
+ 1
_
(1 +:
t1
)
1
c
t1
=
1
1
(1 +:
t1
)
1
c
t1
which is equivalent to
c
t1
= (1 ) (1 +:
t1
) \
t
. (2.7)
It follows that
c
t
= \
t
. (2.8)
Apparently, optimal consumption decisions imply that consumption when young is
given by a share of the present value \
t
of life-time income at time t of the individual
under consideration. Consumption when old is given by the remaining share 1
plus interest payments, c
t1
= (1 +:
t1
) (1 ) \
t
. Assuming preferences as in (2.5)
makes modelling sometimes easier than with more complex utility functions. The
drawback is that the share of lifetime income consumed and thereby the savings
decision is independent of the interest rate, which appears implausible. A way out
is given by the CES utility function which also allows for closed-form solutions of
consumption (cf. ch. 7.5.4). More generally speaking, such a simplication should be
justied if some aspect of an economy that are fairly independent of savings are the
focus of some analysis. For an example concerning bequests and wealth distributions,
see Kleiber, Sexauer and Wlde (2005).
2.2.2 Optimal consumption with prices
Consider now again the utility function (2.1) and maximize it subject to the con-
straints j
t
c
t
+ :
t
= n
t
and j
t1
c
t1
= n
t1
+ (1 + :
t1
) :
t
. The dierence to the
introductory example in ch. 2.1 consists in the introduction of an explicit price j
t
for the consumption good. The rst-period budget constraint now equates nominal
expenditure for consumption (j
t
c
t
is measured in, say, Euro, Dollar or Yen) plus nom-
inal savings to nominal wage income. The second period constraint equally equates
nominal quantities. How does an optimal consumption rule as in (2.4) now look like?
The Lagrangian is, using now an intertemporal budget constraint with prices,
/ = l (c
t
. c
t1
) +`
_
n
t
+ (1 + :
t1
)
1
n
t1
j
t
c
t
(1 +:
t1
)
1
j
t1
c
t1
.
8
The rst-order conditions for c
t
and c
t1
are
/
c
I
= l
c
I
(c
t
. c
t1
) `j
t
= 0.
/
c
I+1
= l
c
I+1
(c
t
. c
t1
) `(1 +:
t1
)
1
j
t1
= 0
and the intertemporal budget constraint. Combining them gives
l
c
I
(c
t
. c
t1
)
j
t
=
l
c
I+1
(c
t
. c
t1
)
j
t1
[1 +:
t1
]
1
=
l
c
I
(c
t
. c
t1
)
l
c
I+1
(c
t
. c
t1
)
=
j
t
j
t1
[1 +:
t1
]
1
. (2.9)
This equation says that marginal utility of consumption today relative to marginal
utility of consumption tomorrow equals the relative price of consumption today and
tomorrow. This optimality rule is identical for a static 2-good decision problem
where optimality requires that the ratio of marginal utilities equals the relative price.
The relative price here is expressed in a present value sense as we compare marginal
utilities at two points in time. The price j
t
is therefore divided by the price tomorrow,
discounted by next periods interest rate, j
t1
[1 +:
t1
]
1
.
2.2.3 Some useful denitions
In order to be able to discuss results in subsequent sections easily, we review some
denitions here that will be useful. We are mainly interested in the intertemporal
elasticity of substitution and the time preference rate. We start with the
Marginal rate of substitution (MRS)
Let utility be given by n(c
1
. c
2
. ..... c
a
) . The MRS between good i and good ,
given a consumption bundle (c
1
. c
2
. ..... c
a
) = (.) is then dened by
`1o
i)
(.) =
Jn (.) ,Jc
i
Jn (.) ,Jc
)
.
It gives the additional amount of consumption of good , that is required to keep the
utility level at n(c
1
. c
2
. ..... c
a
) when the amount of i is decreased marginally.
Why this is so can easily be shown: Consider the total dierential of n(c
1
. c
2
. ..... c
a
) .
keeping all consumption levels apart from c
i
and c
)
x. This yields
dn(c
1
. c
2
. ..... c
a
) =
Jn(.)
Jc
i
dc
i
+
Jn (.)
Jc
)
dc
)
.
The overall utility level n(c
1
. c
2
. ..... c
a
) does not change if
dn(.) = 0 =
dc
)
dc
i
=
Jn (.)
Jc
i
,
Jn (.)
Jc
)
= `1o
i)
(.) .
Equivalent terms
9
As a reminder, the equivalent term to the MRS in production theory is the mar-
ginal rate of technical substitution `11o
i)
(.) =
0)(.)0a
.
0)(.)0a
where the utility function

was replaced by a production function and consumption c
I
was replaced by factor
inputs r
I
.
More on an economy wide level, there is the marginal rate of transformation
`11
i)
(.) =
0G(.)0j
.
0G(.)0j
where the utility function was now replaced by a transformation

function G (maybe better known as production possibility curve) and the
I
are
output of good /. The marginal rate of transformation gives the increase in output
of good , when output of good i is marginally decreased.
(Intertemporal) elasticity of substitution
Though our main interest is a measure of intertemporal substitutability, we rst
dene the elasiticty of substition in general. The most common denition for two
consumption goods is c
i)
=
o(c
.
c
)
o(j
.
j
)
j
.
j
c
.
c
: how much does relative consumption c

i
,c
)
adjust when relative prices j
i
,j
)
change (proportionally to the current relative-price
relative-consumption ratio
j
.
j
c
.
c
)? This denition can be expressed alternatively (see

ex. 5 for details) in a way which is more useful here. Here, the elasticity of substitution
is dened by the derivative of the log of relative consumption with respect to the log
of the marginal rate of substitution,
c
i)
=
d ln (c
i
,c
)
)
d ln `1o
i)
. (2.10)
Inserting the MRS gives
c
i)
=
d ln (c
i
,c
)
)
d ln
_
n
c
.
,n
c
_ =
n
c
.
,n
c
c
i
,c
)
d (c
i
,c
)
)
d
_
n
c
.
,n
c
_
The advantage of an elasticity when compared to a normal derivative as e.g. the MRS
is that an elasticity is measureless. It is expressed in percentage changes. (This can
be seen in the following example and in exercise number 5a.) It can both be applied
to static utility or production functions or to intertemporal utility functions.
The intertemporal elasticity of substitution for a utility function n(c
t
. c
t1
) is then
simply the elasticity of substitution of consumption at two points in time,
c
t,t1
=
n
c
I
,n
c
I+1
c
t
,c
t1
d (c
t
,c
t1
)
d
_
n
c
I
,n
c
I+1
_.
Using as example the Cobb-Douglas utility function from (2.5), we obtain
c
t,t1
=

c
I
,
1
c
I+1
c
t
,c
t1
d (c
t
,c
t1
)
d
_
c
I
,
1
c
I+1
_ = 1.
where the last step used
d (c
t
,c
t1
)
d
_
c
I
,
1
c
I+1
_ =
1
d
_
1
c
I+1
c
I
_
d (c
t1
,c
t
)
=
1
1
(c
t1
,c
t
)
2
.
10
The time preference rate
Intuitively, the time preference rate is the rate at which future instantaneous
utilities are discounted. To illustrate, imagine a discounted income stream
r
0
+
1
1 +:
r
1
+
_
1
1 +:
_
2
r
2
+...
where discounting takes place at the interest rate :. Replacing income r
t
by instanta-
neous utility and the interest rate by j, j would be the time preference rate. Formally,
the time preference rate is the marginal rate of substitution of instantaneous utilities
(not of consumption levels) minus one,
2
111 = `1o
t,t1
1.
As an example, consider the following standard utility function which we will use
very often in later chapters,
l
0
=
1
t=0
,
t
n(c
t
) . , =
1
1 +j
. j 0. (2.11)
Let j be a positive parameter and , the implied discount factor, capturing the idea of
impatience: By multiplying instantaneous utility functions n(c
t
) by ,
t
, future utility
is valued less than present utility. This utility function generalizes (2.5) in two ways:
First and most importantly, there is a much longer planning horizon than just two
periods. In fact, the individuals overall utility l
0
stems from the sum of discounted
instanteneous utility levels n(c
t
) over periods 0. 1. 2. ... up to innity. The idea
behind this objective function is not that individuals live forever but that individuals
care about the well-being of subsequent generations. Second, the instantaneous utility
function n(c
t
) is not logarithmic as in (2.5) but of a more general nature where one
would usually assume positive rst and negative second derivatives, n
0
0. n
00
< 0.
The marginal rate of substitution is then
`1o
c,c1
(.) =
Jl
0
(.) ,Jn(c
c
)
Jl
0
(.) ,Jn(c
c1
)
=
(1, (1 +j))
c
(1, (1 +j))
c1
= 1 +j.
The time preference rate is therefore given by j.
Now take as example the utility function (2.5). Computing the MRS minus one,
we have
j =

1
1 =
2 1
1
. (2.12)
The time preference rate is positive if 0.5. This makes sense for (2.5) as one
should expect that future utility is valued less than present utility.
Does consumption increase over time?
2
An alternative formulation implying the same denition is used by Buiter (1981, p. 773). He
denes the pure rate of time preference as the marginal rate of substitution between consumption
in two periods when equal amounts are consumed in both periods, minus one.
11
This denition of the time preference rate allows us to provide a precise answer
to the question whether consumption increases over time. We simply compute the
condition under which c
t1
c
t
by using (2.7) and (2.8),
c
t1
c
t
=(1 ) (1 +:) \
t
\
t
=
1 +:

1
=:
1 +
1
=: j.
Consumption increases if the interest rate is higher than the time preference rate.
The time preference rate of the individual (being represented by ) determines how
to split the present value \
t
of total income into current and future use. If the interest
rate is suciently high to overcompensate impatience, i.e. if (1 ) (1 +:) in
the rst line, consumption rises.
Note that even though we computed the condition for rising consumption for our
special utility function (2.5), the result that consumption increases when the interest
rate exceeds the time preference rate holds for more general utility functions as well.
We will get to know various examples for this in subsequent chapters.
Under what conditions are savings positive?
Savings are given by
:
t
= n
t
c
t
= n
t
_
n
t
+
1
1 +:
n
t1
_
= n
_
1

1 +:
_
where the last equality assumed an invariant wage level, n
t
= n
t1
. Savings are
positive if and only if
:
t
0 =1

1 +:
==1 +:

1
==
:
2 1
1
==: j (2.13)
This means that savings are positive if interest rate is larger than time preference
rate. Clearly, this result does not necessarily hold for n
t1
n
t
.
2.3 The idea behind the Lagrangian
2.3.1 Where the Lagrangian comes from I
The maximization problem
Let us consider a maximization problem max
a
1
,a
2
1(r
1
. r
2
) subject to some con-
straint q(r
1,
r
2
) = /. (The presentation here is strongly inspired by Intriligator, 1971,
p. 28 - 30). If we rewrite the constraint as r
2
= /(r
1
) (which is possible if the
function q (.) is not too complex, i.e. not e.g. of the type q (.) = c
a
1
a
2
+ r
1
r
2
), the
maximisation problem can be written as
max
a
1
1 (r
1,
/(r
1
)) .
12
The derivatives of implicit functions
As we will use derivatives of implicit functions here and in later chapters, we
briey show how to compute them. Consider a function , (r
1
. r
2
. .... r
a
) = 0. Its
dierential is
d, (.) =
J, (.)
Jr
1
dr
1
+
J, (.)
Jr
2
dr
2
+... +
J, (.)
Jr
a
dr
a
= 0.
When we keep r
S
to r
a
constant, we can solve this to get
dr
2
dr
1
=
J, (.) ,Jr
1
J, (.) ,Jr
2
. (2.14)
This equation can be seen as an application of the implicit function theorem which
- stated simply - says that the function , (r
1
. r
2
. .... r
a
) = 0 implicitly denes (un-
der suitable assumptions concerning the properties of , (.)) a functional relationship
r
2
= q (r
1
. r
S
. r
1
. .... r
a
) . Hence, the derivative (2.14) is the derivative of this implicit
function q (.) with respect to r
1
when all other variables are held constant. Most of
the time, we will compute derivatives of implicit functions stemming from functions
with two variables, i.e. , (.) = , (r
1
. r
2
)
Solution of the maximization problem
This has a standard optimality condition
J1
Jr
1
+
J1
Jr
2
d/
dr
1
= 0. (2.15)
Taking into consideration that from the implicit function theorem applied to the
constraint,
d/
dr
1
=
dr
2
dr
1
=
Jq(r
1
. r
2
),Jr
1
Jq(r
1
. r
2
),Jr
2
. (2.16)
the optimality condition (2.15) can be written as
01
0a
1
010a
2
0j0a
2
0j(a
1
,a
2
)
0a
1
= 0. Now dene
the Lagrange multiplier ` =
010a
2
0j0a
2
and obtain
J1
Jr
1
`
Jq(r
1
. r
2
)
Jr
1
= 0. (2.17)
As can be easily seen, this is the rst-order condition of the Lagrangian
/ = 1 (r
1,
r
2
) +`[/ q (r
1,
r
2
)]
with respect to r
1
.
Undertaking the same steps for r
2
would yield the second rst-order condition
J1
Jr
2
`
Jq(r
1
. r
2
)
Jr
2
= 0.
We have thereby shown where the Lagrangian comes from.
13
2.3.2 How to understand shadow prices
The idea
We can now also give an interpretation of the meaning of the multipliers `. Start-
ing from the denition of ` in (2.17) and rewriting them according to
` =
J1,Jr
2
Jq,Jr
2
=
J1
Jq
=
J1
J/
.
we see that ` equals the change in 1 as a function of /:
3
By how much does 1 increase
(e.g. your utility) when your constraint / (your bank account) is relaxed? How much
does the social welfare function change when the economy has more capital? How
much do prots of rms change when the rm has more workers?
A derivation
A more rigorous derivation is as follows (cf. Intriligator, 1971, ch. 3.3). Compute
the derivative of the maximized Lagrangian with respect to /,
J/(r
1
(/) . r
2
(/))
J/
=
J
J/
(1(r
1
(/). r
2
(/)) +`(/) [/ q (r
1
(/). r
2
(/))]
= 1
a
1
Jr
1
J/
+1
a
2
Jr
2
J/
+`
0
(/) [/ q(.)] +`(/)
_
1 q
a
1
Jr
1
J/
q
a
2
Jr
2
J/
_
= `(/)
The last equality results from rst-order conditions and the fact that the budget
constraint holds.
As /(r
1
. r
2
) = 1(r
1
. r
2
) due to the budget constraint holding with equality,
`(/) =
J/(r
1
. r
2
)
J/
=
J1(r
1
. r
2
)
J/
Shadow prices
Sometimes to often, ` is referred to as shadow price. One can think of price
in the sense of a price in a currency e.g. in Euro only if the objective function is
some nominal expression like prots or GDP, otherwise it is a price expressed e.g. in
utility terms. This can be nicely seen in the following example. Consider a central
planner that maximizes social welfare n(r
1
. r
2
) subject to technological and resource
constraints,
max n(r
1
. r
2
)
subject to
r
1
= , (1
1,
1
1
) . r
2
= q (1
2,
1
2
) .
1
1
+1
2
= 1. 1
1
+1
2
= 1.
3
The term @x
2
cancels out as by denition
@f(x
1
;:::;x
n
)
@x
i
=
lim
4x
i
!0
f(x
1
;:::;x
i
+4x
i
;:::;x
n
)f(x
1
;:::;x
n
)
lim
4x
i
!0
4x
i
:
14
Using as multipliers j
1
. j
2
. n
1
and n
1
. the Lagrangian reads
/ = n(r
1
. r
2
) +j
1
[, (1
1,
1
1
) r
1
] +j
2
[q (1
2
. 1
2
) r
2
]
+n
1
[1 1
1
1
2
] +n
2
[1 1
1
1
2
]
and rst-order conditions are
J/
Jr
1
=
Jn
Jr
1
j
1
= 0.
J/
Jr
2
=
Jn
Jr
2
j
2
= 0.
J/
J1
1
= j
1
J,
J1
1
n
1
= 0.
J/
J1
2
= j
2
Jq
J1
2
n
1
= 0.
J/
J1
1
= j
1
J,
J1
1
n
1
= 0.
J/
J1
2
= j
2
Jq
J1
2
n
1
= 0.
Here we see that the rst multiplier j
1
is not a price expressed in some currency
but the derivative of the utility function with respect to good 1, i.e. marginal utility.
By contrast, if we looked at the multiplier n
1
only in the third rst-order condition,
we would then conclude that it is a price. Inserting then the rst rst-order condition
shows however that it really stands for the increase in utility when the capital stock
used in production of good 1 rises,
n
1
= j
1
J,
J1
1
=
Jn
Jr
1
J,
J1
1
=
Jn
J1
1
.
Hence n
1
and all other multipliers are prices in utility units.
It is now also easy to see that all shadow-prices are prices expressed in some
currency if the objective function was not utility but, e.g. GDP. Such a maximization
problem could read max j
1
r
1
+j
2
r
2
subject to the constraints as above.
2.4 A general equilibrium analysis
We will now analyze many households jointly and see how their consumption and
saving behaviour aects the evolution of the economy as a whole. We will get to
know the Euler theorem and how it is used to sum factor incomes to yield GDP.
We will also understand how the interest rate in the households budget constraint is
related to marginal productivity of capital and the depreciation rate. All this jointly
yields time paths of aggregate consumption, the capital stock and GDP.
A model in general equilibrium is described by fundamentals of the model and
market and behavioural assumptions. Fundamentals are technologies of rms, prefer-
ences of households and factor endowments. Adding clearing conditions for markets
and behavioural assumptions for agents completes the description of the model.
2.4.1 Technologies
The rms
15
Let there be many rms who use a technology
1
t
= 1 (1
t
. 1) (2.18)
which is characterized by constant returns to scale. Letting rms act under perfect
competition, the rst-order conditions from prot maximization, where prots are
given by :
t
= 1
t
n
1
t
1 n
1
t
1, read
J1
t
J1
t
= n
1
t
.
J1
t
J1
t
= n
1
t
. (2.19)
They equate in each period t the marginal productivity of capital to the factor price
n
1
t
for capital and the marginal productivity of labour to labours factor reward n
1
t
.
Eulers theorem
Eulers theorem shows that for a linear-homogenous function , (r
1
. r
2
. .... r
a
) the
sum of partial derivatives times the variables with respect to which the derivative was
computed equals the original function , (.) .
, (r
1
. r
2
. .... r
a
) =
J, (.)
Jr
1
r
1
+
J, (.)
Jr
2
r
2
+... +
J, (.)
Jr
a
r
a
. (2.20)
Provided that the technology used by rms to produce 1
t
has constant returns to
scale, we obtain from this theorem that
1
t
=
J1
t
J1
t
1
t
+
J1
t
J1
t
1
t
. (2.21)
Using the optimality conditions (2.19) of the rm for the applied version of Eulers
theorem (2.21) yields
1
t
= n
1
t
1 +n
1
t
1. (2.22)
Total output in this economy, 1
t
. is identical to total factor income. This result is
usually given the economic interpretation that under perfect competition all revenue
of rms is used to pay factors of production. As a consequence, prots :
t
of rms are
zero.
2.4.2 Households
Individual households
Households live again for two periods. The utility function is therefore as in (2.5)
and given by
l
t
= ln c
j
t
+ (1 ) ln c
c
t1
. (2.23)
It is maximized subject to the intertemporal budget constraint
n
t
= c
j
t
+ (1 + :
t1
)
1
c
c
t1
.
16
This constraint diers slightly from (2.6) in that people work only in the rst period
and retire in the second. Hence, there is labour income only in the rst period on
the left-hand side. Savings of the rst period are used to nance consumption in the
second period.
Given that the present value of lifetime wage income is n
t
. we can conclude from
(2.7) and (2.8) that individual consumption expenditure and savings are given by
c
j
t
= n
t
. c
c
t1
= (1 ) (1 +:
t1
) n
t
. (2.24)
:
t
= n
t
c
j
t
= (1 ) n
t
. (2.25)
Aggregation
We assume that in each period 1 individuals are born and die. Hence, the number
of young and the number of old is 1 as well. As all individuals within a generation
are identical, aggregate consumption within one generation is simply the number of,
say, young times individual consumption. Aggregate output in t is therefore given by
C
t
= 1c
j
t
+ 1c
c
t
. Using the expressions for individual consumption from (2.24) and
noting the index t (and not t + 1) for the old yields
C
t
= 1c
j
t
+1c
c
t
= (n
t
+ (1 ) (1 +:
t
) n
t1
) 1.
2.4.3 Goods market equilibrium and accumulation identity
The goods market equilibrium requires that supply equals demand, 1
t
= C
t
+ 1
t
.
where demand is given by consumption plus gross investment. Next periods capital
stock is - by an accounting identity - given by 1
t1
= 1
t
+ (1 o) 1
t
such that net
investment amounts to the change in the capital stock, 1
t1
1
t
= 1
t
o1
t
. given by
gross investment 1
t
minus depreciation, where o is the depreciation rate. Replacing
gross investment by the goods market equilibrium, we obtain the resource constraint
1
t1
= (1 o) 1
t
+1
t
C
t
. (2.26)
For our OLG setup, it is useful to slightly rewrite this constraint,
1
t
+ (1 o) 1
t
= C
t
+1
t1
. (2.27)
In this formulation, it reects the goods market equilibrium where the left-hand side
shows supply as current production and capital held by the old. The old sell capital
as it is of no use for them, given that they will not be able to consume anything
next period. Demand for the aggregate good is given by aggregate consumption (i.e.
consumption of the young plus consumption of the old) plus the capital stock to be
held next period by the currently young.
2.4.4 The reduced form
Once all maximization problems are solved, the objective consists in understanding
properties of the model, i.e. understand its predictions. This is usually done by rst
17
simplifying the structure of the system of equations coming out of the model as much
as possible. In the end, after inserting and reinserting, a system of : equations in :
unknowns results.
Ideally, there is only one equation left and this equation gives an explicit solution
of the endogenous variable. In static models, an example would be 1
A
= o1. i.e.
employment in sector A is given by a utility parameter o times total exogenous
labour supply 1. This would be an explicit solution. If one is left with just one
equation but one obtains on an implicit solution, one would obtain something like
, (1
A
. o. 1) = 0.
Deriving the reduced form
We now derive, given the results we obtained by now, how large the capital stock
in the next period is. Splitting aggregate consumption into consumption of the young
and consumption of the old and using the output-factor reward identity (2.22) for the
resource constraint in the OLG case (2.27), we obtain
n
1
t
1
t
+n
1
t
1 + (1 o) 1
t
= C
j
t
+C
c
t
+1
t1
.
Dening the interest rate :
t
as the dierence between factor rewards n
1
t
for capital
and the depreciation rate o.
:
t
= n
1
t
o. (2.28)
we nd
:
t
1
t
+n
1
t
1 +1
t
= C
j
t
+C
c
t
+1
t1
.
The interest rate denition (2.28) shows the net income of capital owners per unit of
capital. They earn the gross factor rewards n
1
t
but, at the same time, they experience
a loss from depreciation. Net income therefore only amounts to :
t
. As the old consume
the capital stock plus interest c
c
t
1 = (1 +:
t
)1
t
4
, we obtain
1
t1
= n
t
1 C
j
t
= :
t
1. (2.29)
which is the aggregate version of the savings equation (2.25). Hence, we have found
that savings :
t
of young at t is the capital stock at t + 1.
Note that equation (2.29) is often present on intuitive grounds. The old in
period t have no reason to save as they will not be able to use their savings in t + 1.
Hence, only the young will save and the capital stock in t + 1. being made up from
savings in the previous period, must equal to the savings of the young.
The one-dimensional dierence equation
In our simple dynamic model considered here, we obtain indeed the ideal case
where we are left with only one equation that gives us the solution for one variable,
4
Now we see that the interest rate in (2.24) must refer to period t + 1.
18
the capital stock. Inserting the individual savings equation (2.25) into (2.29) gives
with the rst-order condition (2.19) of the rm
1
t1
= (1 ) n
1
t
1 = (1 )
J1 (1
t
. 1)
J1
1. (2.30)
The rst equality shows that a share 1 of labour income becomes capital in the
next period. Interestingly, the depreciation rate does not have an impact on the
capital stock in period t + 1. Economically speaking, the depreciation rate aects
wealth of the old but - with logarithmic utility - not saving of the young.
2.4.5 Properties of the reduced form
Equation (2.30) is a dierence equation in 1
t
. All other quantities in this equation
are constant. This equation determines the entire path of capital in this dynamic
economy, provided we have an initial condition 1
0
. We have therefore indeed solved
the maximization problem and reduced the general equilibrium model to one single
equation. From the path of capital, we can compute all other variables which are of
interest for our economic questions.
Whenever we have reduced a model to its reduced form and have obtained one or
more dierence equations (or dierential equations in continuous time), we would like
to understand the properties of such a dynamic system. The procedure is in principle
always the same: We rst ask whether there is some solution where all variables (1
t
in our case) are constant. This is then called a steady state analysis. Once we have
understood the steady state (if there is one), we want to understand how the economy
behaves out of the steady state, i.e. what its transitional dynamics are.
Steady state
In the steady state, the capital stock 1
is constant and determined by

1
= (1 )
J1 (1
. 1)
J1
1. (2.31)
All other variables like aggregate consumption, interest rates, wages etc. are constant
as well. Consumption when young and when old can dier as in a setup with nite
lifetimes the interest rate in the steady state does not need to equal the time preference
rate of households.
Transitional dynamics
Dynamics of the capital stock are illustrated in gure 5. The gure plots the
capital stock in period t on the horizontal axis. The capital stock in the next period,
1
t1
. is plotted on the vertical axis. The law of motion for capital from (2.30) then
shows up as the curve in this gure. The 45
line equates 1
t1
to 1
t
.
We start from our initial condition 1
0
. Equation (2.30) or the curve in this gure
then determine the capital stock 1
1
. This capital stock is then viewed as 1
t
such
19
that, again, the curve gives us 1
t1
. which is, given that we now started in 1. the
capital stock 1
2
of period 2. We can continue doing so and see graphically that the
economy approaches the steady state 1
which we had computed in (2.31).

N
N
1
t
0
1
t1
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
1
0
_
'
_
'
1
t
= 1
t1
1
t1
= (1 c)n(1
t
)1
Figure 5 Convergence to the steady state
Summary
We started with a description of technologies in (2.18), preferences in (2.23) and
factor endowment given by 1
0
. With behavioural assumptions concerning utility and
prot maximization and perfect competition on all markets plus a description of mar-
kets in 2.4.3 and some juggling of equations, we ended up with a one-dimensional
dierence equation (2.30) which describes the evolution of the economy over time and
in the long-run steady state. Given this formal analysis of the model, one could now
start answering economic questions.
2.5 More on dierence equations
2.5.1 Two useful proofs
The following result will be useful in what follows. As the proof of this result also
has an esthetic value, there will be a second proof of another result to be done in the
exercises.
Lemma 1 For any c ,= 1.
T
i=1
c
i
= c
1 c
T
1 c
.
T
i=0
c
i
=
1 c
T1
1 c
Proof. The left hand side is given by
T
i=1
c
i
= c +c
2
+c
S
+. . . +c
T1
+c
T
. (2.32)
Multiplying this sum by c yields
c
T
i=1
c
i
= c
2
+c
S
+. . . +c
T
+c
T1
. (2.33)
20
Now subtract (2.33) from (2.32) and nd
(1 c)
T
i=1
c
i
= c c
T1
=
T
i=1
c
i
= c
1 c
T
1 c
. (2.34)
Lemma 2
T
i=1
ic
i
=
1
1 c
_
c
1 c
T
1 c
1c
T1
_
The proof is left as exercise 7.
2.5.2 The simplest dierence equation
The simplest dierence equation is
r
t1
= cr
t
. c 0. (2.35)
This equation appears too simple to be worth being analysed. We do it here as we
get to know the standard steps in analysing dierence equations which we will also
use for more complex dierence equations. The objective here is therefore not this
dierence equation c: :ic/ but what is done with it.
Solution by inserting
The simplest way to solve it is simply to insert and reinsert this equation su-
ciently often. Doing it three times gives
r
1
= cr
0
.
r
2
= cr
1
= c
2
r
0
.
r
S
= cr
2
= c
S
r
0
.
When we look at this solution for t = 3 long enough, we see that the general solution
is
r
t
= c
t
r
0
.
This could formally be proven by a proof by induction.
Examples for solutions
The sequence of r
t
given by this solution, given dierent initial conditions r
0
.
are shown in the following gure for c 1. The parameter values chosen are c = 2,
r
0
0.5. 1. 2 and t runs from 0 to 10.
21
0 2 4 6 8 10
0
2
4
6
8
10
12
14
expwa.m
Figure 6 Solution to a dierence equation for c 1
Long-run behaviour
We can now ask whether r
t
approaches a constant when time goes to innity.
This gives
lim
t!1
r
t
= r
0
lim
t!1
c
t
=
_
_
_
0
r
0
_
_
_
=
_
_
_
c < 1
c = 1
c 1
.
Hence, r
t
approaches a constant only for c < 1.
A graphical analysis
For more complex dierence equations, it often turns out to be useful to analyze
their behaviour in a phase diagram. Even though this simple dierence equation
can be understood easily analytically, we will illustrate its properties in the following
gure. This allows to understand how analyses of this type can be undertaken also
for more complex dierence equations.
N
Z
Z
Z
Z
Z
Z
Z
Z
Z
Z
Z
Z
Z
Z
Z
N
r
t
r
0
r
t
N
Z
Z
Z
Z
Z
Z
Z
Z
Z
Z
Z
Z
Z
Z
r
t
= r
t1
N
r
t1
= cr
t
r
t
= r
t1
r
t1
= cr
t
r
0
_
_
_
r
t1
r
t1
_
'
'
_
22
Figure 7 A phase diagram for c < 1 in the left and c 1 in the right panel
The left panel shows how r
t
evolves over time, starting at r
0
. In this case of c < 1.
we see how r
t
approaches zero. When we graphically illustrate the case of c 1. the
evolution of r
t
is as shown in the right panel.
2.5.3 A slightly less but still simple dierence equation
We now consider a slightly more general dierence equation. Compared to (2.35), we
just add a constant / in each period,
r
t1
= cr
t
+/. c 0. (2.36)
Solution by inserting
We solve again by inserting. In contrast to the solution for (2.35), we start from
an initial value of r
t
. Hence, we imagine we are in t (t as today) and compute what
the level of r will be tommorrow and the day after tommorrow etc. We nd for r
t2
and r
tS
that
r
t2
= cr
t1
+/ = c [cr
t
+/] +/ = c
2
r
t
+/ [1 +c] .
r
tS
= c
S
r
t
+/
_
1 +c +c
2
.
This suggests that the general solution is
r
ta
= c
a
r
t
+/
a1
i=0
c
i
= c
a
r
t
+/
c
a
1
c 1
.
The last equality used the rst lemma from ch. 2.5.1.
Limit for : and c < 1
The limit for : going to innity and c < 1 is given by
lim
a!1
r
ta
= lim
a!1
c
a
r
t
+/
c
a
1
c 1
=
/
1 c
. (2.37)
A graphical analysis
The left panel of the following gure studies the evolution of r
t
for the stable case,
i.e. where 0 < c < 1 and / 0. Starting today in t with r
t
. we end up in r
. As
we chose c smaller than one and a positive /. r
is positive as (2.37) shows. We will

return to the right panel in a moment.
23
N
N N
r
t1
N
Z
Z
Z
Z
Z
Z
Z
Z
Z
Z
Z
Z
Z
Z
Z
Z
Z
Z
Z
Z
Z
Z
Z
Z
Z
Z
Z
Z
Z
Z
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
r
t
r
_
'
_
_
'
_
'
_
r
t
r
r
t
= r
t1
r
t1
= cr
t
+/
'
_
'
_
r
t
= r
t1
r
t1
= cr
t
+/
r
t1
Figure 8 Phase diagrams for a more general dierence equation
2.5.4 Fix points and stability
Denition
We can use these examples to dene two concepts that will be useful also at later
stages.
Denition 1 (Fixpoint) A xpoint r
of a function , (r) is dened by

r
= , (r
) . (2.38)
For dierence equations of the type r
t1
= , (r
t
) . the xpoint r
of the function
, (r
t
) is the point where r stays constant. This is usually called the long-run equilib-
rium point of some economy or its steady or stationary state. Whenever an economic
model, represented by its reduced form, is analysed, it is generally useful to rst try
and nd out whether xpoints exist and what its economic properties are.
For the dierence equation of last section, we obtain
r
t1
= r
t
= r
==r
= cr
+/ ==r
=
/
1 c
.
Local and global stability
Once a xpoint has been identied, one can ask whether it is stable.
Denition 2 (Global stability) A xpoint r
is globally stable if, starting from an

initial value r
0
,= r
. r
t
converges to r
.
The concept of global stability usually refers to initial values r
0
that are econom-
ically meaningful. An initial physical capital stock that is negative would not be
considered to be economically meaningful.
Denition 3 (Local stability and instability) A xpoint r
is
_
unstable
locally stable
_
if,
starting from an initial value r
+. where is small, r
t
_
diverges from
converges to
_
r
.
24
For illustration purposes consider the xpoint r
in the left panel of g. 8 - it is

globally stable. In the right panel of the same gure, it is unstable. As can easily
be seen, the instability follows from simply letting the r
t1
line intersect the 45
-line
from below. In terms of the underlying dierence equation (2.36), this requires / < 0
and c 1.
Clearly, economic systems can be much more complex and generate several x-
points. Imagine the link between r
t1
and r
t
is not linear as in (2.36) but nonlinear,
r
t1
= , (r
t
) . Unfortunately for economic analysis, a nonlinear relationship is the
much more realistic case. The next gure provides an example for some function
, (r
t
) that implies an unstable r
&
and a locally stable xpoint r
c
.
N
N
r(t)
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
r
01
r
02
r
c
r
0S
r
&
r
t1
_
'
_
_
'
_
_
'
_
Figure 9 A locally stable xpoint r
c
and an unstable xpoint r
&
2.5.5 An example: Deriving a budget constraint
A frequently encountered dierence equation is the budget constraint. We have
worked with budget constraints at various points before but we have hardly spent
thoughts about their origin. We more or less simply wrote them down. Budget
constraints, however, are tricky objects, at the latest when we think about general
equilibrium setups. What is the asset we save in? Is there only one asset or are
there several? What is the price of this asset? How does it relate to the price of the
consumption good, i.e. do we express the value of assets in real or nominal terms?
This section will derive a budget constraint. We assume that there is only one
asset. The price of one unit of the asset will be denoted by
t
. Its relation to the
price j
t
of the consumption good will be left unspecied, i.e. we will discuss the
most general setup which is possible for the one-asset case. It diers from the budget
constraint in ch. 3.6.3 where there was also only one asset, capital 1. but the price of
one unit of capital was equal to the price of the output good and this price was unity
as the output good was the numeraire. The budget constraint with these simplifying
assumptions was relatively simple - as can be seen in (3.33). Here, we will see that a
budget constraint will become pretty complex as soon as the price of the asset is not
normalized. This complexity, however, is needed when it comes e.g. to capital asset
pricing - see further below in ch. 8.4.
25
Deriving a budget constraint is in principle straightforward. One just denes
wealth of the household (taking into account which types of assets the household can
hold for saving purposes and what their prices are), computes the dierence between
wealth today and tommorrow (this is where the dierence equation aspect comes
in) and uses an equation which relates current savings to current changes in the
number of assets. In a nal step, one will naturally nd out how the interest rate
showing up in budget constraints relates to more fundamental quantities like value
marginal products and depreciation rates.
In our one-asset case, wealth c
t
of a household is given by the number /
t
of stocks
the household owns (say the number of machines she owns) times the price
t
of one
stock (or machine), c
t
=
t
/
t
. Computing the rst dierence yields
c
t1
c
t
=
t1
/
t1
t
/
t
=
t1
(/
t1
/
t
) + (
t1
t
) /
t
. (2.39)
Wealth changes depend on the acquisition
t1
(/
t1
/
t
) of new assets and on
changes in the value of assets that are already held, (
t1
t
) /
t
.
Savings are given by capital income (net of depreciation losses) and labour income
minus consumption expenditure, n
1
t
/
t
o
t
/
t
+ n
t
j
t
c
t
. Savings in t are used for
buying new assets in t for which the period-t price needs to be paid,
n
1
t
/
t
o
t
/
t
+n
t
j
t
c
t
t
= /
t1
/
t
. (2.40)
We can rewrite this equation slightly, which will simplify the interpretation of subse-
quent results, as
/
t1
= (1 o) /
t
+
n
1
t
/
t
+n
t
j
t
c
t
t
.
Wealth in the next period expressed in number of stocks (and hence not in nominal
terms) is given by wealth which is left over from the current period, (1 o) /
t
, plus
new acquisitions of stocks which amount to gross capital plus labour income minus
consumption expenditure divided by the price of one stock. This is a dierence
equation in /
t
but not yet a dierence equation in wealth c
t
.
Inserting this (2.40) into (2.39) yields
c
t1
c
t
=

t1
t
_
n
1
t
/
t
o
t
/
t
+n
t
j
t
c
t
_
+ (
t1
t
) /
t
=

t1
t
_
n
1
t
t
c
t
oc
t
+n
t
j
t
c
t
_
+
_
t1
t
1
_
c
t
=
c
t1
=

t1
t
__
1 +
n
1
t
t
o
_
c
t
+n
t
j
t
c
t
_
. (2.41)
What does this equation tell us? Each unit of wealth c
t
(say Euro, Dollar, Yen
...) gives 1 o units at the end of the period as o% is lost due to depreciation
plus dividend payments n
1
t
,
t
. Wealth is augmented by labour income minus
consumption expenditure. This end-of-period wealth is expressed in wealth c
t1
at
26
the beginning of the next period by dividing it through
t
(which gives the number
/
t
of stocks at the end of the period) and multiplying it by the price
t1
of stocks in
the next period. We have thereby obtained a dierence equation in c
t
.
This general budget constraint is fairly complex, however, which implies that in
practice it is often expressed dierently. One possibility consists in choosing the
capital good as the numeriare good and setting
t
= 1 \t. This simplies (2.41) to
/
t1
= (1 +:
t
) /
t
+n
t
j
t
c
t
.
which is very similar to (3.33) from ch. 3.6.3. The simplication in this expression
consists also in the denition of the interest rate :
t
as :
t
= n
1
t
o.
An alternative consists in equalizing the price of the capital good to the price of the
consumption good, j
t
=
t
. This is often the case in general equilibrium approaches
- think e.g. of growth models - when there is an aggregate technology which is used
both for producing the consumption and the investment good. The constraint (2.41)
then simplies to
c
v
t1
= (1 +:
t
) c
v
t
+
n
t
j
t
c
t
.
where real wealth and the interest rate are dened by c
v
t
=
o
I
j
I
and :
t
=
&
1
I
j
I
o,
respectively.
If one is not willing to x any prices or relative prices - which should indeed not
be done if one is interested e.g. in capital asset pricing issues, to be treated later in
ch. 8.4 - one can write the somehow cumbersome budget constraint (2.41) in a more
elegant way. To this end, divide by
t1
and use the denition of c
t
to get the number
of stocks held tommorrow as
/
t1
=
_
1 +
n
1
t
t
o
_
/
t
+
n
t
t

j
t
t
c
t
. (2.42)
Rearranging such that expenditure is on the left- and disposable income on the right-
hand side yields
j
t
c
t
+
t
/
t1
=
t
/
t
+
_
n
1
t

t
o
_
/
t
+n
t
.
This equation has a simpler form than (2.41) and it also lends itself to a simple in-
terpretation: On the left-hand side is total expenditure in period t. consisting of con-
sumption expenditure j
t
c
t
plus expenditure for buying the number of capital goods,
/
t1
, the household wants to hold in t + 1. As this expenditure is made in t. total
expenditure for capital goods amounts to
t
/
t1
. The right-hand side is total dispos-
able income which splits into income
t
/
t
from selling all capital inherited from the
previous period, capital income
_
n
1
t

t
o
_
/
t
and labour income n
t
. This is the form
budget constraints are often expressed in capital asset pricing models. Note that this
is in principle also a dierence equation in /
t
.
2.6 Exercises chapter 2 and further reading
Rates of substition are discussed in many books on microeconomics; see e.g. Mas-
Colell, Whinston and Green (1995) or Varian (1992). The OLG model goes back to
27
Samuelson. For presentations in textbooks, see e.g. Blanchard and Fischer (1989),
Azariadis (1993) or de la Croix and Michel (2002). The presentation of the Lagrangian
is inspired by Intriligator (1971). Treatments of shadow prices are available in many
other textbooks (Dixit, 1989, ch. 4; Intiriligator, 1971, ch. 3.3). More extensive
treatments of dierence equations and the implicit function theorem can be found in
many introductory mathematics for economists books.
28
Exercises chapter 2
Optimal consumption in two-period discrete time models
1. Optimal choice of household consumption
Consider the following maximization problem,
max
c
I
,c
I+1
l
t
= (c
t
) +
1
1 +j
(c
t1
) (2.43)
subject to
n
t
+ (1 + :)
1
n
t1
= c
t
+ (1 + :)
1
c
t1
.
(a) What is the optimal consumption path?
(b) Under what conditions does consumption rise?
2. Capital market restrictions
Now consider the following budget constraint. This is a budget constraint that
would be appropriate if you want to study the education decisions of households.
The parameter / amounts to schooling costs. Inheritance of this individual
under consideration is :.
l
t
= ln c
t
+ (1 ) ln c
t1
subject to
/ +: + (1 + :)
1
n
t1
= c
t
+ (1 + :)
1
c
t1
.
(a) What is the optimal consumption prole under no capital market restric-
tions?
(b) Assume loans for nancing education are not available, hence savings need
to be positive, :
t
_ 0. What is the consumption prole in this case?
3. Optimal investment
Consider a monopolist investing in its technology. Technology is captured by
marginal costs c
t
. The chief accountant of the rm has provided the manger of
the rm with the following information,
= :
1
+1:
2
. :
t
= j (r
t
) r
t
c
t
r
t
1
t
. c
t1
= c
t
, (1
1
) .
Assume you are the manager. What is the optimal investment sequence 1
1
, 1
2
?
29
4. A particular utility function
Consider the utility function l = c
t
+ ,c
t1
. where 0 < , < 1. Maximize l
subject to an arbitrary budget constraint of your choice. Derive consumption
in the rst and second period. What is strange about this utility function?
5. Intertemporal elasiticity of substitution
Consider the utility function l = c
1o
t
+,c
1o
t1
.
(a) What is the intertemporal elasticity of substitution?
(b) How can the denition in (2.10) of the elasticity of substitution be trans-
formed into the maybe better known denition
c
i)
=
d ln (c
i
,c
)
)
d ln
_
n
c
.
,n
c
_ =
j
i
,j
)
c
i
,c
)
d (c
i
,c
)
)
d (j
i
,j
)
)
?
6. General equilibrium
Consider the Diamond model for a Cobb-Douglas Production function of the
form 1 = 1
1
1
and a logarithmic utility function.
(a) Derive the dierence equation for 1
t
.
(b) Draw a phase diagram.
(c) What are the steady state consumption level and capital stock?
7. Sums
(a) Proof the statement of the second lemma in ch. 2.5.1,
T
i=1
ic
i
=
1
1 c
_
c
1 c
T
1 c
1c
T1
_
.
The idea is identical to the rst proof in ch. 2.5.1.
(b) Show that (exam)
I1
c=0
c
I1c
1
i
c
=
c
I
1
i
I
c
1
i
.
Both parameters obey 0 < c
1
< 1 and 0 < < 1. Hint: Rewrite the sum
as c
I1
1

I1
c=0
(i,c
1
)
c
and observe that the rst lemma in ch. 2.5.1 holds for
c which are larger or smaller than 1.
8. Dierence equations
Consider the following linear dierence equation system
t1
= c
t
+/. c < 0 < /.
(a) What is the xpoint of this equation?
(b) Is this point stable?
(c) Draw a phase diagram.
30
3 Innite horizon models
This chapter looks at decision processes where the time horizon is longer than two
periods. In most cases, the planning horizon will be innity. In such a context,
Bellmans optimality principle is very useful. Is it, however, not the only way how to
solve maximization problems with innite time horizon. For comparison purposes, we
therefore start with the Lagrange approach, as in the last section. Bellmans principle
will be introduced afterwards when intuition for the problem and relationships will
have been increased.
3.1 The problem
The objective function is given by the utility function of an individual,
l
t
=
1
t=t
,
tt
n(c
t
) . (3.1)
where again
, = (1 +j)
1
. j 0 (3.2)
is the discount factor and j is the positive time preference rate. We know this utility
function already from the denition of the time preference rate, see (2.11). The
utility function is to be maximized subject to a budget constraint. The dierence to
the formulation in the last section is that consumption does not have to be determined
for two periods only but for innitely many. Hence, the individual does not choose
one or two consumption levels but an entire path of consumption. This path will be
denoted by c
t
. where t [t. [ . Note that the utility function is a generalization of
the one used above in (2.1), but is assumed to be time separable. The corresponding
two period utility function was used in exercise set 1, cf. equation (2.43).
The budget constraint can be expressed in the intertemporal version by
1
t=t
(1 +:)
(tt)
c
t
= c
t
+
1
t=t
(1 +:)
(tt)
n
t
. (3.3)
where c
t
= j
t
c
t
. It states that the present value of expenditure equals current wealth
c
t
plus the present value of labour income n
t
. Labour income n
t
and the interest rate
: are exogenously given to the household, her wealth level c
t
is given by history. The
only quantity that is left to be determined is therefore the path c
t
. Maximizing
(3.1) subject to (3.3) is a standard Lagrange problem.
3.2 Solving by Lagrange
The Lagrangian reads
/ =
1
t=t
,
tt
n(c
t
) +`
_
1
t=t
(1 +:)
(tt)
c
t
c
t
1
t=t
(1 +:)
(tt)
n
t
_
.
where ` is the Lagrange mulitplier. It is constant as before. First order conditions
are
/
c
:
= ,
tt
n
0
(c
t
) +` [1 +:]
(tt)
j
t
= 0. t _ t < . (3.4)
/
A
= 0.
31
where the latter is, as in the OLG case, the budget constraint. Again, we have as
many conditions as variables to be determined.
Do these rst-order conditions tell us something? Take the rst-order condition
for period t and for period t + 1. They read
,
tt
n
0
(c
t
) = `[1 +:]
(tt)
j
t
.
,
t1t
n
0
(c
t1
) = ` [1 +:]
(t1t)
j
t1
.
Dividing them gives
,
1
n
0
(c
t
)
n
0
(c
t1
)
= (1 +:)
j
t
j
t1
==
n
0
(c
t
)
,n
0
(c
t1
)
=
j
t
(1 +:)
1
j
t1
. (3.5)
Rearranging allows an intuitive interpretation: Comparing the instantaneous gain in
utility n
0
(c
t
) with the future gain, discounted at the time preference rate, ,n
0
(c
t1
) .
must yield the same ratio as the price j
t
that has to be paid today relative to the price
that has to be paid in the future, also appropriately discounted to its present value
price (1 +:)
1
j
t1
. This interpretation is identical to the two-period interpretation
in (2.9) in ch. 2.2.2. If we normalize prices to unity, (3.5) is just the expression we
obtained in the solution of the OLG example (2.43).
3.3 The envelope theorem
We saw how the Lagrangian can be used to solve optimization problems with many
time periods. In order to understand how dynamic programming works, it is use-
ful to understand a theorem which is frequently used when employing the dynamic
programming method: the envelope theorem.
3.3.1 The theorem
In general, the envelope theorem says
Theorem 1 Let there be a function q (r. ) . Choose r such that q (.) is maximized
for a given . (Assume q (.) is such that a smooth interior solution exists.) Let ,()
be the resulting function of .
, () = max
a
q (r. ) .
Then the derivative of , with respect to equals the partial derivative of q with respect
to , if q is evaluated at that r = r() that maximizes q.
d , ()
d
=
J q (r. )
J
a=a(j)
.
Proof. , () is constructed by
0
0a
q (r. ) = 0. This implies a certain r = r () .
provided that second order conditions hold. Hence, , () = max
a
q (r. ) = q (r () . ) .Then,
o )(j)
o j
=
0 j(a(j),j)
0 a
o a(j)
o j
+
0 j(a(j),j)
0 j
. The rst term of the rst term is zero.
32
3.3.2 Illustration
The plane depicts the function q (r. ). The maximum of this function with respect
to r is shown as max
a
q (r. ), which is , (). Given this gure, it is obvious that the
derivative of , () with respect to is the same as the partial derivative of q (.) with
respect to at the point where q (.) has its maximum with respect to r: The partial
derivative
0j
0j
is the derivative in the direction of (Richtungsableitung). Choosing
the highest point of q (.) with respect to r, this Richtungsableitung must be the
same as
o)(j)
oj
at the back of the gure.
( ) f y
x
y
( ) max ,
x
g x y
( ) g x y ,
( ) ( ) ( ) ( ) ( )
( ) ( )
df y
dy
g x y y
x
dx
dy
g x y y
y
g x y y
y
= +
=
, ,
,
Figure 10 Illustrating the envelope theorem
5
3.3.3 An example
There is a central planner of an economy. The social welfare function is given by
l (. 1), where and 1 are consumption goods. The technologies available for
producing these goods are = (c1
) and 1 = 1(1
1
) and the economys resource
constraint is 1
+1
1
= 1.
5
This gure is contained in the le envelope.ds4. It was originally created in Mathematica, using
the command Plot3D[-(x-.5*y)
^
2-(y-2)
^
2,x,0,2,y,0,4, Axes -> noaxes, PlotRange ->-8,5,
Boxed -> False]
33
The planner is interested in maximizing the social welfare level and allocates
labour according to max
1
/
l ((c1
) . 1(1 1
)) . The optimality condition is

Jl
J
0
c
Jl
J1
1
0
= 0. (3.6)
This makes 1
a function of c,
1
= 1
(c) . (3.7)
The central planner now asks what happens to the social welfare level when the
technology parameter c increases and she still maximizes the social welfare function,
i.e. (3.6) continues to hold. Without using the envelope theorem, the answer is
d
dc
l ((c1
(c)) . 1(1 1
(c)))
= l
(.)
0
[1
(c) +c1
0
(c)] +l
1
(.) 1
0
[1
0
(c)] = l
(.)
0
1
(c) 0
where the last equality follows from inserting the optimality condition (3.6). Eco-
nomically, this result means that the eect of a better technology on overall welfare
is given by the direct eect in sector . The indirect eect through reallocation of
labour vanishes as due to the rst-order condition (3.6) the marginal contribution of
each worker is identical across sectors. Clearly, this only holds for a small change in
c.
If one is interested in nding an answer by using the envelope theorem, one would
start by dening a function \ (c) = max
1
/
l ((c1
) . 1(1 1
)) . Then, according
to the envelope theorem,
d
dc
\ (c) =
J
Jc
l ((c1
) . 1(1 1
))
1
/
=1
/
(c)
= l
(.)
0
1
1
/
=1
/
(c)
= l
(.)
0
1
(c) 0.
Apperently, both approaches yield the same answer. Applying the envelope theorem
gives the answer faster.
3.4 Solving by dynamic programming
We will not get to know how dynamic programming works. Let us study a slightly
more general maximization problem than the one in ch. 3.1. We use the same utility
function as in (3.1).
6
The constraint, however, will be represented in a more general
way than in (3.3). We stipulate that wealth of the household evolves according to
r
t1
= , (r
t
. c
t
) . (3.8)
We denote wealth by r
t
to make clear that this constraint can represent also other
aspects than a budget constraint of a household. In fact, this dierence equation
6
All intertemporal utility functions in this book will use exponential discounting. This is clearly
a special case. Models with non-exponential or hyperbolic discounting imply fundamentally dierent
dynamic behaviour and time inconsistencies.
34
can be non-linear (e.g. in central planner problem where the constraint is a resource
constraint like (3.48)) or linear (e.g. a budget constraint like (3.14)). We treat here
this general case rst before we go to more specic examples further below.
The consumption level c
t
and - more generally speaking - other variables whose
value is directly chosen by individuals, e.g. investment levels or shares of wealth
held in dierent assets, are called control variables. Variables which are not under
the direct control of individuals are called state variables. In many maximization
problems, state variables depend indirectly on the behaviour of individuals as in
(3.8). State variables can also be completely exogenous like e.g. the TFP level in an
exogenous growth model.
Optimal behaviour is dened by max
fc
:
g
l
t
subject to (3.8), where the value of
this optimal behaviour or optimal program is denoted by
\ (r
t
) = max
fc
:
g
l
t
subject to r
t1
= , (r
t,
c
t
) . (3.9)
\ (r
t
) is called the value function. It is a function of the state variable r
t
and not
of the control variable c
t
. The latter point is easy to understand if one takes into
account that the control variable c
t
is, when behaving optimally, a function of the
state variable r
t
.
The value function \ (.) could also be a function of time t (e.g. in problems with
nite horizon) but we will not discuss this further as it is of no importance in the
problems we will encounter. Generally speaking, r
t
and c
t
could be vectors and time
could then be part of the state vector r
t
. The value function is always a function of
the states of the system or the maximization problem.
Given this description of the maximization problem, solving by dynamic program-
ming essentially requires to go through three steps. This three-step approach will be
followed here, in continuous time later, and also in models with uncertainty.
DP1: Bellman equation and rst-order conditions
The rst step establishes the Bellman equation and computes rst-order condi-
tions. The objective function l
t
in (3.1) is additively separable which means that it
can be written in the form
l
t
= n(c
t
) +,l
t1
. (3.10)
Bellmans idea consists in rewriting the maximization problem in the optimal program
(3.9) as
\ (r
t
) = max
c
I
n(c
t
) +,\ (r
t1
) (3.11)
subject to
r
t1
= , (r
t
. c
t
) .
Equation (3.11) is known as Bellman equation. In this equation, the problem
with potentially innitely many control variables c
t
was broken down in many
problems with one control variable c
t
. Note that there are two steps involved: First,
the additive separability of the objective function is used. Second, more importantly,
35
l
t
is replaced by \ (r
t1
). This says that we assume that the optimal problem for
tomorrow is solved and we worry about the maximization problem of today only.
We can now compute the rst-order condition which is of the form
n
0
(c
t
) +,\
0
(r
t1
)
J, (r
t
. c
t
)
Jc
t
= 0. (3.12)
In principle, this is the solution of our maximization problem. Our control variable
c
t
is by this expression implicitly given as a function of the state variable, c
t
= c (r
t
) .
as r
t1
by the constraint (3.8) is a function of r
t
and c
t
. Hence, as \ is well-dened
above, we have obtained a solution.
As we know very little about the properties of \ at this stage, however, we need
to go through two further steps in order to eliminate \ from this rst-order condition
and obtain a condition that uses only functions (like e.g. the utility function or the
technology for production in later examples) of which properties like signs of rst and
second derivatives are known.
DP2: Evolution of the costate variable
The second step of the dynamic programming approach starts from the maxi-
mized Bellman equation. The maximized Bellman equation is obtained by replacing
the control variables in the Bellman equation, i.e. the c
t
in (3.11) in the present case,
by the optimal control variables as given by the rst-order condition, i.e. by c (r
t
)
resulting from (3.12). Logically, the max operator disappears (as we insert the c (r
t
)
which imply a maximimum) and the maximized Bellman equation reads
\ (r
t
) = n(c (r
t
)) +,\ (, (r
t
. c(r
t
))) .
The derivative with respect to r
t
reads
\
0
(r
t
) = n
0
(c (r
t
))
dc (r
t
)
dr
t
+,\
0
(, (r
t
. c (r
t
)))
_
,
a
I
+,
c
dc (r
t
)
dr
t
_
.
This step shows why it matters that we use the maximized Bellman equation here:
Now control variables are a function of state variable and we need to compute the
derivative of c
t
with respect to r
t
when computing the derivative of the value function
\ (r
t
) . Inserting the rst-order condition gives
\
0
(r
t
) = ,\
0
(, (r
t
. c
t
(r
t
))) ,
a
I
= ,\
0
(r
t1
) ,
a
I
(3.13)
This equation is a dierence equation for the costate variable \
0
(r
t
). This derivative
of the value function is the shadow price of the state variable r
t
. It says how much
an additional unit of the state variable (say e.g. of wealth) is valued: As \ (r
t
) gives
the value of optimal behaviour between t and the end of the planning horizon, \
0
(r
t
)
says by how much this value changes when r
t
is changed marginally. Hence, equation
(3.13) describes how the shadow price of the state variable changes over time when
the agent behaves optimally. If we had used the envelope theorem, we would have
immediately ended up with (3.13) without having to insert the rst-order condition.
36
DP3: Inserting rst-order conditions
Now insert the rst-order condition (3.12) twice into (3.13) to replace the unknown
value function and to nd an optimality condition depending on n
t
only. This will
then be the Euler equation. We do not do this here explicitly as many examples will
go through this step in detail in what follows.
3.5 Examples
3.5.1 Individual utility maximization
The individuals budget constraint is given in the dynamic formulation (in contrast
to an intertemporal version (3.3)) by
c
t1
= (1 +:
t
) (c
t
+n
t
c
t
) . (3.14)
Note that this dynamic formulation corresponds to the intertemporal version in the
sense that (3.3) implies (3.14) and (3.14) with some limit condition implies (3.3). The
rst part of this statement can easily be seen: Write (3.3) for the next period as
1
t=t1
(1 +:)
(tt1)
c
t
= c
t1
+
1
t=t1
(1 +:)
(tt1)
n
t
. (3.15)
Express (3.3) as
c
t
+
1
t=t1
(1 +:)
(tt)
c
t
= c
t
+n
t
+
1
t=t1
(1 +:)
(tt)
n
t
=
c
t
+ (1 + :)
1
1
t=t1
(1 +:)
(tt1)
c
t
= c
t
+n
t
+ (1 + :)
1
1
t=t1
(1 +:)
(tt1)
n
t
=
1
t=t1
(1 +:)
(tt1)
c
t
= (1 +:) (c
t
+n
t
c
t
) +
1
t=t1
(1 +:)
(tt1)
n
t
.
Insert (3.15) and nd (3.14). The second part is part of the exercises.
The budget constraint (3.14) can be found e.g. in Blanchard and Fischer (1989, p.
280). The timing as implicit in (3.14) is illustrated in the following gure. All events
take place at the beginning of the period. Our individual owns a certain amount
of wealth c
t
at teh beginning of t and receives here wage income n
t
and spends c
t
on consumption also at the beginning. Hence, savings :
t
can be used during t for
production and interest are paid on :
t
which in turn gives c
t1
at the beginning of
period t + 1.
t
t +1
c
t +1 c
t
( ) a r s
t t t +
= +
1
1 s a w c
t t t t
= +
Figure 11 The timing in an innite horizon discrete time model
37
The consistency of (3.14) with technologies in general equilibrium is not self-
evident. We will encounter more conventional budget constraints of the type (??)
further below. For understanding how to apply dynamic programming methods,
however, this budget constraint is perfectly ne.
The objective of the individual is to maximize her utility function (3.1) subject
to the budget constraint by choosing a path of consumption levels c
t
. denoted by
c
t
. t [t. ] . The value of the optimal program c
t
is, given her initial endow-
ment with c
t
,
\ (c
t
) = max
fc
:
g
l
t
(3.16)
It is called value function.
We know that the utility function can be written as l
t
= n(c
t
) + ,l
t1
. Now
assume that the individual behaves optimally as from t + 1. Then we can insert the
value function. The utility function reads l
t
= n(c
t
) + ,\ (c
t1
) . Inserting this in
the value function, we obtain the recursive formulation
\ (c
t
) = max
c
I
n(c
t
) +,\ (c
t1
) . (3.17)
known as the Bellman equation.
Again, this breaks down a many-period problem into a two-period problem: The
objective of the individual was max
fc
:
g
(3.1) subject to (3.14), as shown by the value
function in equation (3.16). The Bellman equation (3.17), however, is a two period
decision problem, the trade-o between consumption today and more wealth tomor-
row (under the assumption that the function \ is known). This is what is known as
Bellmans principle of optimality: Whatever the decision today, subsequent decisions
should be made optimally, given the situation tomorrow. History does not count,
apart from its impact on the state variable(s).
We now derive a rst-order condition for (3.17). It reads
d
dc
t
n(c
t
) +,
d
dc
t
\ (c
t1
) = n
0
(c
t
) +,\
0
(c
t1
)
d c
t1
dc
t
= 0.
Since dc
t1
,dc
t
= (1 +:
t
) by the budget constraint (3.14), this gives
n
0
(c
t
) , (1 +:
t
) \
0
(c
t1
) = 0. (3.18)
Again, this equation makes consumption a function of the state variable, c
t
= c
t
(c
t
) .
Following the rst-order condition (3.12) in the general example, we wrote c
t
= c (r
t
) .
i.e. consumption c
t
changes only when the state variable r
t
changes. Here, we write
c
t
= c
t
(c
t
) . indicating that there can be other variables which can inuence consump-
tion other than wealth c
t
. An example for such an additional variable in our setup
would be the wage rate n
t
or interest rate :
t
showing up in (3.14). If we had dened
the value function as having arguments c
t
. n
t
and :
t
. i.e. \ = \ (c
t
. n
t
. :
t
) . we would
have obtained a rst order condition which would allow us to write something like
c
t
= c (c
t
. n
t
. :
t
. ...) . As this is much more cumbersome from a notational point of
view, the shortcut c
t
= c
t
(c
t
) is often preferred in this book.
38
The derivative of the maximized Bellman equation reads, using the envelope the-
orem,
\
0
(c
t
) = ,\
0
(c
t1
)
Jc
t1
Jc
t
. (3.19)
We compute the partial derivative of c
t1
with respect to c
t
as the functional rela-
tionship of c
t
= c
t
(c
t
) should not (because of the envelope theorem) be taken into
account.
From the budget constraint we know that
0o
I+1
0o
I
= 1 + :
t
. Hence, the evolution of
the shadow price under optimal behaviour is described by
\
0
(c
t
) = , (1 +:
t
) \
0
(c
t1
) .
We can insert the rst-order condition (3.18) on the RHS. We can also rewrite the
rst-order condition (3.18), by lagging it one period, as , (1 +:
t1
) \
0
(c
t
) = n
0
(c
t1
)
and can insert this on the LHS. This gives
n
0
(c
t1
) ,
1
(1 +:
t1
)
1
= n
0
(c
t
) =n
0
(c
t
) = , (1 +:
t
) n
0
(c
t1
) . (3.20)
This is the same result as the one we have obtained when we used the Lagrange
method in equation (3.5).
It is also the same result as for the 2-period saving problem which we found in
OLG models - see e.g. (2.9) or (2.43) in the exercises. This might be suprising as
the planning horizons dier considerably between a 2- and an innite-period decision
problem. Apperently, whether we plan for two periods or many more, the change
between two periods is always the same when we behave optimally. It should be kept
in mind, however, that consumption levels (and not changes) do depend on the length
of the planning horizon.
3.5.2 Optimal R&D eort
Let us turn to a second example, originally inspired by Grossman and Shapiro (1986).
At some known point in time 1, a research project has to be nished. This research
project has a certain value at point 1 and we denote it by 1 like dissertation. In
order to reach this goal, a path of a certain length 1 needs to be gone through (das
Ziel ist das Ziel), think of 1 as a certain number of pages, a certain number of papers
or - probably better - a certain quality of a xed number of papers. Going through
this process of walking and writing is costly, it causes eort c
t
at each point in time
t _ t. Summing over these cost - think of them as hours worked per day - yields
eventually the desired amount of pages,
T
t=t
, (c
t
) = 1. (3.21)
39
where ,(.) is the page of quality production function: More eort means more output,
,
0
(.) 0. but any increase in eort implies a lower increase in output, ,
00
(.) < 0.
The objective function of our student is given by
l
t
= ,
Tt
1
T
t=t
,
tt
c
t
. (3.22)
The value of the completed dissertation is given by 1 and its present value is obtained
by discounting at the discount factor ,. Total utility l
t
stems from this present value
minus the present value of research cost c
t
. The maximization problem consists in
maximizing (3.22) subject to the constraint (3.21) by choosing an eort path c
t
.
The question now arises how these costs are optimally spread over time. How many
hours should be worked per day?
An answer can be found by using the Lagrange-approach with (3.22) as the objec-
tive function and (3.21) as the constraint. We will use here the dynamic programming
approach, however. Before we can apply it, we need to derive a dynamic budget con-
straint. We therefore dene
`
t
=
t1
t=1
,(c
t
)
as the amount of the pages that have already been written by today. This then implies
`
t1
`
t
= ,(c
t
). (3.23)
The increase in the number of completed pages between today and tommorrow de-
pends on eort-induced output , (c
t
) today. We can now apply the three dynamic
programming steps.
The value function is given by \ (`
t
) = max
fc
:
g
l
t
subject to the constraint. The
objective function written recursively reads
l
t
= ,
_
,
T(t1)
1 ,
1
T
t=t
,
tt
c
t
_
= ,
_
,
T(t1)
1 ,
1
_
c
t
+
T
t=t1
,
tt
c
t
_
_
= ,
_
,
T(t1)
1
T
t=t1
,
t(t1)
c
t
_
c
t
= ,l
t1
c
t
.
Assuming that the individual behaves optimally as from tomorrow, this reads l
t
=
c
t
+,\ (`
t1
) and the Bellman equation reads
\ (`
t
) = max
c
I
c
t
+,\ (`
t1
) . (3.24)
The rst-order conditions is 1 +,\
0
(`
t1
)
oA
I+1
oc
I
= 0. which, using the dynamic
budget constraint, becomes
1 = ,\
0
(`
t1
),
0
(c
t
). (3.25)
Again as in (3.12), implicitly and with (3.23), this equation denes a functional
relationship between the control variable and the state variable, c
t
= c (`
t
) .
40
To provide some variation, we will now go through the second step of dynamic
programming without using the envelope theorem. Consider the maximized Bellman
equation, where we insert c
t
= c (`
t
) and (3.23) into Bellman equation (3.24),
\ (`
t
) = c (`
t
) +,\ (`
t
+,(c (`
t
))) .
The derivative with respect to `
t
is
\
0
(`
t
) = c
0
(`
t
) +,\
0
(`
t
+,(c (`
t
)))
d [`
t
+, (c (`
t
))]
d`
t
= c
0
(`
t
) +,\
0
(`
t1
) [1 + ,
0
(c (`
t
)) c
0
(`
t
)] .
Using the rst-order condition (3.25) simplies this derivative to \
0
(`
t
) = ,\
0
(`
t1
) .
Expressed for t + 1 gives
\
0
(`
t1
) = ,\
0
(`
t2
) (3.26)
The nal step inserts the rst-order condition (3.25) twice to replace \
0
(`
t1
)
and \
0
(`
t2
) .
,
1
(,
0
(c
t
))
1
= (,
0
(c
t1
))
1
=
,
0
(c
t1
)
,
0
(c
t
)
= ,. (3.27)
The interpretation is now simple. As ,
00
(.) < 0 and , < 1. eort c
t
is increasing under
optimal behaviour, i.e. c
t1
c
t
. Optimal writing of a dissertation implies more work
every day.
What about levels?
The optimality condition in (3.27) species only how eort c
t
changes over time,
it does not provide information on the level of eort required every day. This is
a property of all solutions of intertemporal problems. They only give information
about changes of levels, not about levels themselves. The basic idea how to obtain
information about levels can be easily illustrated, however.
Assume , (c
t
) = c
o
t
. with 0 < o < 1. Then (3.27) implies (with t being replaced
by t) c
o1
t1
,c
o1
t
= , =c
t1
= ,
1(1o)
c
t
. Solving this dierence equation yields
c
t
= ,
(t1)(1o)
c
1
. (3.28)
where c
1
is the (at this stage still) unknown initial eort level. Inserting this solution
into the constraint (3.21) yields, starting in t = 1 on the rst day,
T
t=1
,
_
,
(t1)(1o)
c
1
_
=
T
t=1
,
(t1)o(1o)
c
o
1
= 1.
41
This gives us the initial eort level as (the sum can be solved by using the proofs in
ch. 2.5.1)
c
1
=
_
1
T
t=1
,
(t1)o(1o)
_
1o
.
With (3.28), we have now also computed the level of eort every day.
Behind these simple steps, there is a general principle. Solutions to intertemporal
problems are dierence or dierential equations. Any dierence (or also dierential)
equation when solved gives a unique solution only if an initial or boundary condition is
provided. Here, we have solved the dierence equation in (3.27) assuming some initial
condition c
1
. The meaningful initial condition then followed from the constraint (3.21).
Hence, in addition to the optimality rule (3.27), we always need some additional
constraint which allows to compute the level of optimal behaviour. We return to this
point when looking at problems in continuous time in ch. 5.3.
3.6 A general equilibrium analysis
We have so far analyzed maximization problems of households in partial equilibrium.
In two-period models, we have analyzed how households can be aggregated and what
we learn about the evolution of the economy as a whole. We will now do the same
for innite horizon problems.
We will, however, not simply aggregate over households and then put things to-
gether as we did in ch. 2.4, but we will rather present a model as it should be pre-
sented: First, technologies are specied. This shows what is technologically feasible
in this economy. Which goods are produced, how can be saved, is there uncertainty
in the economy? Second, preferences of the household are presented and the budget
constraint of households is derived from the technologies presented before. This is the
reason why technologies should be presented before households: budget constraints
are endogenous and depend on knowledge what households can do.
3.6.1 Technologies
The technology is a simple Cobb-Douglas technology
1
t
=
t
1
c
t
1
1c
t
. (3.29)
This good can be used for consumption and investment and equilibrium on the goods
market requires
1
t
= C
t
+1
t
.
Gross investment 1
t
is turned into net investment by taking depreciation into account,
1
t1
= (1 o) 1
t
+1
t
.
Taking these two equations together give the resource constraint of the economy,
1
t1
= (1 o) 1
t
+1
t
C
t
. (3.30)
As this constraint is simply a consequence of technologies and market clearing, it is
identical to the one used in the OLG setup in (2.26).
42
3.6.2 Firms
Firms maximize prots by employing optimal quantities of labour and capital, given
the technology in (3.29). First order conditions are
J1
t
J1
t
= n
1
t
.
J1
t
J1
t
= n
1
t
(3.31)
as in (2.19), where we have again chosen the consumption good as numeraire.
3.6.3 Households
Preferences of households are described as in the intertemporal utility function (3.1).
As the only way households can transfer savings from one period to the next is by
buying investment goods, an individuals wealth is given by the number of machines
/
t
, she owns. Clearly, adding up over all individual wealth stocks gives the total
capital stock,
1
/
t
= 1
t
. Wealth /
t
increases over time if the household spends less
on consumption than what she earns through capital plus labour income, corrected
for the loss in wealth each period caused by depreciation,
/
t1
/
t
= n
1
t
/
t
o/
t
+n
1
t
c
t
=
/
t1
=
_
1 +n
1
t
o
_
/
t
+n
1
t
c
t
.
If we now dene the interest rate to be given by
:
t
= n
1
t
o. (3.32)
we obtain our budget constraint
/
t1
= (1 +:
t
) /
t
+n
1
t
c
t
. (3.33)
Note that the derivation of this budget constraint was simplied as compared
to ch. 2.5.5 as the price
t
of an asset is, as we measure it in units of the consumption
good, by 1.
Given the preferences and the constraint, the solution to this maximization prob-
lem is given by (see exercise 5)
n
0
(c
t
) = , (1 +:
t1
) n
0
(c
t1
) . (3.34)
Structurally, this is the same expression as in (3.20). The interest rate, however, refers
to t+1, due to the change in the budget constraint. Remembering that , = 1, (1 +j),
this shows that consumption increases as long as :
t1
j.
3.6.4 Aggregation and reduced form
Aggregation
43
To see that individual constraints add up to the aggregate resource constraint,
we simply need to take into account as in (2.21) that individual income adds up to
output 1
t
. n
1
t
1
t
+n
1
t
1
t
= 1
t
.
The optimal behaviour of all household taken together can be gained from (3.34)
by summing over all households. This is done analytically correctly by rst applying
the inverse function of n
0
to this equation and then summing individual consumption
levels over all households. Applying the inverse function again gives
n
0
(C
t
) = , (1 +:
t1
) n
0
(C
t1
) . (3.35)
where C
t
is aggregate consumption in t.
Reduced form
We now need to understand how our economy evolves in general equilibrium. Our
rst equation is (3.35), telling us how consumption evolves over time. This equation
contains consumption and the interest rate as endogenous variables.
Our second equation is therefore the denition of the interest rate in (3.32) which
we combine with the rst-order condition of the rm in (3.31) to yield
:
t
=
J1
t
J1
t
o. (3.36)
This equation contains the interest rate and the capital stock as endogenous variable.
Our nal equation is the resource constraint (3.30), which provides a link between
capital and consumption. Hence, (3.35), (3.36) and (3.30) gives a system in three
equations and three unknowns. When we insert the interest rate into the optimality
condition for consumption , we obtain as our reduced form
n
0
(C
t
) = ,
_
1 +
0Y
I+1
01
I+1
o
_
n
0
(C
t1
) .
1
t1
= (1 o) 1
t
+1
t
C
t
.
(3.37)
This is a two-dimensional system of non-linear dierence equations which gives a
unique solution for the time path of capital and consumption, provided we have two
initial conditions 1
0
and C
0
.
3.6.5 Steady state and transitional dynamics
Steady state
When trying to understand a system like (3.37), the easiest is to understand the
steady state. In a steady state, all variables are constant and we obtain
1 = ,
_
1 +
J1
J1
o
_
=
J1
J1
= j +o. C = 1 o1.
where we used the denition of , in (3.2). In the steady state, the marginal produc-
tivity of capital is given by the time preference rate plus depreciation rate. Consump-
tion equals output minus depreciation, i.e. minus replacement investment. These two
equations determine two variables 1 and C.
44
Transitional dynamics
Understanding transitional dynamics is not as straightforward as understanding
the steady state. Its analysis follows the same idea as in continuous time, however,
and we will analyze transitional dynamics in detail there.
3.6.6 The intertemporal budget constraint
Consider the budget constraint derived in (??) as just used in (3.33),
c
t1
= (1 +:
t
) c
t
+n
t
c
t
.
What is the intertemporal version of this budget constraint?
To nd this out, we now solve it recursively. Rewrite it as
c
t
=
c
t1
+c
t
n
t
1 +:
t
. c
ti
=
c
ti1
+c
ti
n
ti
1 +:
ti
.
Inserting suciently often yields
c
t
=
o
I+2
c
I+1
&
I+1
1v
I+1
+c
t
n
t
1 +:
t
=
c
t2
+c
t1
n
t1
(1 +:
t1
) (1 +:
t
)
+
c
t
n
t
1 +:
t
=
o
I+3
c
I+2
&
I+2
1v
I+2
+c
t1
n
t1
(1 +:
t1
) (1 +:
t
)
+
c
t
n
t
1 +:
t
=
c
tS
+c
t2
n
t2
(1 +:
t2
) (1 +:
t1
) (1 +:
t
)
+
c
t1
n
t1
(1 +:
t1
) (1 +:
t
)
+
c
t
n
t
1 +:
t
= ... = lim
i!1
c
ti
(1 +:
ti1
) (1 +:
t1
) (1 +:
t
)
+
1
i=0
c
ti
n
ti
(1 +:
ti
) (1 +:
t1
) (1 +:
t
)
.
The expression in the last line is hopefully instructive but not very stringent. We can
write it in a more stringent way as
c
t
= lim
i!1
c
ti
i1
c=0
(1 +:
ti
)
+
1
i=0
c
ti
n
ti
i
c=0
(1 +:
tc
)
where indicates a product, i.e.
i
c=0
(1 +:
ti
) = 1+:
t
for i = 0 and
i
c=0
(1 +:
ti
) =
(1 +:
ti
) (1 +:
t
) for i 0. For i = 1.
i
c=0
(1 +:
ti
) = 1 by denition.
Letting the limit be zero, we obtain
c
t
=
1
i=0
c
ti
n
ti
i
c=0
(1 +:
tc
)
=
1
t=t
c
t
n
t
tt
c=0
(1 +:
tc
)
=
1
t=t
c
t
n
t
1
t
where the last but one equality substituted t +i by t. We can write this as
1
t=t
c
t
1
t
= c
t
+
1
t=t
n
t
1
t
. (3.38)
With a constant interest rate, this reads
1
t=t
(1 +:)
(tt1)
c
t
= c
t
+
1
t=t
(1 +:)
(tt1)
n
t
.
45
Equation (3.38) is the intertemporal budget constraint that results from a dynamic
budget constraint as specied in (??). Comparing it with (3.3) shows that the present
values on both sides discount one time more than in (3.3). The economic dierence
lies in whether we look at values at the beginning or the end of a period.
The budget constraint (??) and (3.33) are the natural budget constraint in the
sense that it can be derived easily as above in (??) and in the sense that it easily
aggregates to economy wide resource constraints. We will therefore work with them in
what follows. The reason for not working with them right from the beginning is that
the intertemporal version (3.3) has some intuitive appeal. Showing its shortcomings
explicitly should make clearer why - despite its intuitive appeal - it is not so useful
in general equilibrium setups.
3.7 A central planner
3.7.1 Optimal factor allocation
One of the most often solved maximization problems in economics is the central
planner solution. The choice of a central planner given a social welfare function and
technological constraints provides information about the rst-best factor allocation.
This is a benchmark for many analyses in normative economics. We consider the
probably simplest case of optimal factor allocation in a dynamic setup.
Let preferences be given by
l
t
=
1
t=t
,
tt
n(C
t
) . (3.39)
where C
t
is aggregate consumption of all households at a point in time t. This function
is maximized subject to a resource accumulation constraint which reads
1
t1
= 1 (1
t
. 1
t
) + (1 o) 1
t
C
t
. (3.40)
The production technology is given by a neoclassical production function 1 (1
t
. 1
t
)
with standard properties.
The Lagrangian
This setup diers from the ones one got to know before in that there is an innity
of constraints in (3.40). This constraint holds for each point in time t _ t. As a
consequence, the Lagrangian is formulated with innitly many Lagrangian multipliers,
/ =
1
t=t
,
tt
n(C
t
) +
1
t=t
`
t
[1
t1
1 (1
t
. 1
t
) (1 o) 1
t
+C
t
] . (3.41)
The rst part of the Lagrangian is standard,
1
t=t
,
tt
n(C
t
), it just copies the ob-
jective function. The second part consists in a sum from t to innity, one constraint
46
for each point in time, each multiplied by its own Lagrange multiplier `
t
. In order to
make the maximization procedure clearer, we rewrite the Lagrangian as
/ =
1
t=t
,
tt
n(C
t
) +
c2
t=t
`
t
[1
t1
1 (1
t
. 1
t
) (1 o) 1
t
+C
t
]
+`
c1
[1
c
1 (1
c1
. 1
c1
) (1 o) 1
c1
+C
c1
]
+`
c
[1
c1
1 (1
c
. 1
c
) (1 o) 1
c
+C
c
]
+
1
t=c1
`
t
[1
t1
1 (1
t
. 1
t
) (1 o) 1
t
+C
t
] .
where we simply explicitly wrote out the sum for : 1 and :.
Now maximize the Lagrangian both with respect to the control variable C
t
. the
multipliers `
t
and the state variables 1
t
. Maximization with respect to 1
t
might
appear unusual at this stage; we will see a justication for this in the next chapter.
First order conditions are
/
C
s
= ,
ct
n
0
(C
c
) +`
c
= 0 ==`
c
= n
0
(C
c
) ,
ct
. (3.42)
/
1
s
= `
c1
`
c
_
J1
J1
c
+ 1 o
_
= 0 ==
`
c1
`
c
= 1 +
J1
J1
c
o. (3.43)
/
A
s
= 0. (3.44)
Combining the rst and second rst-order condition gives
&
0
(C
s1
)o
s1I
&
0
(C
s
)o
sI
= 1+
0Y
01
s
o.
This is equivalent to
n
0
(C
c
)
,n
0
(C
c1
)
=
1
1
1
OY
O1
s+1
c
. (3.45)
This expression has the same interpretation as e.g. (3.5) or (3.20). When we replace :
by t. this equation with the constraint (3.40) is a two-dimensional dierence equation
system which allows to determine the paths of capital and consumption, given two
boundary conditions, which the economy will follow when factor allocation is opti-
mally chosen. The steady state of such an economy is found by setting C
c
= C
c1
and 1
c
= 1
c1
in (3.45) and (3.40).
3.7.2 Where the Lagrangian comes from II
Let us now see how we can derive the same expression as in (3.45) without using the
Lagrangian. This will allow us to give an intuitive explanation for why we maximized
the Lagrangian in the last chapter with respect to both the control and the state
variable.
Maximization without Lagrange
Insert the constraint (3.40) into the objective function (3.39) and nd
l
t
=
1
t=t
,
tt
n(1 (1
t
. 1
t
) + (1 o) 1
t
1
t1
) max
f1
:
g
This is now maximized by choosing a path 1
t
for capital. Choosing the state
variable implicitly pins down the path C
t
of the control variable consumption and
47
one can therefore think of this maximization problem as one where consumption is
optimally chosen.
Now rewrite the objective function as
l
t
=
c2
t=t
,
tt
n(1 (1
t
. 1
t
) + (1 o) 1
t
1
t1
)
+,
c1t
n(1 (1
c1
. 1
c1
) + (1 o) 1
c1
1
c
)
+,
ct
n(1 (1
c
. 1
c
) + (1 o) 1
c
1
c1
)
+
1
t=c1t
,
tt
n(1 (1
t
. 1
t
) + (1 o) 1
t
1
t1
) .
Choosing 1
c
optimally implies
,
c1t
n
0
(1 (1
c1
. 1
c1
) + (1 o) 1
c1
1
c
)
+,
ct
n
0
(1 (1
c
. 1
c
) + (1 o) 1
c
1
c1
)
_
J1 (1
c
. 1
c
)
J1
c
+ 1 o
_
= 0.
Reeinserting the constraint (3.40) and rearranging gives
n
0
(C
c1
) = ,n
0
(C
c
)
_
1 +
J1 (1
c
. 1
c
)
J1
c
o
_
.
This is the standard optimality condition for consumption which we obtained in
(3.45). As : can stand for any point in time between t and innity, we could replace
: by t, t or t + 1.
Back to the Lagrangian
When we now go back to the maximization procedure where the Lagrangian was
used, we see that the partial derivative of the Lagrangian with respect to 1
t
in
(3.43) captures how `
t
changes over time. The simple reason why the Lagrangian is
maximized with respect to 1
t
is therefore that an aditional rst-order condition is
needed as `
t
needs to be determined as well.
In static maximization problems with two consumption goods and one constraint,
the Lagrangian is maximized by choosing consumption levels for both consumption
goods and by choosing the Lagrange multiplier. In the Lagrange setup above, we
choose in (3.42) to (3.44) both endogenous variables 1
t
and C
t
plus the multiplier `
t
and thereby determine optimal paths for all three variables. Hence, it is a technical
- mathematical - reason that 1
t
is chosen, three unknowns simply require three
rst-order conditions to be determined, but economically, the control variable is C
t
which is economically chosen while the state variable 1
t
adjusts indirectly as a
consequence of the choice of C
t
.
3.8 Exercises Chapter 3 and further reading
The dynamic programming approach was developed by Bellman (1957). Maximiza-
tion using the Lagrange method is widely applied by Chow (1997).
48
Exercises chapter 3
Dynamic programming in discrete deterministic time
1. The envelope theorem I
Let the utility function of an individual be given by
l = l (C. 1) .
where consumption C increases utility and supply of labour L decreases utility.
Let the budget constraint of the individual be given by
n1 = C.
Let the individual maximize utility with respect to consumption and the amount
of labour supplied.
(a) What is the optimal labour supply function (in implicit form)? How much
does an individual consume? What is the indirect utility function?
(b) Under what conditions does an individual increase labour supply when
wages rise? (No analytical solution)
(c) Assume higher wages lead to higher labour supply. Does disutility arising
from higher labour supply overcompensate utility from higher consump-
tion? Does utility rise if there is no disutility from working? Start from the
indirect utility function derived in a) and apply the proof of the envelope
theorem and the envelope theorem itself.
2. The envelope theorem II
(a) Compute the derivative of the Bellman equation (3.19) without using the
envelope theorem. Hint: Compute the derivative with respect to the state
variable and then insert the rst-order condition.
(b) Do the same with (3.26)
3. The additively separable objective function
(a) Show that the objective function can be written as in (3.10).
(b) Find out whether (3.10) implies the objective function. (It does not.)
4. Intertemporal and dynamic budget constraints
49
(a) Show that the intertemporal budget constraint
T
t=t
_
t1
I=t
1
1 +:
I
_
c
t
= c
t
+
T
t=t
_
t1
I=t
1
1 +:
I
_
i
t
(3.46)
implies the dynamic budget constraint
c
t1
= (c
t
+i
t
c
t
) (1 +:
t
) . (3.47)
(b) Under which conditions does the dynamic budget constraint imply the
intertemporal budget constraint?
(c) Now consider c
t1
= (1 +:
t
) c
t
+ n
t
c
t
. What intertemporal budget
constraint does it imply?
5. The standard saving problem
Consider the objective function from (3.1), l
t
=
1
t=t
,
tt
n(c
t
), and maximize
it by chosing a consumption path c
t
subject to the constraint (3.33), /
t1
=
(1 +:
t
) /
t
+n
1
t
c
t
. The solution is given by (3.34).
(a) Solve this problem by dynamic programming methods.
(b) Solve this by using the Lagrange approach. Choose a multiplier `
t
for an
innite sequence of constraints.
6. A benevolent central planner
You are the omniscient omnipotent benevolent central planner of an economy.
You want to maximize a social welfare function
l
t
=
1
t=t
,
tt
n(C
t
)
for your economy by choosing a path of aggregate consumption levels C
t
subject to a resource constraint

1
t1
1
t
= 1 (1
t
. 1
t
) o 1
t
C
t
(3.48)
(a) Solve this problem by dynamic programming methods.
(b) Solve this by using the Lagrange approach. Choose a multiplier `
t
for an
innite sequence of constraints.
(c) Discuss how the central planner solution is related to the decentralized
solution from exercise 5.
7. Environmental economics
Imagine you are an economist only interested in maximizing the present value of
your endowment. You own a renewable resource, for example a piece of forest.
The amount of wood in your forest at a point in time t is given by r
t
. Trees
grow at /(r
t
) and you harvest at t the quantity c
t
.
50
(a) What is the law of motion for r
t
?
(b) What is your objective function if prices at t per unit of wood is given by
j
t
, your horizon is innity and you have perfect information?
(c) How much should you harvest per period when the interest rate is constant?
Does this change when the interest rate is time-variable?
8. Exam question (winter term 2005/06, Wrzburg)
Sie berlegen sich, fr die Wirtschaftswissenschaftliche Fakultt am Residen-
zlauf teilzunehmen. Dieser ndet in ` Monaten statt. Sie wissen, da Sie dafr
trainieren mssen, was Ihnen Anstrengung c
0
abverlangt und Ihren Nutzen n(.)
reduziert. Gleichzeitig ziehen Sie einen Nutzen aus einer kurzen Laufzeit |. Die
Laufzeit ist umso krzer, je grer Ihre Anstrengung.
(a) Formulieren Sie ein Maximierungsproblem mit 2 Perioden. Sie trainieren
heute und erzielen damit eine gewisse Laufzeit in ` Monaten. Spezizieren
Sie eine angemessene Nutzenfunktion und einen Zusammenhang zwischen
Laufzeit und Trainingsanstrengung c
0
.
(b) Lsen Sie das Maximierungsproblem durch Herleitung einer Bedingung
erster Ordnung. Interpretieren Sie diese.
Betrachten Sie folgende Zielfunktion eines zentralen Planers,
l
0
=
1
t=0
,
t
n(C
t
) . (3.49)
Die Nebenbedingung sei gegeben durch
1
t1
= 1 (1
t
. 1
t
) + (1 o) 1
t
C
t
. (3.50)
(a) Erlutern Sie Zielfunktion und Nebenbedingung verbal.
(b) Lsen Sie das Maximierungsproblem, indem Sie (3.50) in (3.49) einsetzen
und nach 1
t
maximieren. Zeigen Sie, da als Ergebnis
&
0
(C
I
)
o&
0
(C
I+1
)
= 1 +
0Y (1
I+1
,1
I+1
)
01
I+1
o resultiert und interpretieren Sie dieses verbal.
(c) Erlutern Sie nun, wieso bei der Verwendung der Lagrangefunktion (siehe
bung) auch bezglich 1
t
maximiert wird, obwohl 1
t
eine Zustandsvari-
able ist.
51
Part II
Deterministic models in continuous
time
4 Dierential equations
There are many excellent textbooks on dierential equations. This chapter will there-
fore be relatively short. Its objective is more to recall basic concepts taught in other
courses and to serve as a background for later applications.
4.1 Some denitions and theorems
4.1.1 Denitions
The following denitions are standard and follow Brock and Malliaris (1989).
Denition 4 An ordinary dierential equation system (ODE system) is of the type
dr(t)
dt
= _ r (t) = , (t. r (t)) . (4.1)
where t lies between some starting point and innity, t [t
0
. ] . r can be a vector,
r 1
a
and , maps from 1
a1
into 1
a
. When r is not a vector, i.e. for : = 1. (4.1)
obviously is an ordinary dierential equation.
An autonomous dierential equation is an ordinary dierential equation where
,(.) is independent of time t.
dr
dt
= _ r (t) = , (r (t)) . (4.2)
The dierence between a dierential equation and a normal equation obviously
lies in the fact that dierential equations contain derivatives of variables like _ r (t).
An example of a dierential equation which is not an ODE is the partial dierential
equation. A linear example is
c (r. )
Jn (r. )
Jr
+/ (r. )
Jn (r. )
J
= c (r. ) .
where c (.) . / (.) . c (.) and n(.) are functions with nice properties. The dierence
obviously lies in the presence of two derivatives. While in an ODE, there is one deriv-
ative (often with respect to time), a partial dierential equation contains derivatives
with respect to several variables.
Denition 5 An initial value problem (Anfangswertproblem) is described by
_ r = ,(t. r). r (t
0
) = r
0,
t [t
0
. 1] .
A boundary value problem is of the form
_ r = , (t. r) . r (1) = r
T
. t [t
0
. 1]
52
4.1.2 Two theorems
Theorem 2 Existence (Brock and Malliaris, 1989)
If , (t. r) is continuous function on rectangle 1 = (t. r)[ [t t
0
[ _ c. [r r
0
[ _ /
then there exists a continuous dierentiable solution r (t) on interval [t t
0
[ _ c that
solves initial value problem
_ r = , (t. r) . r (t
0
) = r
0
. (4.3)
This theorem only proves that a solution exists. It is still possible that there are
many solutions.
Theorem 3 Uniqueness
If , and J,,Jr are continuous functions on 1. the initial value problem (4.3) has a
unique solution for
t
_
t
0
. t
0
+ min
_
c.
/
max [, (t. r)[
__
If this condition is met, an ODE with an initial or boundary condition has a
unique solution. More generally speaking, a dierential equation system consisting
of : ODEs that satisfy these conditions (which are met in the economic problems we
encounter here) has a unique solution provided that there are : initial or boundary
conditions. Knowing about a unique solution is useful as one knows that changes in
parameters imply unambiguous predictions about changes in endogenous variables.
If the government changes some tax, we can unambiguously predict whether employ-
ment goes up or down.
4.2 Analysing ODEs through phase diagrams
This section will present tools that allow to determine properties of solutions of dif-
ferential equations and dierential equation systems. The analysis will be qualitative
in this chapter as most economic systems are too non-linear to allow for an explicit
general analytic solution. Explicit solutions for linear dierential equations will be
treated in ch. 4.3.
4.2.1 One dimensional systems
We start with a one-dimensional dierential equation _ r (t) = , (r (t)), where r 1
and t 0. This will also allow us to review the concepts of xpoints, local and global
stability and instability as used already when analysing dierence equations in ch.
2.5.4.
Unique xpoint
Let , (r) be represented by the graph in the following gure, with r (t) being
shown on the horizontal axis. As , (r) gives the change of r (t) over time, _ r (t) is
plotted on the vertical axis.
53
N
N
r(t)
_ r(t)
r
NN
NN
Figure 12 Qualitative analysis of a dierential equation
As in the analysis of dierence equations in ch. 2.5.4, we rst look for the xpoint
of the underlying dierential equation. A xpoint is dened similarily in spirit but -
thinking now in continuous time - dierently in detail: A xpoint r
is a point where
r (t) does not change; this is the similarity in spirit. Formally, this means _ r (t) = 0
which, from the denition (4.2) of the dierential equation, requires , (r
) = 0 (and
not , (r
) = r
as in (2.38)). Looking at the above graph of , (r) . we nd r
at the
point where , (r) crosses the horizontal line.
We then inquire the stability of the xpoint. When r is to the left of r
. , (r) 0
and therefore r increases, _ r (t) 0. This increase of r is represented in this gure by
the arrows on the horizontal axis. Similarily, for r r
. , (r) < 0 and r (t) decreases.

We have therefore found that the xpoint r
is globally stable and have also obtained

a feeling for the behaviour of r (t) . given some initial conditions.
We can now qualitatively plot the solutions with time t on the horizontal axis.
As the discussion has just shown, the solution r (t) depends on the initial value from
which we start, i.e. on r (0) . For r (0) r
. r(t) decreases, for r (0) < r
. r(t)
increases: any changes over time are monotonic. There is one solution for each initial
condition. The following gure shows three solutions of _ r (t) = , (r (t)), given three
dierent intitial conditions.
54
N
N
r(t)
r
0S
r
r
02
r
01
t t
0
Figure 13 Qualitative solutions of an ODE for dierent initial conditions I
Multiple xpoints and equilibria
Of course, more sophisticated functions than , (r) can be imagined. Consider now
a dierential equation _ r (t) = q (r (t)) where q (r) is non-monotonic as plotted in the
next gure.
N
N
r
r
N
NN
NN
r
1
r
2
r
S
r
1
Figure 14 Multiple equilibria
As this gure shows, there are four xpoints. Looking at whether q (r) is positive
or negative, we know whether r (t) increases or decreases over time. This allows us
to plot arrows on the horizontal axis as in the previous example. The dierence to
before consists in the fact that some xpoints are unstable and some are stable. Any
small deviation of r from r
1
implies an increase or decrase of r. The xpoint r
1
is
55
therefore unstable, given the denition in ch. 2.5.4. Any small deviation r
2
. however,
implies that r moves back to r
2
. Hence, r
2
is (locally) stable. The xpoint r
S
is also
unstable, while r
1
is again locally stable: While r converges to r
1
for any r r
1
(in
this sense r
1
could be called globally stable from the right), r converges to r
1
from
the left only if r is not smaller than or equal to r
S
.
If an economy can be represented by such a dierential equation, one would call
xpoints long-run equilibria. There are stable equilibria and unstable equilibria and
it depends on the underlying system (the assumptions that implied the dierential
equation _ r = q (r)) which equilibrium would be considered to be the economically
relevant one.
As in the system with one xpoint, we can qualitatively plot solutions of r (t)
over time, given dierent initial values for r (0). This is shown in the following gure
which again highlights the stability properties of xpoints r
1
to r
1
.
N
N
t
r
1
r
2
r
1
r
S
t
0
Figure 15 Qualitative solutions of an ODE for dierent initial conditions II
4.2.2 Two-dimensional systems I - An example
We now extend our qualitative analysis of dierential equations to two-dimensional
systems. This latter case allows for an analysis of more complex systems than simple
one-dimensional dierential equations. We start here with an example before we
analyse two dimensional systems more generally in the next chapter.
The system
56
Consider the following dierential equation system,
_ r
1
= r
c
1
r
2
. _ r
2
= / +r
1
1
r
2
. 0 < c < 1 < /.
Assume that for economic reasons we are interested in properties for r
i
0.
Fixpoint
The rst question is as always whether there is a xpoint at all. In a two-
dimensional system, a xpoint r
= (r
1
. r
2
) is two-dimensional as well. The xpoint
is dened such that both variables do not change over time, i.e. _ r
1
= _ r
2
= 0. If such
a point exists, it must satisfy
_ r
1
= _ r
2
= 0 =(r
1
)
c
= r
2
. r
2
= / + (r
1
)
1
.
By inserting the second equation into the rst, r
1
is determined by (r
1
)
c
= /+(r
1
)
1
and r
2
follows from r
2
= (r
1
)
c
. Analysing the properties of the equation (r
1
)
c
= / +
(r
1
)
1
would then show that r
1
is unique: The left-hand side increases monotonically
from 0 to innity for r
1
[0. [ while the right-hand side decreases monotonically
from innity to /. Hence, there must be an intersection point and there can be only
one as functions are monotonic. As r
1
is unique, so is r
2
= (r
1
)
c
.
General evolution
Having derived the xpoint, we now need to understand the behaviour of the
system more generally. What happens to r
1
and r
2
when (r
1
. r
2
) ,= (r
1
. r
2
)? To
answer this question, the concept of zero-motion lines is very useful. A zero-motion
line is a line for a variable r
i
which marks the points for which the variable r
i
does
not change, i.e. _ r
i
= 0. For our two-dimensional dierential equation system, we
obtain two zero-motion lines,
_ r
1
_ 0 =r
2
_ r
c
1
.
_ r
2
_ 0 =r
2
_ / +r
1
1
.
(4.4)
In addition to the equality sign, we also analyse here at the same time for which
values r
i
rises. Why this is useful will soon become clear. We can now plot the
curves where _ r
i
= 0 in a diagram. In contrast to the one-dimensional graphs in the
previous chapter, we now have the variables r
1
and r
2
on the axes (and not the change
of one variable on the vertical axis). The intersection point of the two zero-motion
lines gives the x point r
= (r
1
. r
2
) which we derived analytically above.
57
2
x
1
x
0
1
= x&
0
2
= x&
b
IV
I
II
III
Figure 16 A phase diagram
In addition to showing where variables do not change, the zero-motion lines also
delimit regions where variables do change. Looking at (4.4) again shows (why we
used the _ and not the = sign and) that the variable r
1
increases whenever r
2
< r
c
1
.
Similarily, the variable r
2
increases, whenever r
2
< / + r
1
1
. The directions in which
variables change can then be plotted into this diagram by using arrows. In this
diagram, there are two arrows per region as two directions (one for r
1
. one for r
2
)
need to be indicated. This is in principle identical to the arrows we used in the
analysis of the one-dimensional systems. If the system nds itself in one of these four
regions, we know qualitatively, how variable change over time: Variables move to the
southeast in region I, to the northeast in region II, to the northwest in region III and
to the southwest in region IV. We will analyse the implications of such a behaviour
further once we have generalized the derivation of a phase diagram.
4.2.3 Two-dimensional systems II - The general case
After this specic example, we will now look at a more general dierential equation
system and will analyse it by using a phase diagram.
The system
Consider two dierential equations where functions , (.) and q (.) are continuous
and dierentiable,
_ r
1
= , (r
1
. r
2
) . _ r
2
= q (r
1
. r
2
) . (4.5)
For the following analysis, we will need four assumptions on partial derivatives; all of
them are positive apart from ,
a
1
(.) .
,
a
1
(.) < 0. ,
a
2
(.) 0. q
a1
(.) 0. q
a
2
(.) 0. (4.6)
58
Note that, provided we are willing to make the assumptions required by the theorems
in ch. 4.1.2, we know that there is a unique solution to this dierential equation
system, i.e. r
1
(t) and r
2
(t) are unambiguously determined given two boundary
conditions.
Fixpoint
The rst question to be tackled is whether there is an equilibrium at all. Is there
a xpoint r
such that _ r
1
= _ r
2
= 0? To this end, set , (r
1
. r
2
) = 0 and q (r
1
. r
2
) = 0
and plot the implicitly dened functions in a graph.
2
x
1
x
*
2
x
*
1
x
( ) 0 ,
2 1
= x x f
( ) 0 ,
2 1
= x x g
Figure 17 Zero motion lines with a unique steady state
By the implicit function theorem - see (2.14) - and the assumptions made in (4.6),
one zero motion line is increasing and one is decreasing. If we are further willing to
assume that functions are not monotonically approaching an upper and lower bound,
we know that there is a unique xpoint (r
1
. r
2
) = r
.
General evolution
Now we ask again what happens if the state of the system diers from r
. i.e. if
either r
1
or r
2
or both dier from their steady state values. To nd an answer, we
have to determine the sign of , (r
1
. r
2
) and q (r
1,
r
2
) for some (r
1
. r
2
) . Given (4.5). r
1
would increase for a positive , (.) and r
2
would increase for a positive q (.) . For any
known functions , (r
1
. r
2
) and q (r
1,
r
2
) . one can simply plot a 3-dimensional gure
with r
1
and r
2
on the axes in the plane and with time derivatives on the vertical axis.
59
Figure 18 A three-dimensional illustration of two dierential equations and their
zero-motion lines
When working with two-dimensional gures and without the aid of computers, we
start from the zero-motion line for, say, r
1
.
N
N
r
1
r
2
(~ r
1,
~ r
2
)
r
_
_
_
_
_
_
_
_
_
_
,(r
1
. r
2
) = 0
Figure 19 Step 1 in constructing a phase diagram
Now consider a point (~ r
1
. ~ r
2
) . As we know that _ r
1
= 0 on , (r
1
. r
2
) = 0 and that
from (4.6) ,
a
2
(.) 0. we know that moving from ~ r
1
on the zero-motion line vertically
to (~ r
1
. ~ r
2
) . , (.) is increasing. Hence, r
1
is increasing, _ r
1
0. at (~ r
1
. ~ r
2
) . As this
line of reasoning holds for any (~ r
1
. ~ r
2
) above the zero-motion line, r
1
is increasing
everywhere above , (r
1
. r
2
) = 0. As a consequence, r
1
is decreasing everywhere below
the zero-motion line. This movement is indicated by the arrows in the above gure.
60
N
N
r
1
r
2
r
(~ r
1,
~ r
2
)
'
q(r
1
. r
2
) = 0
'
'
'
'
'
'
'
Figure 20 Step 2 in constructing a phase diagram
Let us now consider the second zero-motion line, _ r
2
= 0 = q (r
1
. r
2
) = 0. and
look again at the point (~ r
1
. ~ r
2
) . When we start from the point ~ r
2
on the zero motion
line and move towards (~ r
1
. ~ r
2
) . q (r
1
. r
2
) is decreasing, given the derivative q
a1
(.) 0
from (4.6). Hence, for any points to the left of q (r
1
. r
2
) = 0. r
2
is decreasing. Again,
this is shown in the above gure.
Plots and xpoints, the solution
We can now represent the directions in which r
1
and r
2
are moving into one
phase-diagram by plotting one arrow each into each of the four regions limited by the
zero-motion lines. Given that the arrows can either indicate an increase or a decrease
for both r
1
and r
2
. there are 2 times 2 dierent combinations of arrows. When we
add some representative trajectories, a complete phase diagram results.
0
2
= x&
0
1
= x&
1
x
2
x
Figure 21 A saddle path
61
4.2.4 Types of phase diagrams and xpoints
Some denitions
As has become clear by now, the partial derivatives in (4.6) are crucial for the
slope of the zero-motion lines and for the direction of movements of variables r
1
and
r
2
. Depending on the signs of the partial derivatives, various phase diagrams can
occur. These phase diagrams can be classied into four typical groups, depending on
the properties of their xpoint.
Denition 6 A xpoint is called a
center
saddle point
focus
node
_
_
==
_
_
0
2
all
all
_
_
trajectories pass through the xpoint
and
_
_ on
_
at least one trajectory, 0
all trajectories, 1 or 2
_
variables are monotonic
A node and a focus can be either stable or unstable.
Some graphs
Here is now an overview of some typical phase diagrams.
node
(ii) node (Knoten)
- all integral curves pass
through node
(iia) (iib)
stable or unstable
Figure 22 Phase diagram for a node
62
This rst phase diagram shows a node. A node is a xpoint through which all
trajectories go and where the time paths implied by trajectories are monotonic. As
drawn here, it is a stable node, i.e. for any initial conditions, the system ends up in
the xpoint. An unstable node is a xpoint from which all trajectories start. A phase
diagram for an unstable node would look like the one above but with all directions of
motions reversed.
focus
(iii) focus
- all integral curves move
in spirals around fixpoint.
Either towards this point
or away.
(iii) a and b stable or unstable
Figure 23 Phase diagram for a focus
A phase diagram with a focus looks similar to one with a node. The dierence
consists in the non-monotonic paths of the trajectories. As drawn here, r
1
or r
2
rst
increase and then decrease on some trajectories.
(iv) centre
- no integral curve passes
through fixpoint
predator-prey-model
centre
Figure 24 Phase diagram for a center
63
A circle is a very special case for a dierential equation system. It is rarely found
in models with optimizing agents. The standard example is the predator-prey model,
_ r = cr ,r, _ = + or, where c, ,, , and o are positive constants. This is
also called the Lotka-Volterra model. No closed form solution has been found so far.
Limitations
It should be noted that a phase diagram analysis allows to identify a saddle point.
If no saddle point can be identied, it is generally not possible to distinguish between
a node, focus or center. In the linear case, more can be deduced from a graphical
analysis. This is generally not necessary, however, as there is a closed form solution.
4.2.5 Multidimensional systems
If we have higher-dimensional problems where r 1
a
and : 2. phase diagrams are
obviously dicult to be drawn. In the three-dimensional case, plotting zero motion
surfaces sometimes helps to gain some intuition. A graphical solution will generally,
however, not allow to identify equilibrium properties like saddle-path or saddle-plane
behaviour.
4.3 Linear dierential equations
This section will focus on a special case of the general ODE dened in (4.1). The
special aspect consists in making the function , (.) in (4.1) linear in r (t) . Doing this,
we obtain a linear dierential equation,
_ r (t) = c (t) r (t) +/ (t) . (4.7)
where c (t) and / (t) are functions of time. This is the most general case for a one
dimensional linear dierential equation.
4.3.1 Rules on derivatives
Before analysing (4.7) in more detail, we rst need some other results which will
be useful later. This section therefore rst presents some rules on how to compute
derivatives. It is by no means intended to be comprehensive or go in any depth. It
presents rules which have shown by experience to be of importance.
Integrals
Denition 7 A function 1 (r) =
_
, (r) dr is the indenite integral (Stammfunk-
tion) of a function , (r) if
d
dr
1 (r) =
d
dr
_
, (r) dr = , (r) (4.8)
This denition implies that there are innitely many integrals of , (r) . If 1 (r) is
an integral, then 1 (r) +c. where c is a constant, is an integral as well.
64
Leibniz rule
We present here a rule for computing the derivative of an integral function. Let
there be a function . (r) with argument r. dened by the integral
. (r) =
_
b(a)
o(a)
, (r. ) d.
where c (r) . / (r) and , (r. ) are dierentiable functions. Then, the Leibniz rule
says that the derivative of this function with respect to r is
d
dr
. (r) = /
0
(r) , (r. / (r)) c
0
(r) , (r. c (r)) +
_
b(a)
o(a)
J
Jr
, (r. ) d. (4.9)
The following gure illustrates this rule.
( ) y x f ,
( ) x a ( ) x b
( ) y x f ,
y
( ) x z
( ) x x z +
( ) x z
( ) x x z +
Figure 25 Illustration of the dierentiation rule
Let r increase by a small amount. Then the integral changes at three margins:
The upper bound, the lower bound and the function , (r. ) itself. As drawn here,
the upper bound / (r) and the function , (r. ) increase and the lower bound c (r)
decreases in r. As a consequence, the area below the function , (.) between bounds
c (.) and / (.) increases because of three changes: the increase to the left because of
c (.) . the increase to the right because of / (.) and the increase upwards because of
, (.) itself. Clearly, this gure changes when the derivatives of c (.), / (.) and , (.)
with respect to r have a dierent sign than the ones assumed here.
Integration by parts (Partielle Integration)
Proposition 4 For two dierentiable functions n(r) and (r) .
_
n(r)
0
(r) dr = n(r) (r)
_
n(r) (r)
0
dr. (4.10)
65
Proof. We start by observing that
(n(r) (r))
0
= n(r)
0
(r) +n(r) (r)
0
.
where we used the product rule. Integrating both sides,
_
dr. gives
n(r) (r) =
_
n(r)
0
(r) dr +
_
n(r) (r)
0
dr.
Rearranging gives (4.10).
Equivalently, one can show (see the exercises) that
_
b
o
_ rdt = [r]
b
o
_
b
o
r _ dt.
4.3.2 Forward and backward solutions of a linear dierential equation
We now return to our linear dierential equation _ r (t) = c (t) r (t) + / (t) from (4.7).
It will now be solved. A solution to this dierential equation is a function r (t) which
satises this equation. A solution can be called time path of r. when t is representing
time.
General solution of the non-homogenous equation
The dierential equation in (4.7) has, as all dierential equations, an innity of
solutions. The general solution reads
r (t) = c
R
o(t)ot
_
~ r +
_
c
R
o(t)ot
/ (t) dt
_
. (4.11)
Here, ~ r is some arbitrary constant. As this constant is arbitrary, (4.11) provides
indeed an innity of solutions to (4.7).
To see that (4.11) is a solution to (4.7) indeed, remember the denition of what
a solution is. A solution is a time path r (t) which satises (4.7). Hence, we simply
need to insert the time path given by (4.11) into (4.7) and check whether (4.7) then
holds. To this end, compute the time derivative of r(t),
d
dt
r (t) = c
R
o(t)ot
c (t)
_
~ r +
_
c
R
o(t)ot
/ (t) dt
_
+c
R
o(t)ot
c
R
o(t)ot
/ (t) .
where we have used the denition of the integral in (4.8),
o
oa
_
, (r) dr = , (r) .
Note that we do not have to apply the product or chain rule since, again by (4.8),
o
oa
_
q (r) /(r) dr = q (r) /(r) . Inserting (4.11) gives
_ r (t) = c (t) r (t) +/ (t) .
Hence, (4.11) is a solution to (4.7).
Determining the constant ~ r
66
To obtain one particular solution, some value r (t
0
) has to be xed. Depending
on whether t
0
lies in the future or in the past, t
0
t or t
0
< t, the equation is solved
forward or backward.
We start with the backward solution, i.e. where t t
0
. Let r (t
0
) = r
0
. Then the
solution of (4.7) is
r (t) = c
R
I
I
0
o(t)ot
__
r (t
0
) +
_
t
t
0
c
R
:
I
0
o(&)o&
/ (t) dt
__
= c
R
I
I
0
o(t)ot
r (t
0
) +
_
t
t
0
c
R
I
:
o(&)o&
/ (t) dt. (4.12)
The forward solution is required if t
0
. which we rename 1 for ease of distinction,
lies in the future of t, 1 t. The solution then reads
r (t) = c
R
T
I
o(t)ot
r (1)
_
T
t
c
R
:
I
o(&)o&
/ (t) dt. (4.13)
Verication
We now show that (4.12) and (4.13) are indeed a solution for (4.7). The time
derivative of (4.12) is given by
_ r (t) = c
R
I
I
0
o(t)ot
c (t) r (t
0
) +c
R
I
I
o(&)o&
/ (t) +
_
t
t
0
c
R
I
:
o(&)o&
/ (t) dtc (t)
= c (t)
_
c
R
I
I
0
o(t)ot
r (t
0
) +
_
t
t
0
c
R
I
:
o(&)o&
/ (t) dt
_
+/ (t)
= c (t) r (t) +/ (t) .
The time derivative of (4.13) is
_ r (t) = c
R
T
I
o(t)ot
c (t) r (1) +/ (t)
_
T
t
c
R
:
I
o(&)o&
/ (t) dtc (t)
= c (t)
_
c
R
T
I
o(t)ot
r (1)
_
T
t
c
R
:
I
o(&)o&
/ (t) dt
_
+/ (t)
= c (t) r (t) +/ (t) .
4.3.3 Dierential equations as integral equations
Any dierential equation can be written as an integral equation. While we will work
with the usual dierential equation representation most of the time, we introduce
the integral representation here as it will be frequently used later when comput-
ing moments in stochastic setups. Understanding the integral version of dierential
equations in this deterministic setup allows for an easier understanding of integral
representations of stochastic dierential equations later.
The principle
67
The non-autonomous dierential equation _ r = , (t. r) can be written equivalently
as an integral equation. To this end, write this equation as dr = , (t. r) dt or, after
substituting : for t, as dr = , (:. r) d:. Now apply the integral
_
t
0
on both sides. This
gives the integral version of the dierential equation _ r = , (t. r) which reads
_
t
0
dr = r (t) r (0) =
_
t
0
, (:. r) d:.
An example
The dierential equation _ r = c (t) r is equivalent to
r (t) = r
0
+
_
t
0
c (:) r (:) d:. (4.14)
Computing the derivative of this equation with respect to time t gives, using (4.9),
_ r (t) = c (t) r (t) again.
The presence of an integral in (4.14) should not lead one to confuse (4.14) with
a solution of _ r = c (t) r in the sense of the last section. Such a solution would read
r (t) = r
0
c
R
I
0
o(c)oc
.
4.3.4 Example 1: Budget constraints
As an application of dierential equations, consider the budget constraint of a house-
hold. As in discrete time in ch. 3.6.6, budget constraints can be expressed in a
dynamic and an intertemporal way. We rst show here how to derive the dynamic
version and then how to obtain the intertemporal version from solving the dynamic
one.
Deriving a nominal dynamic budget constraint
Let us rst, following ch. 2.5.5, derive the dynamic budget constraint. We start
from the denition of nominal wealth. It is given by c = /. where / is the households
physical capital stock and is the value of one unit of the capital stock. By computing
the time derivative, wealth of a household changes according to
_ c =
_
/ +/ _ .
If the household wants to save, it can buy capital goods at the end of period t.
Nominal savings of the household in period t are given by : = n
I
/ + n c. the
dierence between factor rewards for capital (value marginal product times capital
owned), labour income and expenditure. Dividing savings by the value of a capital
stock at the end of period t gives the number of capital stocks bought
_
/ =
n
I
/ +n c
. (4.15)
68
Inserting into the _ c equation gives
_ c = n
I
/ +n c +/ _ = (n
I
+ _ ) / +n c =
n
I
+ _
c +n c.
Dening the nominal interest rate as
i =
n
I
+ _
. (4.16)
we have the nominal budget constraint
_ c (t) = i (t) c (t) +n(t) c (t) . (4.17)
Finding the intertemporal budget constraint
We can now obtain the intertemporal budget constraint from solving the dynamic
one in (4.17). Using the forward solution from (4.13), we take c (1) as the boundary
condition lying with 1 t in the future. The solution is then
c (t) = c
R
T
I
i(t)ot
c (1)
_
T
t
c
R
:
I
i(&)o&
[n(t) c (t)] dt =
_
T
t
1(t) n(t) dt +c (t) = 1(1) c (1) +
_
T
t
1(t) c (t) dt.
where 1(t) = c
R
:
I
i(&)o&
denes the discount factor. As we have used the forward
solution, we have obtained an expression which easily lends itself to an economic
interpretation. Think of an individual who - at the end of his life - does not want to
leave any bequest, i.e. c (1) = 0 or, for a positive c (1) . does not want to use any
of this c (1) for own consumption. In both cases, c (1) = 0 in the above expression.
Then, this intertemporal budget constraint requires that current wealth on the left-
hand side, consisting of the present value of life-time labour income plus nancial
wealth c (t) needs to equal the present value of current and future expenditure on the
right-hand side.
Real wealth
We can also start from the denition of real wealth of the household, measured in
units of the consumption good, whose price is j. Real wealth is then
c
v
=
/
j
.
The change in real wealth over time is then (apply the log on both sides and derive
with respect to time),
_ c
v
c
v
=
_
/
/
+
_

_ j
j
.
69
Inserting the increase in the capital stock obtained from (4.15) gives
_ c
v
c
v
=
n
I
/ +n c
/
+
_

_ j
j
=
(n
I
/ +n c) j
1
/j
1
+
_

_ j
j
= _ c
v
= n
I
/
j
+
n c
j
+
_
c
v
_ j
j
c
v
=
n
I
c
v
+
n c
j
+
_
c
v
_ j
j
c
v
=
_
n
I
+ _

_ j
j
_
c
v
+
n c
j
= :c
v
+
n c
j
. (4.18)
Hence, the real interest rate : is - by denition -
: =
n
I
+ _

_ j
j
.
Solving the dierential equation (4.18) again provides the intertemporal budget con-
straint as in the nominal case above.
Now assume that the price of the capital good equals the price of the consumption
good, = j. This is the case in an economy where there is one homogenous output
good as in (2.26) or in (8.14). Then, the real interest rate is equal to the marginal
product of capital, : = n
I
,j.
4.3.5 Example 2: Forward solutions
The capital market no-arbitrage condition
Imagine you own wealth of worth (t) . You can invest it on a bank account which
pays a certain return : (t) per unit of time or you can buy shares of a rm which cost
(t) and which yield dividend payments : (t) and are subject to changes _ (t) in its
worth. In a world of perfect information and assuming that in some equilibrium
agents hold both assets, the two assets must yield identical income streams,
: (t) (t) = : (t) + _ (t) . (4.19)
This is a linear dierential equation in (t). As just motivated, it can be considered
as a no-arbitrage condition. Note, however, that it is structurally equivalent to (4.16),
i.e. this no-arbitrage condition can be seen to just dene the interest rate : (t).
Whatever the interpretation of this dierential equation is, solving it forward gives
(t) = c
R
T
I
v(t)ot
(1) +
_
T
t
c
R
:
I
v(&)o&
: (t) dt.
Letting 1 go to innity, we have
(t) =
_
1
t
c
R
:
I
v(&)o&
: (t) dt + lim
T!1
c
R
T
I
v(t)ot
(1) .
This forward solution stresses the economic interpretation of (t) : The value of
an asset depends on the future income stream - dividend payments : (t) - that are
70
generated from owning this asset. Note that it is usually assumed that there are no
bubbles, i.e. the limit is zero such that the fundamental value of an asset is given by
the rst term.
For a constant interest rate and dividend payments and no bubbles, the expression
for (t) simplies to = :,:.
The utility function
Consider an intertemporal utility function as it is often used in continuous time
models,
l (t) =
_
1
t
c
j[tt[
n(c (t)) dt. (4.20)
This is the standard expression which corresponds to (2.11) or (3.1) in discrete time.
Again, instantaneous utility is given by n(.) . It depends here on consumption only,
where households consume continuously at each instant t. Impatience is captured as
before by the time preference rate j: Higher values attached to present consumption
are captured by the discount function c
j[tt[
. whose discrete time analog in (3.1) is
,
tt
.
Using (4.9), dierentiating with respect to time gives
_
l (t) = n(c (t)) +
_
1
t
d
dt
_
c
j[tt[
n(c (t))
dt = n(c (t)) +jl (t) .

Solving this linear dierential equation forward gives
l (t) = c
j[Tt[
l (1) +
_
T
t
c
j[tt[
n(c (t)) dt.
Letting 1 go to innity, we have
l (t) =
_
T
t
c
j[tt[
n(c (t)) dt + lim
T!1
c
j[Tt[
l (1) .
The second term is related to the transversality condition.
4.3.6 Example 3: A growth model
Consider an example inspired by growth theory. Let the capital stock of the economy
follow
_
1 = 1 o1, gross investment minus depreciation gives the net increase of the
capital stock. Let gross investment be determined by a constant saving rate times
output, 1 = :1 (1) and the technology be given by a linear 1 specication (as in
e.g. Rebelo, 1991). The complete dierential equation then reads
_
1 = :1 o1 = (: o) 1.
Its solution is (see ch. 4.3.2) 1 (t) = c
(cc)t
. As in the qualitative analysis above, we
found a multitude of solutions (a Schar von Lsungen), depending on the constant
. If we specify an initial condition, say we know the capital stock at t = 0. i.e.
1 (0) = 1
0
, then we can x the constant by = 1
0
and our solution nally reads
1 (t) = 1
0
c
(cc)t
.
71
4.4 Linear dierential equation systems
A dierential equation system consists of 2 or more dierential equations which are
mutually related to each other. An example is
_ r (t) = r (t) +/.
where the vector r (t) is given by r = (r
1
. r
2
. r
S
. .... r
a
)
0
. is an : : matrix with
elements c
i)
and / is a vector / = (/
1
. /
2
. /
S
. .... /
a
)
0
. Note that elements of and /
can be functions of time but not functions of r.
Such a system can be solved in various ways, e.g. by determining so-called Eigen-
values and Eigenvectors. These systems either result from economic models directly
or are the outcome of a linearization of some non-linear system around a steady state.
This latter approach plays an important role for local stability analyses (compared to
the global analyses we undertook above with phase diagrams). These local stability
analyses can be performed for systems of almost arbitrary dimension and are there-
fore more general and (for the local sourrounding of a steady state) more informative
than phase diagram analyses.
Please see the references in Further reading on many textbooks that treat these
issues.
There are very many textbooks that treat dierential equations and dierential equa-
tion systems. Any library search tool will provide very many hits. This chapter ows
insights to Gandolfo (1996) on phase diagram analysis and dierential equations and
- inter alia - to Brock and Malliaris (1989), Braun (1975) and Chiang (1984) on dif-
ferential equations. See also Gandolfo (1996) on dierential equation systems. The
predator-prey model is treated in various biology textbooks. It can also be found on
many sites in the internet. The Leibniz rule was taken from Fichtenholz (1997) and
can be found in many other textbooks on dierentiation and integration.
72
Exercises chapter 4
Using phase diagrams
1. Phase diagram I
Consider the following dierential equation system,
_ r
1
= , (r
1
. r
2
) . _ r
2
= q (r
1
. r
2
) .
Assume
,
a
1
(r
1
. r
2
) < 0. q
a
2
(r
1
. r
2
) < 0.
dr
2
dr
1
)(a
1
,a
2
)=0
< 0.
dr
2
dr
1
j(a
1
,a
2
)=0
0.
a) Plot a phase diagram for the positive quadrant.
b) What type of xpoint can be identied with this setup?
2. Phase diagram II
(a) Plot 2 phase diagrams for
_ r = r c. _ = /; c 0. (4.21)
by varying the parameter b.
(b) What type of xpoints do you nd?
3. Phase diagram III
(a) Plot paths through points marked by \ in the gure below.
(b) What type of xpoints are A and B?
y
O
x
& x = 0
& y = 0
A
B
73
4. Local stability analysis
Study local stability properties of the xpoint of the dierential equation system
(4.21).
5. Exam question (winter term 2005-06, Wrzburg)
Grossman und Helpman (1991) untersuchen ein Wachstumsmodell mit steigen-
der Anzahl der Varianten eines gleichen Produktes. Die reduzierte Form der
konomie wird durch folgendes, zweidimensionales Dierenzialgleichungssys-
tem beschrieben:
_ :(t) =
1
c

c
(t)
_ (t) = j (t)
1 c
:(t)
wobei 0 < c < 1 und c 0. Die Variablen (t) und :(t) bezeichnen jeweils
den Wert der Firma bzw. die Anzahl der Varianten eines gleichen Produktes.
Die positiven Konstanten j und 1 bezeichnen jeweils die Zeitprferenzrate der
Haushalte bzw. die Anzahl der Erwerbspersonen in der konomie.
Zeichnen Sie das Phasendiagramm fr den positiven Quadranten. Bestimmen
Sie den Fixpunkt. Um was fr einen Fixpunkt handelt es sich?
6. Solving linear dierential equations
Solve c _ (t) +, (t) = . (:) = 17 for
(a) t :.
(b) t < :.
(c) What is the forward, what is the backward solution? How do they relate
to each other?
7. Derivatives of integrals
Compute the following derivatives.
(a)
o
o j
_
j
o
, (:) d:.
(b)
o
o j
_
j
o
, (:. ) d:.
(c)
o
o j
_
b
o
, () d.
(d)
o
o j
_
, () d.
(e) Show that
_
b
o
_ rdt = [r]
b
o
_
b
o
r _ dt.
8. Intertemporal and dynamic budget constraints
Consider the intertemporal budget constraint which equates the discounted ex-
penditure stream to asset holdings plus a discounted income stream,
_
1
t
1
v
(t) 1 (t) dt = (t) +
_
1
t
1
v
(t) 1 (t) dt. (4.22)
74
where
1
v
(t) = exp
_
_
t
t
: (:) d:
_
. (4.23)
A dynamic budget constraint reads
1 (t) +
_
(t) = : (t) (t) +1 (t) . (4.24)
(a) Show that solving the dynamic budget constraint yields the intertemporal
budget constraint if and only if
lim
T!1
(1) exp
_
_
T
t
: (t) dt
_
= 0. (4.25)
(b) Show that dierentiating the intertemporal budget constraint yields the
dynamic budget constraint.
9. A budget constraint with many assets
Consider an economy with two assets whose prices are
i
(t). A household owns
:
i
(t) assets of each type such that total wealth at time t of the household is given
by c (t) =
1
(t) :
1
(t) +
2
(t) :
2
(t) . Each asset pays a ow of dividends :
i
(t) .
Let the household earn wage income n(t) and spend j (t) c (t) on consumption
per unit of time. Show that the budget constraint of the household is given by
_ c (t) = : (t) c (t) +n(t) j (t) c (t)
where the interest rates are dened by
: (t) = o (t) :
1
(t) + (1 o (t)) :
2
(t) . :
i
(t) =
:
i
(t) + _
i
(t)
i
(t)
and o (t) =
1
(t) :
1
(t) ,c (t) is dened as the share of wealth held in asset 1.
10. Optimal saving
Let optimal saving and consumption behaviour (see ch. 5, e.g. eq. (5.5)) be
described by the two-dimensional system
_ c = qc. _ c = :c +n c.
where q is the growth rate of consumption, given e.g. by q = : j or q =
(: j) ,o. Solve this system for time paths of consumption c and wealth c.
11. ODE systems
Study transitional dynamics in a two-country world.
(a) Compute time paths of the number :
i
(t) of rms in country i. The laws
of motion are given by (Grossman and Helpman, 1991; Wlde, 1996)
_ :
i
=
_
:
+:
1
_
1
i
:
i
c(1 +j) . i = . 1. 1 = 1
+1
1
. c. j 0.
Hint: Eigenvalues are q = (1 c) 1 cj 0 and ` = c(1 +j).
(b) Plot the time path of :
. Choose appropriate initial conditions.

75
5 Optimal control theory - Hamiltonians
One widely used approach to solve deterministic intertemporal optimization problems
in continuous time is to use the so-called Hamiltonian function. Given a certain
maximization problem, this function can be adapted - just like a recipe - to yield a
straightforward solution. The rst section will provide an introductory example. It
shows how easy it can sometimes be to solve a maximization problem.
It is useful to understand, however, where the Hamiltonian comes from. A list
of examples can never be complete, so it helps to be able to derive the appropriate
optimality conditions in general. This will be done in the subsequent section. Section
5.3 then discusses how boundary conditions for maximization problems look like and
how they can be motivated. The innite planning horizon problem is then presented
and solved in section 5.4 which includes a section on transversality and bounded-
ness conditions. Various examples follow in section 5.5. Section 5.6 nally shows
how to work with present-value Hamiltonians and how they relate to current-value
Hamiltonians (which are the ones used in all previous sections).
5.1 An introductory example
Consider an individual that wants to maximize a utility function as encountered
already in (4.20),
max
fc(t)g
_
T
t
c
j[tt[
ln c (t) dt. (5.1)
The budget constraint of this individual equates changes in wealth, _ c (t) . to current
savings, i.e. the dierence between capital and labour income, : (t) c (t) +n(t) . and
consumption expenditure c (t),
_ c (t) = : (t) c (t) +n(t) c (t) .
This maximization problemcan be solved by using the present-value or the current-
value Hamiltonian. We will work with the current-value Hamiltonian here and in
what follows (see section 5.6 on the present-value Hamiltonian). The current-value
Hamiltonian reads
H = ln c (t) +`(t) [: (t) c (t) +n(t) c (t)] . (5.2)
where `(t) is called the costate variable; costate as it corresponds to the state
variable c (t) . In maximization problems with more than one state variable, there is
one costate variable for each state variable. The costate variable could also be called
Hamilton multiplier - similar to the Lagrange multiplier. We show further below that
`(t) is the shadow price of wealth.
Omitting time arguments, optimality conditions are
JH
Jc
=
1
c
` = 0. (5.3)
_
` = j`
JH
Jc
= j` :`. (5.4)
76
The rst-order condition in (5.3) is a usual optimality condition: the derivative of the
Hamiltonian (5.2) with respect to the consumption level c must be zero. The second
optimality condition - at this stage - just comes out of the blue. Its origin will be
discussed in a second. Applying logs to the rst rst-order condition, ln c = ln `.
and computing derivatives with respect to time yields _ c,c =
_
`,`. Inserting into
(5.4) gives the Keynes-Ramsey rule
_ c
c
= j : =
_ c
c
= : j. (5.5)
5.2 Deriving laws of motion
This subsection shows where the Hamiltonian comes from. More precisely, it shows
how the Hamiltonian can be deduced from the optimality conditions resulting from
a Lagrange approach. The Hamiltonian can therefore be seen as a shortcut which is
faster than the Lagrange approach but leads to (it needs to) identical results.
5.2.1 Solving by using the Lagrangian
Consider the objective function
l (t) =
_
T
t
c
j[tt[
n( (t) . . (t) . t) dt (5.6)
which we now maximize subject to the constraint
_ (t) = Q( (t) . . (t) . t) . (5.7)
The function Q(.) is left fairly unspecied. It could be a budget constraint of a
household, a resource constraint of an economy or some other constraint. We assume
that Q(.) has nice properties, i.e. it is continuous and dierentiable everywhere.
The objective function is maximized by an appropriate choice of the path .(t) of
control variables.
This problem can be solved by using the Lagrangian
/ =
_
T
t
c
j[tt[
n(t) dt +
_
T
t
j (t) [Q(t) _ (t)] dt.
The utility function n(.) and other functions are presented as n(t) . This shortens
notation compared to full expressions in (5.6) and (5.7). The intuition behind this
Lagrangian is similar to the one behind the Lagrangian in the discrete time case in
(3.41) in ch. 3.7 where we also looked at a setup with many constraints. The rst
part is simply the objective function. The second part refers to the constraints. In the
discrete-time case, each point in time had its own constraint with its own Lagrange
multiplier. Here, the constraint (5.7) holds for a continuum of points t. Hence,
instead of the sum in the discrete case we now have an integral over the product of
multipliers j (t) and constraints.
77
This Lagrangian can be rewritten as follows,
/ =
_
T
t
c
j[tt[
n(t) +j (t) Q(t) dt
_
T
t
j (t) _ (t) dt
=
_
T
t
c
j[tt[
n(t) +j (t) Q(t) dt +
_
T
t
_ j (t) (t) dt [j (t) (t)]
T
t
(5.8)
where the last step integrated by parts and [j (t) (t)]
T
t
is the integral function of
j (t) (t) evaluated at 1 minus its level at t. Now assume that we could choose
not only the control variable . (t) but also the state variable (t) at each point in
time. The intuition for this is the same as in discrete time in ch. 3.7.2. We maximize
therefore this Lagrangian with respect to and . at one particular t [t. 1].
For the control variable .. we get a rst-order condition
c
j[tt[
n
:
(t) +j (t) Q
:
(t) = 0 =n
:
+c
j[tt[
j (t) Q
:
= 0.
When we dene
`(t) = c
j[tt[
j (t) . (5.9)
we nd
n
:
+`(t) Q
:
= 0. (5.10)
For the state variable . we obtain
c
j[tt[
n
j
(t) +j (t) Q
j
(t) + _ j (t) = 0 =
n
j
(t) j (t) c
j[tt[
Q
j
= c
j[tt[
_ j (t) . (5.11)
Dierentiating (5.9) with respect to time t gives
_
`(t) = jc
j[tt[
j (t) +c
j[tt[
_ j (t) = j`(t) +c
j[tt[
_ j (t) .
Inserting (5.11), we obtain
_
`(t) = j`(t) n
j
(t) `(t) Q
j
(t) . (5.12)
Equations (5.10) and (5.12) are the two optimality conditions that solve the above
maximization problem jointly with the constraint (5.7). We have three equations
which x three variables: The rst condition (5.10) determines the optimal level of
the control variable .. As this optimality condition holds for each point in time t. it
xes an entire path for .. The second optimality condition (5.12) xes a time path for
`. It makes sure, by letting the costate ` follow an appropriate path, that the level
of the state variable (which is not instantaneously adjustable as the maximization of
the Lagrangian would suggest) is as if it had been optimally chosen at each instant.
The constraint (5.7), nally, xes the time path for the state variable .
78
5.2.2 Hamiltonians as a shortcut
Let us now see how Hamiltonians can be justied. Dene the Hamiltonian similar to
(5.2),
H = n(t) +`(t) Q(t) . (5.13)
In fact, this Hamiltonian shows the general structure of Hamiltonians. Take the
instantaneous utility level (or any other function behind the discount term in the
objective function) and add the costate variable ` times the right-hand side of the
constraint. Optimality conditions are then
H
:
= 0. (5.14)
_
` = j` H
j
. (5.15)
These conditions were already used in (5.3) and (5.4) in the introductory example
in the previous chapter and in (5.10) and (5.12): When (5.13) is inserted into (5.14)
and (5.15), this yields equations (5.10) and (5.12). Hamiltonians are therefore just
a shortcut that allow to nd solutions faster than in the case where Lagrangians are
used. Note for later purposes that both ` and j have time t as an argument.
There is an interpretation of the costate variable ` which we simply state at this
point (see ch. 6.2 for a formal derivation): The derivative of the objective function
with respect to the state variable at t, evaluated on the optimal path, equals the
value of the corresponding costate variable at t. Hence, just as in the static Lagrange
case, the costate variable measures the change in utility as a result of a change in
endowment (i.e. in the state variable). Expressing this formally, dene the value
function as \ ( (t)) = max
f:(t)g
l (t) . identical in spirit to the value function in
dynamic programming as we got to know it in discrete time. The derivative of the
value function, the shadow price \
0
( (t)) . is then the change in utility when behaving
optimally resulting from a change in the state (t) . This derivative equals the costate
variable, \
0
( (t)) = `.
5.3 Boundary conditions and sucient conditions
So far, maximization problems were presented without boundary conditions. Usually,
however, boundary conditions are part of the maximization problem. We will now
consider two cases. Both cases will later be illustrated in the phase diagram of section
5.5.
5.3.1 Free value of state variable at endpoint
Many problems are of the form (5.6) and (5.7) were boundary values for the state
variable are given by
(0) =
0
. (1) free. (5.16)
The rst condition is the usual initial value condition. The second condition allows
the state variable to be freely chosen for the end of the planning horizon.
79
We can use the current-value Hamiltonian (5.13) which gives optimality conditions
(5.14) and (5.15). In addition (cf. Feichtinger and Hartl, 1986, p. 20),
`(1) = 0. (5.17)
As ` is the shadow price of the state variable, this simply means that the value of
the state variable at 1 needs to be zero.
5.3.2 Fixed value of state variable at endpoint
Now consider (5.6) and (5.7) with one initial and one endpoint condition,
(0) =
0
. (1) =
T
. (5.18)
In order for this problem to make sense, we assume that a feasible solution exists. This
should generally be the case, but it is not obvious: Consider again the introductory
example in ch. 5.1. Let the endpoint condition be given by the agent is very rich in
1. i.e. c (1) =very large. If c (1) is too large, even zero consumption at each point
in time, c (t) = 0 \t [t. 1] . would not allow wealth c to be as large as required by
c (1) . In this case, no feasible solution would exist.
We assume, however, that a feasible solution exists. Optimality conditions are
then identical to (5.14) and (5.15), plus initial and boundary values. The dierence
from this approach to the one before is that now (1) =
T
is exogenously given, i.e.
part of the maximization problem, whereas before the corresponding (1) = 0 was
endogenously determined as a necessary condition for optimality.
5.3.3 Sucient conditions
So far, we only presented conditions that are necessary for a maximum. We do
not yet know, however, whether these conditions are also sucient. (Think of the
static maximization problem max
a
, (r) where ,
0
(r
) = 0 is necessary for an interior

maximum but not sucient as ,
0
(r
+) could be positive for any ,= 0.) The

following theorem from Kamien and Schwartz (1991, ch. 3, p. 133) tells us more.
Theorem 5 Necessary conditions are sucient if either
(i) the functions n(.) and Q (.) in (5.6) and (5.7) are concave in and . and if
j (t) _ 0\t < t < 1.
(ii) Q(.) is linear in y and z for any j(t).
(iii) Q(.) is convex and j (t) _ 0 \ t < t < 1.
For concavitiy, the proof of this theorememploys ,+, _ (r + r) ,
a
+(n + n) ,
&
.
, = G. Q. There is a further theorem due to Arrow which is also presented by Kamien
and Schwartz (1991, ch. 15, p. 221).
80
5.4 The innite horizon
5.4.1 The maximization problem
In the innite horizon case, the objective function has the same structure as before in
e.g. (5.6) only that the nite time horizon 1 is replaced by an innite time horizon
. The constraints are unchanged and the maximization problem reads
max
f:(t)g
_
1
t
c
j[tt[
n( (t) . . (t) . t) dt.
subject to
_ (t) = Q( (t) . . (t) . t) .
(0) =
0
. (5.19)
We need to assume for this problem that the integral
_
1
t
c
j[tt[
n(.) dt converges
for all feasible paths of (t) and . (t), otherwise the optimality criterion must be
redened. This boundedness condition is important only in this innite horizon case.
If individuals have nite horizon and the utility function n(.) is continuous over the
entire planning period, the objective function is nite and the boundedness problem
disappears. Clearly, making such an assumption is not always innocuous and one
should check, at least after having solved the maximization problem, whether the
objective function converges indeed. This will be done in ch. 5.4.3.
The current-value Hamiltonian as in (5.13) is dened by
H = n(.) +`(t) Q(.) .
and optimality conditions are (5.14) and (5.15), i.e.
JH
J.
= 0.
_
` = j`
JH
J
.
Hence, we have identical optimality conditions to the case of a nite horizon.
5.4.2 The transversality condition
The solution to the maximization problem with an innite horizon also led to a
system of two dierential equations. One boundary condition is provided by (5.19)
for the state variable. Hence, again, we need a second condition to pin down the
initial consumption level. In the nite horizon case, we had conditions of the type
1 (1) = 1
T
or `(1) = 0 in (5.17) and (5.18). The condition used here, which is
related to these conditions, is the transversality condition (TVC),
lim
t!1
j (t) [ (t)
(t)] = 0.
where
(t) is the path of (t) for an optimal choice of control variables.

81
A conceptionally dierent condition from the transversality condition is the No-
Ponzi game condition (4.25)
lim
T!1
(1) exp
_
_
T
t
: (t) dt
_
= 0
where : = JH,J. Note that this No-Ponzi game condition can be rewritten as
lim
T!1
c
jT
`(1) (1) = lim
T!1
j (1) (1) = 0. (5.20)
where the fact that
_
`,` = : j implies `c
jt
= `
0
c
R
T
I
v(t)ot
was used. The for-
mulations in (5.20) of the No-Ponzi game condition is frequently encountered and is
sometimes even called TVC.
There is a considerable literature on the necessity and suciency of the TVC.
Some examples include Mangasarian (1966), Arrow (1968), Arrow and Kurz (1970),
Araujo and Scheinkman (1983), Lonard and Long (1992, p. 288-289), Chiang (1992,
p. 217, p. 252) and Kamihigashi (2001). Counterexamples that the TVC is not
necessary are provided by Michel (1982) and Shell (1969).
5.4.3 The boundedness condition
When introducing an innite horizon objective function, it was stressed right after
(5.19) that objective functions must be nite for any feasible paths of the control
variable. Otherwise, overall utility l (t) would be innitly large and there would
be no objective for optimizing - we are already innitly happy! This problem of
unlimited happiness is particularily severe in models where control variables grow
with a constant rate, think e.g. of consumption in a model of growth. A pragmatic
approach to checking whether growth is not too high is to rst assume that it is not
to high, to then maximize the utility function and to afterwards check whether the
initial assumption is satised.
As an example, consider the intertemporal utility function l (t) =
_
1
t
c
j[tt[
n(C (t)) dt.
where o 0 and the instantaneous utility function n(C (t)) is characterized by con-
stant relative risk aversion (CRRA),
n(C (t)) =
C (t)
1o
1
1 o
.
Assume that consumption grows with a rate of q, where this growth rate results from
utility maximization. Think of this q as representing e.g. the dierence between
the interest rate and the time preference rate, corrected by intertemporal elasticity
of substitution, as will be found later e.g. in the Keynes-Ramsey rule (5.33), i.e.
_ c,c = (: j) ,o = q. With this exponential growth of consumption, the utility
function becomes
l (t) = (1 o)
1
C
0
_
1
t
c
j[tt[
_
c
(1o)j[tt[
1
_
dt.
82
This integral is bounded only if
(1 o) q j < 0.
This can formally be seen by computing the integral explicitly and checking under
which conditions it is nite. Intuitively, this condition makes sense: Instantaneous
utility from consumption grows by a rate of (1 o) q. Impatience implies that future
utility is discounted by the rate j. Only if this time preference rate j is large enough,
the overall expression within the integral, c
j[tt[
_
C (t)
1o
1
_
. falls in t.
This problem has been discovered a relatively long time ago and received renewed
attention in the 90s when the new growth theory was being developed. A more general
treatment of this problem was undertaken by von Weizscker (1965) who compares
utility levels in unbounded circumstances by using overtaking criteria.
5.5 Examples
This section presents various examples that show both how to compute optimality
conditions and how to understand the predictions of the optimality conditions. We
will return to a phase-diagram analysis and understand the meaning of xed and free
values of state variables at the end points.
5.5.1 Free value at end point - Adjustment costs
Let us look at a rm that operates under adjustment costs. Capital can not be rented
instantaneously on a spot market but its installation is costly. The crucial implication
of this simple generalization of the standard theory of production implies that rms
all of a sudden have an intertemporal and no longer a static optimization problem.
As one consequence, factors of production are then no longer paid their value marginal
product as in static rm problems.
The model
A rm maximizes the present value
0
of its future instantaneous prots : (t),
0
=
_
T
0
c
vt
: (t) dt. (5.21)
subject to a capital accumulation constraint
_
1 (t) = 1 (t) o 1 (t) . (5.22)
Gross investment 1 (t) minus depreciation o1 (t) gives the net increase of the rms
capital stock. Instantaneous prots are given by the dierence between revenue and
cost,
: (t) = j1 (1 (t)) (1 (t)) .
Revenue is given by j1 (1 (t)) where the production technology 1 (.) employs capital
only. Output increases in capital input, 1
0
(.) 0. but to a decreasing extent, 1
00
(.) <
83
0. Costs of rm are given by the cost function (1 (t)) . As the rm owns capital, it
does not need to pay any rental costs for capital. The only costs that are captured
by (.) are adjustment costs. They include both the cost of buying and of installing
capital. The initial capital stock is given by 1 (0) = 1
0
. The interest rate : is
exogenous to the rm.
Solution
The current value Hamiltonian reads
H = j1 (1 (t)) (1 (t)) +`(t) [1 (t) o 1]
and optimality conditions are
H
1
=
0
(1 (t)) +`(t) = 0. (5.23)
_
`(t) = : ` H
1
(t) = : ` j1
0
(1 (t)) +`o
= (: +o) ` j1
0
(1 (t)) . (5.24)
`(1) = 0 =
0
(1 (1)) = 0.
The optimality condition for ` shows that the value marginal product of capital,
j1
0
(1 (t)) . still plays a role and it is still compared to the rental price : of capital,
but there is no longer an equality as in static models of the rm. We will return to
this point in exercise 2 of ch. 6.
Optimality conditions can be presented in a simpler way (i.e. with less endogenous
variables). First, solve the rst optimality condition for the costate variable and
compute the time derivative,
_
`(t) =
00
(1 (t))
_
1 (t) .
Second, insert this into the second optimality condition (5.24) to nd
00
(1 (t))
_
1 (t) = (: +o)
0
(1 (t)) j1
0
(1 (t)) . (5.25)
This equation, together with the capital accumulation constraint (5.22), is a two-
dimensional dierential equation system that can be solved, given initial conditions
and some boundary condition. As a solution to a dierential equation system is
usually a path of time variant variables, this shows that with adjustment costs, rms
indeed have an intertemporal problem.
An example
Now assume adjustment costs are of the form
(1) = 1 +1
2
,2. (5.26)
The price to be paid per unit of capital is given by the constant and costs of
installation are given by 1
2
,2. This quadratic term captures the idea that installation
84
costs are low and do not increase fast, i.e. underproportionally to the new capital
stock, at low levels of 1 but increase overproptionally when 1 becomes large. Then,
optimality requires (5.22) and, from inserting (5.26) into (5.25),
_
1 (t) = (: +o) ( +1 (t)) j1
0
(1 (t)) (5.27)
A phase diagram using (5.22) and (5.27) is plotted in the following gure.
0
K
K
I
0 = K
&
0 = I
&
Figure 26 A rm with adjustment cost
We now have to select one of this innity of paths that start at 1
0
. What is the
correct investment level 1
0
? Given that the value of the state variable at the end
point is freely chosen, the optimal path satises
`(1) = 0 =
0
(1 (1)) = 0 =1 (1) = 0.
Hence, the trajectory where the investment level is zero at 1 is the optimal one.
5.5.2 Fixed value at end point - Adjustment costs II
To illustrate this problem, consider the example of the last section where we require
a certain stock of capital at 1. The phase diagram then looks identical. The higher
the endpoint stock of capital, the more you have to invest. If it is relatively low but
time is long, it might be advisable to rst approach the steady state and then fall
back to capital stock 1
T
.
Note the turnpike theorem in this context!
85
5.5.3 Innite horizon - Optimal consumption paths
Let us now look at an example with innite horizon. We focus on optimal behaviour
of a consumer. The classic reference is Ramsey (1928) or Arrow and Kurz (1969).
The problem can be posed in at least two ways. In either case, one part of the
problem is the intertemporal utility function
l (t) =
_
1
t
c
j[tt[
n(c (t)) dt. (5.28)
Due to the general instantaneous utility function n(c (t)), it is somewhat more general
than e.g. (5.1). The second part of the maximization problem is a constraint limiting
the total amount of consumption. Without such a constraint, maximizing (5.28)
would be trivial (or meaningless): With n
0
(c (t)) 0 maximizing the objective simply
means setting c (t) to innity. The way this constraint is expressed determines in
which way the problem is most straightforwardly solved.
Solving by Lagrange
The constraint to (5.28) is given by a budget constraint. The rst way in which
this budget constraint can be expressed is, again, the intertemporal formulation,
_
1
t
1
v
(t) 1 (t) dt = c (t) +
_
1
t
1
v
(t) n(t) dt. (5.29)
where 1 (t) = j (t) c (t) and 1
v
(t) = c
R
:
I
v(&)o&
. The maximization problem is then
given by: maximize (5.28) by choosing a path c(t) subject to the budget constraint
(5.29).
We build the Lagrangean with ` as the time-independent Lagrange multiplier
/ =
_
1
t
c
j[tt[
n(c (t)) dt
`
__
1
t
1
v
(t) 1 (t) dt c (t)
_
1
t
1
v
(t) n(t) dt
_
.
Note that in contrast to section 5.2.1 where a continuum of constraints implied a con-
tinuum of Lagrange multipliers (or, in an alternative interpretation, a time-dependent
multiplier) there is only one constraint here.
The rst-order conditions are (5.29) and the partial derivative with respect to
consumption c (t) at one specic point t in time,
/
c(t)
= c
j[tt[
n
0
(c (t)) `1
v
(t) j (t) = 0
=1
v
(t)
1
c
j[tt[
j (t)
1
= `n
0
(c (t))
1
.
Note that this rst order condition represents an innite number of rst order con-
ditions: one for each point t in time between t and innity. The integral is not part
of the rst-order condition as maximization takes place seperatly for each t. Note
86
the analogy to maximizing a sum as e.g. in (3.4). The integration variable t here
corresponds to the summation index t in (3.4). When one specic point t in time is
chosen (say t = 17), all derivatives of other consumption levels with respect to this
specic c
t
are zero.
Applying logs yields
_
t
t
: (n) dn j [t t] ln j (t) = ln ` ln n
0
(c (t)) .
Dierentiating with respect to time t gives the Keynes-Ramsey rule
n
00
(c (t))
n
0
(c (t))
_ c (t) = : (t)
_ j (t)
j (t)
j. (5.30)
The Lagrange multiplier ` drops out as it is a constant. Note that
&
00
(c(t))
&
0
(c(t))
is Arrows
measure of absolute risk aversion which is a measure of the curvature of the utility
function.
With a logarithmic utility function, n(c (t)) = ln c (t), n
0
(c (t)) =
1
c(t)
and
n
00
(c (t)) =
1
c(t)
2
and
_ c (t)
c (t)
= : (t)
_ j (t)
j (t)
j. (5.31)
Employing the Hamiltonian
In contrast to above, the utility function (5.28) here is maximized subject to the
dynamic (or ow) budget constraint,
j (t) c (t) + _ c (t) = : (t) c (t) +n(t) .
The solution is an exercise.
The interest rate eect on consumption
As an application for the methods we got to know so far, imagine there is a
sudden discovery of a new technology in an economy. All of a sudden, computers
or cell-phones or the internet is available on a large scale in the economy. Imagine
further that this implies a sudden increase in the returns to investment (i.e. the
interest rate :): For any Euro invested, more comes out than before the discovery of
the new technology. What is the eect of this discovery on consumption? To ask this
more precisely: what is the eect of a change in the interest rate on consumption?
To answer this question, we study a maximization problem as the one just solved,
i.e. the objective function is (5.28) and the constraint is (5.29). We simplify the
maximization problem, however, by assuming a a CRRA utility function n(c (t)) =
_
c (t)
1o
1
_
, (1 o) . a constant interest rate and a price being equal to one (imag-
ine the consumption good is the numeraire). This implies that the budget constraint
reads
_
1
t
c
v[tt[
c (t) dt = c (t) +
_
1
t
c
v[tt[
n(t) dt (5.32)
87
and the Keynes-Ramsey rule becomes, from inserting the CRRA n(c (t)) into (5.30),
_ c (t)
c (t)
=
: j
o
. (5.33)
One eect, the growth eect, is straightforward from (5.33) or also from (5.30). A
higher interest rate, ceteris paribus, increases the growth rate of consumption. The
second eect, the eect on the level of consumption, is less obvious, however. In order
to understand it, we undertake the following steps. This is in principle similar to the
analysis of level eects in ch. 3.5.2.
First, we solve the linear dierential equation in c (t) given by (5.33). Following
ch. 4.3, we nd
c (t) = c (t) c
rp
(tt)
. (5.34)
Consumption, starting today in t with a level of c (t) grows exponentially over time
at the rate (: j) ,o to reach the level c (t) at some future t t.
In the second step, we insert this solution into the left-hand side of the budget
constraint (5.32) and nd
_
1
t
c
v[tt[
c (t) c
rp
(tt)
dt = c (t)
_
1
t
c
(
v
rp
)
[tt[
dt = c (t)
1
_
:
vj
o
_
_
c
(
v
rp
)
[tt[
_
1
t
.
The simplication stems from the fact that c (t) . the initial consumption level, can
be pulled out of the term
_
1
t
c
v[tt[
c (t) dt. representing the present value of current
and future consumption expenditure. Please note that c (t) could be pulled out of
the integral also in the case of a non-constant interest rate. Note also that we do not
need to know what the level of c (t) is, it is enough to know that there is some c (t)
in the solution (5.34), whatever its level.
With a constant interest rate, the remaining integral can be solved explicitely.
First note that
_
:
vj
o
_
must be negative. Otherwise consumption growth would
exceed the interest rate and a boundedness condition for the objective function like in
ch. 5.4.3 would eventually be violated. Hence, we assume :
vj
o
= (1 o) : < j.
Clearly, this holds in a steady state where : = j. Therefore, we obtain for the present
value of consumption expenditure
c (t)
1
_
:
vj
o
_
_
c
(
v
rp
)
[tt[
_
1
t
=
c (t)
:
vj
o
=
oc (t)
j (1 o) :
and, inserted into the budget constraint, this yields
c (t) =
j (1 o) :
o
_
c (t) +
_
1
t
c
v[tt[
n(t) dt
_
. (5.35)
For the special case of a logarithmic utility function, the fraction in front of the curly
brackets simplies to j (as o = 1), a result obtained e.g. by Blanchard (1985).
After these two steps, we have two results, both visible in (5.35). One result
shows that initial consumption c (t) is a fraction out of wealth of the household.
88
Wealth needs to be understood in a more general sense than usually, however: It
is nancial wealth c (t) plus, what could be called human wealth (in an economic,
i.e. material sense), the present value of labour income,
_
1
t
c
v[tt[
n(t) dt. Going
beyond t today and realizing that this analysis can be undertaken for any point in
time, the relationship (5.35) of course holds on any point of an optimal consumption
path. The second result is a relationship between the level of consumption and the
interest rate, our original question.
We now need to understand the derivative dc (t) ,d: in order to exploit (5.35)
more. If we focus only on the term in front of the curly brackets, we nd for the
change in the level of consumption when the interest rate changes
dc (t)
d:
=
1 o
o
. R 0 =o R 1.
The consumption level increases when the interest rate rises if o is larger than one, i.e.
if the intertemporal elasticity of substitution o
1
is smaller than unity. This is prob-
ably the empirically more plausible case (compared to o < 1) on the aggregate level.
See however Vissing-Jrgensen (2002) for micro-evidence where the intertemporal
elasticity of substitution can be much larger than unity. This nding is summarized
in the following gure.
N
N
ti:c
:
1
t
c
2
(t)
|:c(t)
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
c
2
(t)
.
.
.
.
.
.
.
.
.
.
.
.
:
2
:
1
o 1
Figure 27 An interest rate change and consumption growth and level for o 1
5.5.4 The central planner and capital accumulation
The setup
This example studies the classic central planner problem: First, there is a social
welfare function like (5.1), expressed slightly more generally as
max
fC(t)g
_
1
t
c
j[tt[
n(C (t)) dt.
89
The generalization consists in the innite planning horizon and the general instanta-
neous utility function n(c (t)) . We will specify it in the most common version,
n(C) =
C
1o
1
1 o
. (5.36)
where the intertemporal elasticity of substition is constant and given by o
1
. The
coecient of relative risk aversion (dened by n
00
(C) C,n
0
(C)) is also constant and
given by o. While the latter aspect is of importance only in models with uncertainty,
it will turn out that this close relationship between this measure of risk-aversion and
the intertemporal elasticity of substitution is not always desirable. A way out are
Epstein-Zin preferences.
Second, there is a resource constraint that requires that net capital investment is
given by the dierence between output 1 (1. 1), depreciation o1 and consumption
C.
_
1 (t) = 1 (1 (t) . 1) o1 (t) C (t) . (5.37)
Assuming for simplicity that the labour force 1 is constant, this completely describes
this central planner problem.
The choice variable of the planner is the consumption level C (t) . to be determined
for each point in time between today t and the far future . The fundamental trade-
o consists in the utility increasing eect of more consumption visible from (5.36) and
the net-investment descreasing eect of more consumption visible from the resource
constraint (5.37). As less capital implies less consumption possibilities in the future,
the trade-o can also be described to consist in more consumption today vs. more
consumption in the future.
The solution
Let us solve this problem by employing the Hamiltonian
H = n(C (t)) +`(t) [1 (1 (t) . 1) o1 (t) C (t)] .
First order conditions are
n
0
(C (t)) = `(t) . (5.38)
_
`(t) = j`(t)
JH
J1
= j`(t) `(t) [1
1
(1 (t) . 1) o] .
Dierentiating the rst-order condition (5.38) with respect to time gives n
00
(C (t))
_
C (t) =
_
`(t) . Inserting this and (5.38) again into the second condition gives with same rear-
ranging
n
00
(C (t))
n
0
(C (t))
_
C (t) = 1
1
(1 (t) . 1) o j.
This is almost identical to the optimality rule we obtained on the individual level
in (5.30). The only dierence consists in capital C instead of c and 1
1
(1 (t) . 1) o
90
instead of : (t)
` j(t)
j(t)
. Instead of the real interest rate on the household level, we have
here the marginal productivity of capital minus depreciation on the aggregate level.
If we assumed a logarithmic instantaneous utility function, n
00
(C (t)) ,n
0
(C (t)) =
1,C (t) and the Keynes-Ramsey rule would look similar to (5.5) or (5.31). In our case
of the more general specication in (5.36), we nd n
00
(C (t)) ,n
0
(C (t)) = o,C (t)
such that the Keynes-Ramsey rule reads
_
C (t)
C (t)
=
1
1
(1 (t) . 1) o j
o
. (5.39)
This could be called the classic result on optimal consumption in general equilib-
rium. Consumption grows if marginal productivity of capital exceeds the sum of the
depreciation rate and the time preference rate. The higher the intertemporal elas-
ticity of substition 1,o. the stronger consumption growth reacts to the dierences
1
1
(1 (t) . 1) o j.
A phase diagram analysis
The resource constraint of the economy in (5.37) plus the Keynes-Ramsey rule
in (5.39) represent a two-dimensional dierential equation system which, given two
boundary conditions, give a unique solution for time paths C (t) and 1 (t) . These
two equations can be analyzed in a phase diagram. This is probably the most often
taught phase diagram in economics.
N
N
1
C
_
C = 0
_
1 = 0
1
0
1
C
_
'
_
'
_
'
_
'
_
'
N
N
N
N
Figure 28 Optimal central planner consumption
Zero-motion lines for capital and labour, respectively, are given by
_
1 (t) _ 0 =C (t) _ 1 (1 (t) . 1) o1 (t) . (5.40)
_
C (t) _ 0 =1
1
(1 (t) . 1) _ o +j. (5.41)
when the inequality signs hold as equalities. Zero motion lines are plotted in the
above gure.
When consumption lies above the 1 (1 (t) . 1) o1 (t) line, (5.40) tells us that
capital increases, below this line, it decreases. When the marginal productivity of
91
capital is larger than o +j. i.e. when the capital stock is suciently small, (5.41) tells
us that consumption increases. These laws of motion are plotted in gure 28 as well.
This allows us to draw trajectories . 1 and C which all satisfy (5.40) and (5.41).
Hence, again, we have a multitude of solutions for a dierential equation system.
As always, boundary conditions allow to pick the single solution to this system.
One boundary condition is the initial value 1
0
of the capital stock. The capital stock
is a state variable and therefore historically given at each point in time. It can not
jump. The second boundary condition should x the initial consumption level C
0
.
Consumption is a control or choice variable and can therefore jump or adjust to put
the economy on the optimal path.
The condition which provides the second boundary condition is, formally speaking,
the transversality condition (see ch. 5.4.2). Taking this formal route, one would have
to prove that starting with 1
0
at the consumption level that puts the economy on
path would violate the TVC. Similarily, it would have to be shown that path C or
any path other than 1 violates the TVC as well. Even though not often admitted, this
is not often done in practice. (For an exception, see exercise 5 in ch. 5.) Whenever
a saddle path is found in a phase diagram, it is argued that the saddle path is the
equilibrium path and the initial consumption level is such that the economy nds itself
on the path which approaches the steady state. While this is a practical approach, it
is also formally satised as this path satises the TVC indeed.
5.5.5 The matching approach to unemployment
Matching functions are widely used in e.g. labour economics and monetary economics.
We will present here the background for their use in labour market modeling, similar
to Pissarides (2000, ch. 1).
Aggregate unemployment
The unemployment rate in an economy is governed by two factors: the speed with
which new employment is created and the speed with which existing employment is
destroyed. The number of new matches per unit of time dt is given by a matching
function. It depends on the number of unemployed l, i.e. the number of those
potentially available to ll a vacancy, and the number of vacancies \ , : = :(l. \ ).
The number of lled jobs that are destroyed per unit of time is given by the product
of the separation rate : and employment, :1. Combining both components, the
evolution of the number of unemployed over time is given by
_
l = :1 :(l. \ ) . (5.42)
Dening employment as the dierence between the size of the labour force ` and the
number of unemployed l, 1 = ` l, we obtain
_
l = : [` l] :(l. \ ) = _ n = : [1 n] :(n. \,`) .
where we divided by the size of the labour force ` in the last step to obtain the
unemployment rate n = l,`. We also assumed constant returns to scale in the
92
matching function :(.) . Note that the matching function can further be rewritten
as
:
_
n.
\
`
_
= :
_
1.
\
l
_
n =
\
l
:
_
l
\
. 1
_
n = o (o) n (5.43)
and we obtain _ n = : [1 n] o (o) n which is equation (1.3) in Pissarides (2000).
Clearly, from (5.42), one can easily obtain the evolution of employment, using
again the denition 1 = ` l.
_
1 = :(` 1. \ ) :1. (5.44)
Firm behaviour
Given this matching process, the choice variable of a rmi is no longer employment
1
i
but the number of vacancies \
i
it creates. Each vacency implies costs measured
in units of the output good 1. The rm also employs capital 1
i
. Hence, the rms
optimization problem reads
max
f\
.
(t),1
.
(t)g
_
1
t
c
v[tt[
1 (1
i
. 1
i
) :1
i
n1
i
\
i
dt
subject to
_
1
i
= :(` 1. \ )
\
i
\
:1
i
.
The constraint now says - in contrast to (5.44) - that only a certain share of all
matches goes to the rm under consideration and that this share is given by the
share of the rms vacancies in total vacancies, \
i
,\ . Alternatively, this says that
the probability that a match in the economy as a whole lls a vacency of rm i
is given by the number of matches in the economy as a whole divided by the total
number of vacancies. As under constant returns to scale for : and using the denition
of (o) implicity in (5.43), :(` 1. \ ) ,\ = :(l,\. 1) = (o) . we can write the
rms constraint as
_
1
i
= (o) \
i
:1
i
.
Assuming small rms, this rate can (o) safely be assumed to be exogenous for the
rms maximization problem.
The current-value Hamiltonian for this problem reads
H = 1 (1
i
. 1
i
) :1
i
n1
i
\
i
+`
i
[ (o) \
i
:1
i
] .
and rst-order conditions are
H
1
.
= 1
1
.
(1
i
. 1
i
) : = 0. H
\
.
= +`
i
(o) = 0.
_
`
i
= :`
i
H
1
.
= :`
i
(1
1
.
(1
i
. 1
i
) n `
i
:) = (: +:) `
i
1
1
.
(1
i
. 1
i
) +n.
The rst condition for capital is the usual marginal productivity condition applied
to capital input. With CRTS production functions, this condition xes the capital
to labour ratio for each rm. This implies that the marginal product of labour is a
93
function of the interest only and therefore identical for all rms, independent of the
labour stock, i.e. the size of the rm. This changes the third condition to
_
` = (: +:) ` 1
1
(1. 1) +n (5.45)
which means that the shadow prices are identical for all rms.
The rst-order condition for vacencies, written as = (o) ` says that the mar-
ginal costs of a vacancy (which in this special case equal average and unit costs)
must be equal to revenue from a vacancy. This expected revenue is given by the
share (o) of vacencies that yield a match times the value of a match. The value of
a match to the rm is given by `. the shadow price of labour. The link between `
and the value of an additional unit of labour (or of the state variable, more generally
speaking) is analyzed in ch. 6.2.
General equilibrium
It appears as if the rst-order condition for vacencies was independent of the
number of vacancies opened by the rm. In fact, given this structure, the individual
rm follows a bang-bang policy. Either the optimal number of vacencies is zero or
innity. In general equilibrium, however, a higher number of vacancies, decreases the
share (o) of existing vacancies that get lled. Hence, this second condition holds in
general equilibrium and we can compute
_
`,` = _ (o) , (o) .
The third condition (5.45) can then be written as
_ (o)
(o)
= : +:
(o)
[1
1
(1. 1) n] (5.46)
which is an equation depending on the number of vacancies and employment only.
This equation together with (5.44) is a two-dimensional dierential equation system
which determines \ and 1. provided there are 2 boundary conditions for \ and 1
and wages and the interest rate are exogenously given. (In a more complete general
equilibrium setup, wages would be determined e.g. by Nash-bargaining. For this, we
need to derive value functions which is done in ch. 10.2).
An example for the matching function
Assume the matching function has CRTS and is of the CD type, : = l
c
\
o
.
As (o) = :(` 1. \ ) ,\. it follws that
` q(0)
q(0)
=
` n
n

`
\
\
. Given our Cobb-Douglas
assumption, we can write this as
_ (o)
(o)
= c
_
l
l
+,
_
\
\

_
\
\
= c
_
l
l
(1 ,)
_
\
\
.
94
Hence, the vacency equation (5.46) becomes
c
_
l
l
+ (1 ,)
_
\
\
= : +:
(o)
[1
1
(1. 1) n] =
(1 ,)
_
\
\
= : +:
(` 1)
c
\
1o
[1
1
(1. 1) n] +c
_
:
1
` 1

\
o
(` 1)
1c
_
where the last equality used (o) = (` 1)
c
\
o1
and (5.42) with : = l
c
\
o
.
Again, this equation with (5.44) is a two-dimensional dierential equation system
which determines \ and 1. just as just described in the general case.
5.6 The present value Hamiltonian
5.6.1 Problems without (or with implicit) discounting
The problem and its solution
Let the maximization problem be given by
max
:(t)
_
T
t
1 ( (t) . . (t) . t) dt (5.47)
subject to
_ (t) = Q( (t) . . (t) . t) (5.48)
(t) =
t
. (1) free (5.49)
where (t) 1
a
. . (t) 1
n
and Q = (Q
1
( (t) . . (t) . t) . Q
2
(.) . ... . Q
a
(.))
T
. A
feasible (zulssig) path is a pair ( (t) . . (t)) which satises (5.48) and (5.49). .(t)
is the vector of control variables, (t) is the vector of state variables.
Then dene the (present-value) Hamiltonian H as
H
1
= 1 (t) +j (t) Q(t) . (5.50)
where j(t) is the costate variable,
j (t) = (j
1
(t) . j
2
(t) . ... . j
a
(t)) (5.51)
Necessary conditions for an optimal solution are
H
1
:
= 0. (5.52)
_ j (t) = H
1
j
. (5.53)
and
j (1) = 0.
(Kamien and Schwartz, p. 126) in addition to (5.48) and (5.49). In order to have a
maximum, we need second order conditions H
::
< 0 to hold (Kamien and Schwartz).
95
Understanding its structure
The rst : equations (5.52) (. (t) 1
n
) solve the control variables .(t) as a
function of state and costate variables, (t) and j(t). Hence
.(t) = .((t). j(t)).
The next 2: equations (5.53) and (5.48) ( (t) 1
a
) constitute a 2: dimensional
dierential equation system. In order to solve it, one needs, in addition to the : initial
conditions given exogenously for the state variables by (5.49), : further conditions
for the costate variables. These are given by boundary value conditions j (1) = 0.
Suciency of these conditions results from theorem as e.g. in section 5.3.3.
5.6.2 Deriving laws of motion
As in the section on the current-value Hamiltonian, a derivation of the Hamiltonian
starting from the Lagrange function can be given for the present value Hamiltonian
as well.
Let the objective function be (5.47) that is to be maximized subject to the con-
straint (5.48) and, in addition, a static constraint
G( (t) . . (t) . t) = 0. (5.54)
This maximization problem can be solved by using a Lagrangian (cf. Intriligator,
ch. 14. A useful starting point is the derivation of the Lagrangian for the static case;
Intriligator, p. 28 - 30.) The Lagrangian reads
/ =
_
T
t
1 (.) +j (t) (Q(.) _ (t)) (t) G(.). t) dt
=
_
T
t
1 (.) +j (t) Q(.) (t) G(.) dt
_
T
t
j (t) _ (t) dt
Using the integrating by parts rule
_
b
o
_ rdt =
_
b
o
r _ dt + [r]
b
o
. derived from inte-
grating (n)
0
= n
0
+n
0
. integrating the last expression gives
_
T
t
j (t) _ (t) dt =
_
T
t
_ j (t) (t) dt + [j (t) (t)]
T
t
and the Lagrangian reads
/ =
_
T
t
1 (.) +j (t) Q(.) + _ j (t) (t) (t) G(.) dt
[j (t) (t)]
T
t
.
96
This is now maximized with respect to (t) and . (t), both the control and the
state variable. We then obtain conditions that are necessary for an optimum.
1
:
(.) +j (t) Q
:
(.) (t) G
:
(.) = 0 (5.55)
1
j
(.) +j (t) Q
j
(.) + _ j (t) (t) G
j
(.) = 0 (5.56)
The last rst-order condition can be rearranged to
_ j (t) = j (t) Q
j
(.) 1
j
(.) + (t) G
j
(.) (5.57)
These necessary conditions will be the one used regularly in maximization problems.
The shortcut
As it is cumbersome to start from a Lagrangian for each dynamic maximization
problem, one can dene the Hamiltonian as a shortcut as
H
1
= 1 (.) +j (t) Q(.) (t) G(.) . (5.58)
Optimality conditions are then
H
1
:
= 0. (5.59)
_ j (t) = H
1
j
. (5.60)
which are the same as (5.55) and (5.57) above and (5.52) and (5.53) in the last section.
5.6.3 The link between CV and PV
If we solve the problem (5.6) and (5.7) via the present value Hamiltonian, we would
start from
H
1
(t) = c
j[tt[
G(.) +j (t) Q(.) (5.61)
rst-order conditions would be
j (1) = 0. (5.62)
JH
1
J.
= c
j[tt[
G
:
(.) +jQ
:
(.) = 0. (5.63)
_ j (t) =
JH
1
J
= c
j[tt[
G
j
(.) jQ
j
(.) (5.64)
and we would be done with solving this problem.
Simplication
We can simplify the presentation of rst-order conditions, however, by rewriting
(5.63) as
G
:
(t) +`(t) Q
:
(t) = 0 (5.65)
where we dened
`(t) = c
j[tt[
j (t) . (5.66)
97
Note that the argument of the costate variables is always time t (and not time t).
When we use this denition in (5.64), this rst-order condition reads
c
j[tt[
_ j (t) = G
j
(.) `Q
j
(.) .
Replacing the left-hand side by
c
j[tt[
_ j (t) =
_
`(t) jc
j[tt[
j (t) =
_
`(t) j`(t) .
which follows from computing the time t derivative of the denition (5.66), we get
_
` = j` G
j
`(t)Q
j
. (5.67)
Inserting the denition (5.66) into the Hamiltonian (5.61) gives
c
j[tt[
H
1
(t) = G(.) +`Q(.) <= H
c
(t) = G(.) +`Q(.)
which denes the link between the present value and the current value Hamiltonian
as
H
c
(t) = c
j[tt[
H
1
(t)
Summary
Hence, instead of rst-order condition (5.63) and (5.64), we get (5.65) and (5.67).
As just shown these rst-order conditions are equivalent.
There are many texts that treat Hamiltonians as a maximization device. Some exam-
ples include Dixit (1990, p. 148), Intriligator (1971), Kamien and Schwartz (1991),
Leonard and Long (1992) or, in German, Feichtinger und Hartl (1986).
98
Exercises chapter 5
Hamiltonians
1. Optimal consumption over an innite horizon
Solve the maximization problem
max
c(t)
_
1
t
c
j[tt[
ln c (t) dt.
subject to
j (t) c (t) +
_
(t) = : (t) (t) +n(t) .
by
(a) using the present-value Hamiltonian. Compare the result to (5.31).
(b) Use n(c (t)) instead of ln c (t).
2. Adjustment costs
Solve the adjustment cost example for
(1) = 1.
What do optimality conditions mean? What is the optimal end-point value for
1 and 1?
3. Consumption over the life cycle
Utility of an individual, born at : and living for 1 periods is given at time t by
n(:. t) =
_
cT
t
c
j[tt[
ln (c (:. t)) dt.
The individuals budget constraint is given by
_
cT
t
1
1
(t) c (:. t) dt = /(:. t) +c (:. t)
where
1
1
(t) = exp
_
_
t
t
: (n) dn
_
. /(:. t) =
_
cT
t
1
v
(t) n(:. t) dt.
This deplorable individual would like to know how it can lead a happy life but,
unfortunately, has not studied optimal control theory!
99
(a) What would you recommend him? Use a Hamiltonian approach and dis-
tinguish between changes of consumption and the initial level. Which in-
formation do you need to determine the initial consumption level? What
information would you expect this individual to provide you with? In other
words, which of the above maximization problems makes sense? Why not
the other one?
(b) Assume all prices are constant. Draw the path of consumption in a (t,c(t))
diagram. Draw the path of asset holdings a(t) in the same diagram, by
guessing how you would expect it to look like. (You could compute it if
you want)
4. Optimal consumption
(a) Derive the optimal allocation of expenditure and consumption over time
for
n(c (t)) =
c (t)
1o
1
1 o
. o 0
by employing the Hamiltonian.
(b) Show that this function includes the logarithmic utility function for o = 1
(apply LHospitals rule).
(c) Does the utility function n(c(t)) make sense for o 1? Why (not)?
5. A central planner
(Rebelo 1990 AK-Model)
You are responsible for the future well-being of 360 Million Europeans and
centrally plan the EU by assigning a consumption path to each inhabitant.
Your problem consists in maximizing a social welfare function of the form
l
i
(t) =
_
1
t
c
j[tt[
_
C
1o
1
_
(1 o)
1
dt
subject to the EU resource constraint
_
1 = 11 C (5.68)
(a) What are optimality conditions? What is the consumption growth rate?
(b) Under which conditions is the problem well dened (boundedness condi-
tion)? Insert the consumption growth rate and show under which condi-
tions the utility function is bounded.
(c) What is the growth rate of the capital stock? Compute the initial con-
sumption level, by using the No-Ponzi-Game condition (5.20).
(d) Under which conditions could you resign from your job without making
anyone less happy than before?
100
6. Optimal consumption levels
(a) Derive a rule for the optimal consumption level for a time-varying interest
rate : (t) . Show that (5.35) can be generalized to
c (t) =
1
_
1
t
c
R
:
I
p(1)r(s)
oc
dt
\ (t) +c (t) .
where \ (t) is human wealth.
(b) What does this imply for the wealth level c (t)?
7. Investment and the interest rate
(a) Use the result of 5 c) and check under which conditions investment is a
decreasing function of the interest rate.
(b) Perform the same analysis for a budget constraint _ c = :c +n c instead
of (5.68).
8. The Ramsey growth model (Cf. Blanchard and Fischer, ch. 2)
Let the resource constraint now be given by
_
1 = 1 (1. 1) C o 1.
Draw a phase diagram. What is the long-run equilibrium? Perform a stability
analysis graphically and analytically (locally). Is the utility function bounded?
Betrachten Sie eine dezentrale (kein Planerproblem) konomie in kontinuier-
licher (nicht diskreter) Zeit. Die konomie ist mit Kapital und Arbeit ausges-
tattet. Arbeit ist x gegeben durch 1, der anfngliche Kapitalbestand ist 1
0
.
Es besteht die Mglichkeit zur Kapitalakkumulation
_
1 (t) = 1 (t) o1 (t).
Kapital ist der einzige Vermgensgegenstand, d.h. Haushalte knnen nur durch
Kapitalerwerb sparen. Haushalte haben eine dazu passende Budgetrestriktion
und eine bliche intertemporale Nutzenfunktion mit Zeitprferenzrate j und un-
endlichem Zeithorizont. Firmen produzieren unter vollstndigem Wettbewerb.
Beschreiben Sie eine solche konomie formal und leiten Sie die reduzierte Form
her. Folgen Sie dabei der folgenden Struktur:
(a) Whlen Sie eine bliche Technologie.
(b) Bestimmen Sie die optimalen Faktornachfragen.
(c) Die Budgetrestriktion der Haushalte sei gegeben durch
_ c (t) = : (t) c (t) +n
1
(t) c (t) .
Formulieren Sie das Maximierungsproblem der Haushalte und lsen Sie es.
101
(d) Aggregieren Sie ber die Haushalte und beschreiben Sie dadurch die opti-
male Konsumentwicklung aggregierten Konsums.
(e) Formulieren Sie das Gleichgewicht auf dem Gtermarkt.
(f) Zeigen Sie die Konsistenz der Budgetrestriktion der Haushalte mit der
Kapitalakkumulationsgleichung der konomie.
(g) Leiten Sie die reduzierte Form
_
1 (t) = 1 (t) C (t) o1 (t)
_
C (t)
C (t)
=
0Y (t)
01(t)
o j
o
durch Befolgen dieser Schritte her und erlutern Sie diese verbal.
102
6 Dynamic programming in continuous time
This chapter reanalyzes maximization problems in continuous time that are known
from the chapter on Hamiltonians. It shows how to solve them with dynamic pro-
gramming methods. The sole objective of this chapter is to present the dynamic
programming method in a well-known deterministic setup such that its use in a sto-
chastic world in subsequent chapters becomes more accessible.
6.1 Household utility maxmization
We consider a maximization problem that is very similar to the introductory example
for the Hamiltonian in section 5.1 or the innite horizon case in section 5.4. Compared
to 5.1, the utility function here is more general and the planning horizon is innity.
None of this is important, however, for understanding the dierences in the approach
between the Hamiltonian and dynamic programming.
Utility of the individual is given by
l (t) =
_
1
t
c
j[tt[
n(c (t)) dt. (6.1)
Her budget constrained equates wealth accumulation with savings,
_ c = :c +n jc. (6.2)
As in models of discrete time, the value \ (c (t)) of the optimal program is dened
by \ (c (t)) = max
fc(t)g
l (t) subject to (6.2). When households behave optimally
between today and innity by choosing the optimal consumption path c (t) . their
overall utility l (t) is given by \ (c (t)) .
A prelude on the Bellman equation
The derivation of the Bellman equation is not as obvious as under discrete time
as under continuous time, even though we do have a today t, we do not have a
clear tommorrow (like a t + 1 in discrete time). We therefore need to construct a
tommorrow by adding a small time intervall t to t. Tommorrow would then be
t +t. Note that this derivation is heuristic and more rigorous approaches exist. See
e.g. Sennewald (2006) for further references to the literature.
Following Bellmans idea, we rewrite the objective function as the sum of two
subperiods,
l (t) =
_
t.t
t
c
j[tt[
n(c (t)) dt +
_
1
t.t
c
j[tt[
n(c (t)) dt.
where t is a small time intervall. When we approximate the rst integral (think
of the area below the function n(c (t)) plotted over time t) by n(c (t)) t and the
103
discounting between t and t +t by
1
1j.t
and we assume that as of t +t we behave
optimally, we can rewrite the value function \ (c (t)) = max
fc(t)g
l (t) as
\ (c (t)) = max
c(t)
_
n(c (t)) t +
1
1 +jt
\ (c (t + t))
_
.
By doing so, we are left with only one choice variable c (t) instead of the entire path
c (t) .When we rst multiply this expression by 1 + jt, then divide by t and
nally move
\ (o(t))
.t
to the right hand side, we get j\ (c (t)) =
max
c(t)
_
n(c (t)) [1 +jt] +
\ (o(t.t))\ (o(t))
.t
_
. Taking the limit lim
.t!0
gives the
Bellman equation,
j\ (c (t)) = max
c(t)
_
n(c (t)) +
d\ (c (t))
dt
_
. (6.3)
This equation again shows Bellmans trick: A maximization problem, consisting
of the choice of a path of a choice variables, was broken down to a maximization
problem where only the level of the choice variable in t has to be chosen. The
structure of this equation can also be understood from a more intuitive perspective:
The term j\ (c (t)) can best be understood when comparing it to :1. capital income
at each instant of an individual who owns a capital stock 1 and the interest rate
is :. A household that behaves optimally owns the value \ (c (t)) from optimal
behaviour and receives a utility stream of j\ (c (t)) . This utility income at each
instant is given by instantaneous utility from consumption plus the change in the
value of optimal behaviour. Note that this structure is identical to the capital-market
no-arbitrage condition (4.19), : (t) (t) = : (t) + _ (t) - the capital income stream
from holding wealth (t) on a bank account is identical to dividend payments : (t)
plus the change _ (t) in the market price when holding the same level of wealth in
stocks.
While the derivation just shown is the standard one, we will now present an
alternative approach which illustrates the economic content of the Bellman equation
and which is more straightforward. Given the objective function in (6.1), we can ask
how overall utility l (t) changes over time. To this end, we compute the derivative
dl (t) ,dt and nd (using the Leibniz rule 4.9 from ch. 4.3.1)
_
l (t) = c
j[tt[
n(c (t)) +
_
1
t
d
dt
c
j[tt[
n(c (t)) dt = n(c (t)) +jl (t) .
Overall utility l (t) reduces as time goes by by the amount n(c (t)) at each instant
(as the integral becomes smaller when current consumption in t is lost and we
start an instant after t) and increases by jl (t) (as we gain because future utilities
come closer to today when today moves into the future). Rearranging this equation
gives jl (t) = n(c (t)) +
_
l (t) . When overall utility is replaced by the value function,
we obtain j\ (c (t)) = n(c (t)) +
_
\ (c (t)) which corresponds in its structure to the
Bellman equation (6.3).
104
We will now follow the three-step procedure to maximization when using the
dynamic programming approach as we got to know it in discrete time setups is section
3.4. When we compute the Bellman equation for our case, we obtain for the derivative
in (6.3) d\ (c (t)) ,dt = \
0
(c (t)) _ c which gives with the budget constraint (6.2)
j\ (c (t)) = max
c(t)
n(c (t)) +\
0
(c (t)) [:c +n jc] . (6.4)
The rst-order condition reads
n
0
(c (t)) = j (t) \
0
(c (t)) (6.5)
and makes consumption a function of the state variable, c (t) = c (c (t)) .
In continuous time, the second step of the dynamic programming approach to
maximization can be subdivided into two steps. (i) In the rst, we look at the
maximized Bellman equation,
j\ (c) = n(c (c)) +\
0
(c) [:c +n jc (c)] .
The rst-order condition (6.5) together with the maximized Bellman equation de-
termine the evolution of the control variable c (t) and \ (c (t)). Again, however, the
maximized Bellman equation does not provide very much insight. Computing the
derivative with respect to c (t) however (and using the envelope theorem) gives an
expression for the shadow price of wealth that will be more useful,
j\
0
(c) = \
00
(c) [:c +n jc] +\
0
(c) : = (6.6)
(j :) \
0
(c) = \
00
(c) [:c +n jc] .
(ii) In the second step, we compute the derivative of \
0
(c) with respect to time,
giving
d\
0
(c)
dt
= \
00
(c) _ c = (j :) \
0
(c) .
where the last equality used (6.6). Dividing by \
0
(c) and using the usual notation
_
\
0
(c) = d\
0
(c) ,dt, this can be written as
_
\
0
(c)
\ (c)
= j :. (6.7)
This equation describes the evolution of the costate variable \
0
(c), the shadow price
of wealth.
105
The derivative of the rst-order condition with respect to time is given by (apply
rst logs)
n
00
(c)
n
0
(c)
_ c =
_ j
j
+
_
\
0
(c)
\
0
(c)
.
Inserting (6.7) gives
n
00
(c)
n
0
(c)
_ c =
_ j
j
+j : ==
n
00
(c)
n
0
(c)
_ c = :
_ j
j
j.
This is the well-known Keynes Ramsey rule.
6.2 Comparing dynamic programming to Hamiltonians
When we compare optimality conditions under dynamic programming with those
obtained when employing the Hamiltonian, we nd that they are identical. Observing
that our costate evolves according to
_
\
0
(c)
\
0
(c)
= j :.
as just shown in (6.7), we obtain the same equation as we had in the Hamiltonian
approach for the evolution of the costate variable, see e.g. (5.4) or (5.38). Comparing
rst-order conditions (5.3) or (5.14) with (6.5), we see that they would be identical if
we had chosen exactly the same maximization problem. This is not surprising, given
our applied view of optimization: If there is one optimal path that maximizes some
objective function, this one path should result, independently of which maximization
procedure is chosen.
A comparison of optimality conditions is also useful for an alternative purpose,
however. As (6.7) and e.g. (5.4) or (5.38) are identical, we can conclude that the
derivative \
0
(c (t)) of the value function with respect to the state variable, in our case
c, is identical to the costate variable ` in the current-value Hamiltonian approach,
\
0
(c) = `. This is where the interpretation for the costate variable in the Hamiltonian
approach in ch. 5.2.2 came from. There, we said that the costate variable ` stands
for the increase in the value of the optimal program when an additional unit of the
state variable becomes available; this is exactly what \
0
(c) stands for. Hence, the
interpretation of a costate variable ` is similar to the interpretation of the Lagrange
multiplier in static maximization problems.
6.3 Dynamic programming with two state variables
As a nal example for maximization problems in continuous time that can be solved
with dynamic programming, we look at a maximization problem with two state vari-
ables. Think e.g. of an agent who can save by putting savings on a bank account or
by accumulating human capital. Or think of a central planner who can increase total
factor productivity or the capital stock. We look here at the rst case.
106
Our agent has a standard objective function
max l (t) =
_
1
t
c
j[tt[
n(c (t)) dt.
It is maximized subject to two constraints. They describe the evolution of the state
variables wealth c and human capital /.
_ c = , (c. /. c) .
_
/ = q (c. /. c) . (6.8)
We do not give explicit expressions for the functions , (.) or q (.) but one can think of
a standard resource constraint for , (.) as in (6.2) and a functional form for q (.) that
captures a trade-o between consumption and human capital accumulation: Human
capital accumulation is faster when c and / are large but decreases in c. To be precise,
we assume that both , (.) and q (.) increase in c and / but decrease in c.
In this case, the Bellman equation reads
j\ (c. /) = max
_
n(c) +
d\ (c. /)
dt
_
= max n(c) +\
o
, (.) +\
I
q (.) .
There are simply two partial derivatives of the value function after the n(c) term
times the dc and d/ term, respectively, instead of one as in (6.4), where there is only
one state variable.
Given that there is still only one control variable - consumption, there is only
one rst-order condition. This is clearly specic to this example. One could think
of a time constraint for human capital accumulation (a trade-o between leisure and
learning - think of the Lucas (1988) model) where agents choose the share of their
time used for accumulating human capital. In this case, there would be two rst-order
conditions. Here, however, we have just the one for consumption, given by
n
0
(c) +\
o
J, (.)
Jc
+\
I
Jq (.)
Jc
= 0 (6.9)
When we compare this condition with the one-state-variable case in (6.5), we see
that the rst two terms n
0
(c)+\
o
0)(.)
0c
correspond exactly to (6.5): If we had specied
, (.) as in the budget constraint (6.2), the rst two terms would be identical to (6.5).
The third term \
I
0j(.)
0c
is new and stems from the second state variable: Consumption
now not only aects the accumulation of wealth but also the accumulation of human
capital. More consumption gives higher instantaneous utility but, at the same time,
decreases future wealth and - now new - also the future human capital stock.
DP2: Evolution of the costate variables
107
As always, we need to understand the evolution of the costate variable(s). In
a setup with two state variables, there are two costate variables, or, economically
speaking, a shadow price of wealth c and a shadow price of human capital /. This
is obtained by partially dierentiating the maximized Bellman equation, rst with
respect to c. then with respect to /. Doing this, we get (employing the envelope
theorem right away)
j\
o
= \
oo
, (.) +\
o
J, (.)
Jc
+\
Io
q (.) +\
I
Jq (.)
Jc
. (6.10)
j\
I
= \
oI
, (.) +\
o
J, (.)
J/
+\
II
q (.) +\
I
Jq (.)
J/
. (6.11)
As in ch. 6.1, the second step of DP2 consists in computing the time derivatives
of the costate variables and in reinserting (6.10) and (6.11). We rst compute time
derivatives, inserting (6.8) in the last step,
d\
o
(c. /)
dt
= \
oo
_ c +\
oI
_
/ = \
oo
, (.) +\
oI
q (.) .
d\
I
(c. /)
dt
= \
Io
_ c +\
II
_
/ = \
Io
, (.) +\
II
q (.) .
Inserting then (6.10) and (6.11), we nd
d\
o
(c. /)
dt
= j\
o
\
o
J, (.)
Jc
\
I
Jq (.)
Jc
.
d\
I
(c. /)
dt
= j\
I
\
o
J, (.)
J/
\
I
Jq (.)
J/
.
The nice feature of this last step is the fact that the cross derivatives \
oI
and \
Io
dis-
appear, i.e. can be substituted out again, using the fact that ,
aj
(r. ) = ,
ja
(r. )
for any twice dierentiable function , (r. ). Writing these equations as
_
\
o
\
o
= j
J, (.)
Jc

\
I
\
o
Jq (.)
Jc
.
_
\
I
\
I
= j
\
o
\
I
J, (.)
J/

Jq (.)
J/
.
allows to give an interpretation that links them to the standard one-state case in
(6.7). The costate variable \
o
evolves as above only that instead of the interest rate,
we nd here
0)(.)
0o
+
\
I
\
a
0j(.)
0o
. The rst derivative J, (.) ,Jc captures the eect a change
in c has on the rst constraint; in fact, if , (.) represented a budget constraint as in
(6.2), this would be identical to the interest rate. The second term Jq (.) ,Jc captures
the eect of a change in c on the second constraint; by how much would / increase if
there was more c? This eect is multiplied by \
I
,\
o
. the relative shadow price of /.
An analagous interpretation is possible for
_
\
I
,\
I
.
108
The nal step consists in computing the derivative of the rst-order condition (6.9)
with respect to time and replacing the time derivatives
_
\
o
and
_
\
I
by the expressions
from the previous step DP2. The principle of how to obtain a solution therefore
remains unchanged when having two instead of one state variable. Unfortunately,
however, it is generally not possible to eliminate the shadow prices from the resulting
equation which describes the evolution of the control variable. Some economically
suitable assumptions concerning , (.) or q (.) could, however, help.
6.4 An example: Nominal and real interest rates and ina-
tion
We present here a simple model which allows to understand the dierence between
the nominal and real interest rate and the determinants of ination. This chapter is
intended (i) to provide another example where dynamic programming can be used and
(ii) to show again how to go from a general decentralized description of an economy
to its reduced form and thereby obtain economic insights.
6.4.1 Firms, the central bank and the government
Firms
Firms use a standard neoclassical technology 1 = 1 (1. 1) . Producing under
perfect competition implies factor demand function of
n
1
= j
J1
J1
. n
1
= j
J1
J1
. (6.12)
The central bank and the government
Studying the behaviour of central banks allows to ll many books. We present the
behaviour of the central bank in a very simple way - so simple that what the central
bank does here (buy bonds directly from the government) is actually illegal in most
OECD countries. Despite this simple presentation, the general result we obtain later
would hold in more realistic setups as well.
The central bank issues money ` (t) in exchange for government bonds 1(t) . It
receives interest payments i1 from the government on the bonds. The balance of the
central bank is therefore i1 +
_
` =
_
1. This equation says that bond holdings by the
central bank increase by
_
1 either when the central bank issues money
_
` or receive
interest payments i1 on bonds it holds.
The governments budget constraint reads G+i1 = 1 +
_
1. General government
expenditure G plus interest payments i1 on government debt 1 is nanced by tax
income 1 and decit
_
1. We assume that only the central bank holds government
bonds and not private households. Combining the government with the central bank
budget therefore yields
_
` = G1.
109
This equation says that an increase in monetary supply either is used to nance
government expenditure minus tax income or, if G = 0 for simplicity, any monetary
increase is given to housholds in the form of negative taxes, 1 =
_
`.
6.4.2 Households
Preferences of households are described by a money-in-utility function,
_
1
t
c
j[tt[
_
ln c (t) + ln
:(t)
j (t)
_
dt.
In addition to utility from consumption c (t), utility is derived from holding a certain
stock :(t) of money, given a price level j (t) . Given that wealth of households consists
of capital goods plus money, c (t) = / (t) + :(t) . and that holding money pays no
interest, the budget constraint of the household can be shown to read (see exercises)
_ c = i [c :] +n 1,1 jc.
Tax payments 1,1 per representative household are lump-sum. If taxes are negative,
1,1 represents transfers of the government to households. The interest rate i is
dened according to
i =
n
1
+ _
.
When households choose consumption and the amount of money held optimally,
consumption growth follows (see exercises)
_ c
c
= i
_ j
j
j.
Money demand is given by (see exercises)
: =
jc
i
.
6.4.3 Equilibrium
The reduced form
Equilibrium requires equality of supply and demand on the goods market. This is
obtained if total supply 1 equals demand C +1. Letting capital accumulation follow
_
1 = 1 o1. we get
_
1 = 1 (1. 1) C o1. (6.13)
This equation determines 1. As capital and consumption goods are traded on the
same market, this equation implies = j and the nominal interest rate becomes with
(6.12)
i =
n
1
j
+
_ j
j
=
J1
J1
+
_ j
j
. (6.14)
110
The nominal interest rate is given by marginal productivity of capital n
1
,j = J1,J1
(the real interest rate) plus ination _ j,j.
Aggregating over households yields aggregate consumption growth of
_
C,C = i
_ j,j j. Inserting (6.14) yields
_
C
C
=
J1
J1
j. (6.15)
Aggregate money demand is given by
` =
jC
i
. (6.16)
Given an exogenous money supply rule and appropriate boundary conditions, these
four equations determine the paths of 1. C. i and the price level j.
One standard property of models with exibel prices is the dichotomy between
real variables and nominal variables. The evolution of consumption and capital - the
real side of the economy - is completely independent of monetary inuences: Equation
(6.13) and (6.15) determine the paths of 1 and C just as in the standard optimal
growth model without money - see (5.37) and (5.39) in ch. 5.5.4. Hence, when
thinking about equilibrium in this economy, we can think about the real side on the
one hand - independently of monetary issues - and about the nominal side on the
other hand. Monetary variables have no real eect but real variables have an eect
on monetary variables like e.g. ination.
Needless to say that the real world does not have perfectly exible prices such
that one should expect monetary variables to have an impact on the real economy.
This model is therefore a starting point to well understand structures and not a fully
developed model to analyse monetary questions in a very realistic way. Price rigidity
would have to be included before doing this.
A stationary equilibrium
Assume the technology 1 (1. 1) is such that in the long-run 1 is constant. As
a consequence, aggregate consumption C is contant as well. Hence, with respect to
real variables (including, in addition to 1 and C. the real interest rate and output),
we are in a steady state as in ch. 5.5.4.
Depending on exogenous money supply, equations (6.14) and (6.16) determine the
price level and the nominal interest rate. Substituting the nominal interest rate out,
we obtain
_ j
j
=
jC
`

J1
J1
.
This is a dierential equation which is plotted in the next gure. As this gure shows,
provided that ` is constant, there is a price level j
which implies that there is zero

ination.
111
N
N
j j
jC
A

0Y
01
0
` j
j
0Y
01
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

Figure 29 The price level j
in a monetary economy
Now assume there is money growth,
_
` 0. The gure then shows that the
price level j
increases as the slope of the line becomes atter. Money growth in an

economy with constant GDP implies ination. By looking at (6.14) and (6.16) again
and by focusing on equilibria with constant ination rates, we know from (6.14)
that a constant ination rate implies a constant nominal interest rate. Hence, by
dierentiating the equilibrium (6.16) on the money market, we get
_ j
j
=
_
`
`
. (6.17)
In an economy with constant GDP and increasing money supply, the ination rate is
identical to the growth rate of money supply.
A growth equilibrium
Now assume there is (exogenous) technological progress at a rate q such that in
the long run
_
1 ,1 =
_
C,C =
_
1,1 = q. Then by again assuming a constant ination
rate (implying a constant nominal interest rate) and going through the same steps
that led to (6.17), we nd by dierentiating (6.16)
_ j
j
=
_
`
`

_
C
C
.
Ination is given by the dierence between the growth rate of money supply and
consumption growth.
Exogenous nominal interest rates
112
The current thinking about central bank behaviour diers from the view that the
central bank chooses money supply ` as assumed so far. The central bank rather
sets nominal interest rates and money supply adjusts (where one should keep in mind
that money supply is more than just cash used for exchange as modeled here). Can
nominal interest rate setting be analysed in this setup?
Equilibrium is described by equations (6.13) to (6.16). They were understood to
determine the paths of 1. C. i and the price level j. given a money supply choice `
of the central bank. If one believes that nominal interest rate setting is more realistic,
these four equations would simply determine the paths of 1. C. ` and the price level
j. given a nominal interest rate choice i of the central bank. Hence, by simply making
an exogenous variable, `. endogenous and making a previously endogenous variable,
i. exogenous, the same model can be used to understand the eects of higher and
lower nominal interest rates on the economy.
Due to perfect price exibility, real quantities remain unaected by the nominal
interest rate. Consumption, investment, GDP, the real interest rate, real wages are all
determined, as before, by (6.13) and (6.15) - the dichotomy between real and nominal
quantities continues given price exibility. By (6.14), a change in the nominal interest
rate aects ination: high (nominal) interest rates imply high ination, low nominal
interest rates imply low ination. From the money market equilibrium in (6.16) one
can then conclude what this implies for money supply, again both for a growing or
a stationary economy. Much more needs to be said about these issues before policy
implications can be discussed. Any analysis in a general equilibrium framework would
however partly be driven by relationships presented here.
An alternative way, which is not based on dynamic programming, to reach the same
conclusion about the interpretation of the costate variable for Hamiltonian maximiza-
tion as here in ch. 6.2 is provided by Intriligator (1971, p. 352). An excellent overview
and introduction to monetary economics is provided by Walsh (2003).
113
Exercises chapter 6
Dynamic Programming in Continuous
Time
1. The envelope theorem (nicht schon wieder ...)
Compute the derivative (6.6) of the maximized Bellman equation without using
the envelope theorem.
2. A rm with adjustment costs
Consider again, as in ch. 5.5.1, a rm with adjustment cost. The rms objective
is
max
f1(t),1(t)g
_
1
t
c
v[tt[
: (t) dt.
In contrast to ch. 5.5.1, the rm now has an innite planning horizon and
employs two factors of production, capital and labour. Instantaneous prots
are
: = j1 (1. 1) n1 1 c1
o
.
where investment 1 also comprises adjustment costs for c 0. Capital, owned
by the rm, accumulates according to
_
1 = :1 o1. All parameters :. o. c. ,
are constant.
(a) Solve this maximization problem by using the dynamic programming ap-
proach. You may choose appropriate (numerical or other) values for para-
meters where this simplies the solution (and does not destroy the spirit
of this exercise).
(b) Show that in the long-run with adjustment cost and at each point in time
under the absence of adjustment cost, capital is paid its value marginal
product. Why is labour being paid its value marginal product at each
point in time?
3. Money in the utility function
Consider an individual with the following utility function
l (t) =
_
1
t
c
j[tt[
_
ln c(t) + ln
:(t)
j(t)
_
dt.
As always, j is the time preference rate and c (t) is consumption. This utility
function also captures demand for money by including a real monetary stock of
114
:(t) ,j (t) in the utility function where :(t) is the amount of cash and j (t)
is the price level of the economy. Let the budget constraint of the individual be
_ c = i [c :] +n 1,1 jc.
where c is total wealth consisting of shares in rms plus money, c = / +: and
i is the nominal interest rate.
(a) Derive the budget constraint by assuming interest payments of i on shares
in rms and zero interest rates on money.
(b) Derive optimal money demand.
4. Nominal and real interest rates in general equilibrium
Put households of exercise 3 in general equilibrium with capital accumulation
and a central bank whoe chooses money supply `. Compute the real and the
nominal interest rate in a long-run equilibrium.
115
6.6 Looking back
This is the end of part I and II. This is often also the end of a course. This is a good
moment to look back at what has been acomplished. After 14 or 15 lectures and the
same number of exercise classes, the amount of material covered is fairly impressive.
In terms of maximization tools, this rst part has covered
Solving by inserting
Lagrange methods in discrete and continuous time
Dynamic programming in discrete time and continuous time
Hamiltonian
With respect to model building components, we got to know
how to build budget constraints
how to structure the presentation of a model
how to derive reduced forms
From an economic perspective, the rst part presented
two-period OLG model
the optimal saving central planner model in discrete and continuous time
the matching approach to unemployment
the decentralized optimal growth model and
an optimal growth model with money
Most importantly, however, the tools having been presented here allow students to
become independent. A very large part of the economics literature (acknowledging
that game theoretic approaches have not at all been covered here) is now open and
accessible and the basis for understanding a paper in detail (and not just the overall
argument) and for presenting own arguments in a scientic language are laid.
Clearly, models with uncertainty present additional challenges. They will be pre-
sented and overcome in part III and part IV.
116
Part III
Stochastic models in discrete time
The following chapter starts part III of this book. We got to know many optimization
methods for deterministic setups, both in discrete and continuous time. All economic
questions that were analysed were viewed as suciently deterministic. If there was
any uncertainty in the setup of the problem, we simply ignored it or argued that it
is of no importance for understanding the basic properties and relationships of the
economic question. This is a good approach for many economic questions.
Real life has few certain components - death is certain, but when? Taxes are
certain, but how high are they? We know that we all exist - but dont ask philosophers.
Part III and part IV will take uncertainty in life seriously and incorporate it explicitly
in the analysis of economic problems. We follow the same distinction as in part I and
II; we rst analyse the eects of uncertainty on economic behaviour in discrete time
setups and then move to continuous time setups in part IV.
7 Stochastic dierence equations and applications
Before we look at dierence equations in section 7.4, we rst spend some sections on
reviewing basic concepts related to uncertain environments. These concepts will be
useful at later stages.
7.1 Basics on random variables
Let us rst have a look at some basics on random variables. This follows Evans,
Hastings and Peacock (2000).
7.1.1 Some concepts
A probabilistic experiment is some occurrence where a complex natural background
leads to a chance outcome. The set of possible outcomes of a probabilistic experiment
is called the possibility space (Mglichkeitsraum, Raum mglicher Ausprgungen). A
random variable (RV) A is a function which maps from the possibility space into a set
of numbers. The set of numbers this RV can take is called the range of this variable
A.
The distribution function 1 (Verteilungsfunktion) associated with the RV A is a
function which maps from the range into the probability domain [0,1],
1 (r) = Prob(A _ r) .
The probability that A has a realization of r or smaller is given by 1 (r) .
We now need to make a distinction between discrete and continuous RVs. When
the RV A has a discrete range then , (r) gives nite probabilities and is called the
117
probability function (Wahrscheinlichkeitsfunktion) or probability mass function. The
probability that A has the realization of r is given by , (r) .
When the RV A is continuous, the rst derivative of the distribution function 1
,(r) =
d1 (r)
dr
is called the probability density function , (Dichtefunktion). The probability that the
realization of A lies between, say, c and / c is given by 1 (/) 1 (c) =
_
b
o
, (r) dr.
Hence the probability that A equals c is zero.
7.1.2 An illustration
Discrete random variable
Consider the probabilistic experiment tossing a coin two times. The possibility
space is given by HH. H1. 1H. 11. Dene the RV Number of hats. The range
of this variable is given by 0. 1. 2 . Assuming that the coin falls on either side with
the same probability, the probability function of this RV is given by
, (r) =
_
_
_
.25
.5
.25
_
_
_
for r =
_
_
_
0
1
2
.
Continuous random variable
Think of next weekend. You might consider going into some pub to meet friends.
Before you go there, you do not know how much time you will spend there. If you
meet many friends, you stay longer, if you just drink one beer, you leave soon. Hence,
going to a pub on a weekend is a probabilistic experiment with a chance outcome.
The set of possible outcomes with respect to the amount of time spent in a pub
is the possibility space. Our random variable 1 maps from this possibility space
into a set of numbers with a range from 0 to, lets say, 4 hours (as the pub closes
at 1 am and you never go there before 9 pm). As time is continous, 1 [0. 4] is a
continuous random variable. The distribution function 1 (t) gives you the probability
that you spend a period of length t or shorter in the pub. The probability that you
spend between 1.5 and two hours in the pub is given by
_
2
1.
, (t) dt where , (t) is the
density function , (t) = d1 (t) ,dt.
7.2 Examples for random variables
We now look at some examples of RVs that are useful for later applications. As a RV
is completely characterized by its range and its probability or density function, we
will describe RVs by providing this information.
118
7.2.1 Discrete random variables
Discrete uniform distribution
range r c. c + 1. .... / 1. /
probability function , (r) = 1, (/ c + 1)
An example for this RV is the die. Its range is 1 to 6, the probability for any
number (at least for fair dice) is 1,6.
The Poisson distribution
range r 0. 1. 2. ...
probability function , (r) =
c
A
A
i
a!
Here ` is some positive parameter. When we talk about stochastic processes in
part IV, this will be calle the arrival rate.
7.2.2 Continuous random variables
Normal distribution
range r ]. +[
density function , (r) =
1
p
2o
2
c
1
2
(
i
)
2
The mean and the standard deviation of A are given by j and o.
Standard normal distribution
This is the normal distribution with mean and standard deviation given by j = 0
and o = 1.
Exponential distribution
range r [0. [
density function , (r) = `c
Aa
Again, ` is some positive parameter.
7.2.3 Higher-dimensional random variables
We have studied so far one-dimensional RVs. In what follows, we will occasionally
work with multidimensional RVs as well. It will suce for our illustration purposes
here to focus on a two-dimensional normal distribution. Consider two random vari-
ables A
1
and A
2
. They are (jointly) normally distributed if the density function of
A
1
and A
2
is given by
, (r
1
. r
2
) =
1
2:o
1
o
2
_
1 j
2
c
1
2(1p
2
)
(
` a
2
1
2j` a
1
` a
2
` a
2
2
)
. (7.1)
where ~ r
i
=
r
i
j
i
o
i
. j =
o
12
o
1
o
2
. (7.2)
119
The mean and standard deviation of the RVs are denoted by j
i
and o
i
. The parameter
j is called the correlation coecient between A
1
and A
2
is dened as the covariance
o
12
devided by the standard deviations (see ch. 7.3.1).
The nice aspects of this two-dimensional normally distributed RV (the same holds
for :-dimensional RVs) is that the density function of each individual RV A
i
. i.e.
the marginal density (Randverteilung), is given by the standard expression which is
independent of the correlation coecient,
, (r
i
) =
1
_
2:o
2
i
c
1
2
` a
2
.
. (7.3)
This implies a very convenient way to go from independent to correlated RVs in
a multi-dimensional setting: When we want to assume independent normally dis-
tributed RVs, we assume that (7.3) holds for each random variable A
i
and set the
correlation coecient to zero. When we want to work with dependent RVs that are
individually normally distributed, (7.3) holds for each RV individually as will but, in
addition, we x a non-zero coecient of correlation j.
7.3 Expected values, variances, covariances and all that
7.3.1 Denitions
We repeat here some denitions which can be found in many textbooks on statistics.
We focus on continuous random variables. For discrete random variables, the integral
is replaced by a sum - in very loose notation,
_
b
o
q (r) dr is replaced by
b
i=o
q (r
i
),
where c and / are constants which can be minus or plus innity and q (r) is some
function. In the following denitions,
_
.dr means the integral over the relevant range
(i.e. from minus to plus innity or from the lower to upper bound of the range of the
RV under consideration).
Mean 1A =
_
r, (r) dr
Variance varA =
_
(r 1A)
2
, (r) dr
/th uncentered moment 1A
I
=
_
r
I
, (r) dr
/th centered moment 1 (A 1A)
I
=
_
(r 1A)
I
, (r) dr
covariance cov(A. 1 ) =
_ _
(r 1A) ( 11 ) , (r. ) dr d
correlation coecient j
AY
= cov(A. 1 ) ,
_
varA var1
independence j (A . 1 1) = 1 (A ) 1 (1 1)
Figure 30 Some basic denitions
7.3.2 Lemmas
Basic
120
On expectations
1 [c +/A] = c +/1A
. = q (r. ) =12 =
_ _
q (r. ) , (r. ) dr d
On variances
varA = 1
_
(A 1A)
2
= 1
_
A
2
2A1A + (1A)
2
= 1A
2
2 (1A)
2
+ (1A)
2
= 1A
2
(1A)
2
(7.4)
var (c +/A) = /
2
varA
var (A +1 ) = varA + var1 + 2cov(A. 1 ) (7.5)
On covariances
cov(A. A) = varA
cov(A. 1 ) = 1 (A1 ) 1A11 ==1 (A1 ) = 1A11 + cov(A. 1 )
cov(c +/A. c +d1 ) = /d cov(A. 1 )
Advanced
We present here a theorem which is very intuitive and highly useful to analytically
study pro- and countercyclical behaviour of endogenous variables in models of business
cycles. The theorem says that if two variables depend in the same sense on some
RV (i.e. they both increase or decrease in the RV), then these two variables have a
positive covariance. If e.g. both GDP and R&D expenditure increase in TFP and
TFP is random, then GDP and R&D expenditure are procyclical.
Theorem 6 Let A a random variable and , (A) and q (A) two functions such that
,
0
(A) q
0
(A) R 0 \ r A. Then
cov (, (A) . q (A)) R 0.
Proof. We only proof the 0 part. We know that
cov(1. 2) = 1 (1 2) 11 12.
With 1 = , (A) and 2 = q (A), we have
cov(, (A) . q (A)) = 1 (, (A) . q (A)) 1, (A) 1q (A)
=
_
, (r) q (r) j (r) dr
_
, (r) j (r) dr
_
q (r) j (r) dr.
where j (r) is the density of A. Hence
cov(, (A) . q (A)) 0 =
_
, (r) q (r) j (A) dr
_
, (r) j (r) dr
_
q (r) j (r) dr.
The last inequality holds for ,
0
(A) q
0
(A) 0 as shown by

Cebyev and presented in
Mitrinovic (1970, Theorem 10, sect. 2.5, p. 40).
121
7.3.3 Functions on random variables
We will occasionally encounter the situation where we compute density functions of
functions of RVs. Here are some examples.
Linearily transforming a normally distributed RV
Consider a normally distributed RV A~` (j. o
2
) . What is the distribution of the
1 = c + /A? We know from ch. 7.3.2 that for any RV, 1 (c +/A) = c + /1A and
\ c: (c +/A) = /
2
\ c: (A). As it can be shown that a linear transformation of a
normally distributed RV gives again a normally distributed RV, 1 is also normally
distributed with 1 ~` (c +/j. /
2
o
2
) .
An exponential transformation
Consider the RV 1 = c
A
. with A being normally distributed. The RV 1 is
lognormally distributed. A variable is lognormally distributed if its logarithm is
normally distributed, ln 1 ~` (j. o
2
). The mean and variance of this distribution are
given by
j
j
= c
j
1
2
o
2
. o
2
j
= c
2jo
2
_
c
o
2
1
_
. (7.6)
Clearly, 1 can only have non-negative realizations.
Transformations of log-normal distributions
Let there be two long-normally distributed variables 1 and 2. Any transformation
of the type 1
c
. where c is a constant, or products like 1 2 are also lognormally
distributed.
To show this, remember that we can express 1 and 2 as 1 = c
A
1
and 2 = c
A
2
,
with the A
i
being (jointly) normally distributed. Hence, for the rst example, we
can write 1
c
= c
cA
1
. As cA
1
is normally distributed, 1
c
is lognormally distributed.
For the second, we write 1 2 = c
A
1
c
A
2
= c
A
1
A
2
. As the A
i
are (jointly) normally
distributed, their sum is as well and 1 2 is lognormally distributed.
Total factor productivity is sometimes argued to be lognormally distributed. Its
logarithm is then normally distributed. See e.g. ch. 7.5.6.
The general case
Consider now a general transformation of the type = (r) where the RV A has
some density , (r) . What is the density of 1 ? The answer comes from the following
Theorem 7 Let A be a random variable with density , (r) and range [c. /] which can
be ] . +[. Let 1 be dened by the monotonically increasing function = (r) .
Then the density q () is given by q () = , (r ())
oa
oj
on the range [(c). (/)].
122
This theorem can be easily proven as follows. The proof is illustrated in g. 31.
The gure plots the RV A on the horizontal and the RV 1 on the vertical axis. A
monotonically increasing function (r) represents the transformation of realizations
r into .
N
N
A
1
1 (r)
c r /
1 (c)
1 ( r)
1 (/)
Figure 31 Transforming a random variable
Proof. The transformation of the range is immediately clear from the gure:
When A is bounded between c and /. 1 must be bounded between (c) and (/) .
The proof for the density of 1 requires some more steps: The probability that is
smaller than some (~ r) is identical to the probability that A is smaller than this
~ r. This follows from the monotonicity of the function 1 (A) . As a consequence, the
distribution function (cumulative density function) of 1 is given by G() = 1 (r)
where = (r) or, equivalently, r = r () . The derivative then gives the density
function, q () =
o
oj
G() =
o
oj
1 (r ()) = , (r ())
oa
oj
.
7.4 Examples of stochastic dierence equations
We now return to the rst main objective of this chapter, the description of stochastic
processes through stochastic dierence equations.
7.4.1 A rst example
The dierence equation
Possibly the simplest stochastic dierence equation is the following
r
t
= cr
t1
+
t
. (7.7)
where the stochastic components are distributed according to some distribution func-
tion on some range which implies a mean j and a variance o
2
.
t
~ (j. o
2
) . We do not
make any specic assumption about
t
at this point. Note that the stochastic com-
ponents
t
are iid (identically and independently distributed) which implies that the
covariance between any two distinct
t
is zero, cov(
t
.
c
) = 0 \t ,= :. An alternative
representation of r
t
would be r
t
= cr
t1
+j +
t
with
t
~ (0. o
2
).
123
Solving by inserting
In complete analogy to deterministic dierence equations in ch. 2.5.3, equation
(7.7) can be solved for r
t
as a function of time and past realizations of
t
. provided
we have a boundary condition r
0
for t = 0. By repeated reinserting, we obtain
r
1
= cr
0
+
1
. r
2
= c [cr
0
+
1
] +
2
= c
2
r
0
+c
1
+
2
.
r
S
= c
_
c
2
r
0
+c
1
+
2
+
S
= c
S
r
0
+c
2
1
+c
2
+
S
and eventually
r
t
= c
t
r
0
+c
t1
1
+c
t2
2
+... +
t
= c
t
r
0
+
t
c=1
c
tc
c
. (7.8)
The mean
A stochastic dierence equation does not predict how the stochastic variable r
t
for t 0 actually evolves, it only predicts the distribution of r
t
for all future points
in time. The realization of r
t
is random. Hence, we can only hope to understand
something about the distribution of r
t
. To do so, we start by analysing the mean of
r
t
for future points in time.
Denote the expected value of r
t
by ~ r
t
.
~ r
t
= 1
0
r
t
. (7.9)
i.e. ~ r
t
is the value expected in t = 0 for r
t
in t 0. The expectations operator 1
0
captures our knowledge when we compute the expectation for r
t
. The 0 says
that we have all information for t = 0 and know therefore r
0
. but we do not know
1
.
2
. etc.
By applying the expectations operator 1
0
to (7.7), we obtain
~ r
t
= c~ r
t1
+j.
This is a deterministic dierence equation which describes the evolution of the ex-
pected value of r
t
over time. There is again a standard solution to this equation
which reads
~ r
t
= c
t
r
0
+j
t
c=1
c
tc
= c
t
r
0
+j
1 c
t
1 c
. (7.10)
where we used
t
c=1
c
tc
= c
0
+c
1
+... +c
t1
=
t1
i=0
c
i
and, from ch. 2.5.1,
a
i=0
c
i
=
(1 c
a1
) , (1 c), for c < 1. This equation shows that the expected value of r
t
changes over time as t changes. Note that ~ r
t
might increase or decrease (see the
exercises).
The variance
124
Let us now look at the variance of r
t
. We obtain an expression for the variance
by starting from (7.8) and observing that the terms in (7.8) are all independent from
each other: c
t
r
0
is a constant and the disturbances
c
are iid by assumption. The
variance of r
t
is therefore given by the sum of variances (compare (7.5) for the general
case),
\ c: (r
t
) = 0 +
t
c=1
_
c
tc
_
2
\ c: (
c
) = o
2
t
c=1
_
c
2
_
tc
= o
2
t1
i=0
_
c
2
_
i
= o
2
1 c
2t
1 c
2
.
(7.11)
We see that it is also a function of t. The fact that the variance becomes larger
the higher t is intuitively clear. The further we look into the future, the more
randomness there is: equation (7.8) shows that a higher t means that more random
variables are added up.
Long-run behaviour
When we considered deterministic dierence equations like (2.36), we found the
long-run behaviour of r
t
by computing the solution of this dierence equation and
by letting t approach innity. This would give us the xpoint of this dierence
equation as e.g. in (2.37). The concept that corresponds to a xpoint in an uncertain
environment, e.g. when looking at stochastic dierence equations like (7.7), is the
limiting distribution. As stated earlier, stochastic dierence equations do not tell
us anything about the evolution of r
t
itself, they only tell us something about the
distribution of r
t
. It would therefore make no sense to ask what r
t
is in the long run
- it will always remain random. It makes a lot of sense, however, to ask what the
distribution of r
t
is for the long run, i.e. for t .
All results so far were obtained without a specic distributional assumption for
t
apart from specifying a mean and a variance. Understanding the long-run distribution
of r
t
in (7.7) is easy if we assume that the stochastic component
t
is normally
distributed for each t.
t
~` (j. o
2
). In this case, starting in t = 0. the variable r
1
is
from (7.7) normally distributed as well. As a weighted sum of two random variables
that are (jointly) normally distributed gives again a random variable with a normal
distribution (where the mean and variance can be computed as in ch. 7.3.2), we
know from (7.7) also that r
1
. r
2
... are all normally distributed. Hence, in order to
understand the evolution of the distribution of r
t
, we only have to nd out how the
expected value and the variance of the variable r
t
evolves.
Neglecting the cases of c _ 1, the mean ~ r of our long-run normal distribution is
given from (7.10) by the xpoint
lim
t!1
~ r
t
= ~ r
1
= j
1
1 c
. c < 1.
The variance of the long-run distribution is from (7.11)
lim
t!1
c: (r
t
) = o
2
1
1 c
2
.
Hence, the long-run distribution of r
t
is a normal distribution with mean j, (1 c)
and variance o
2
, (1 c
2
).
125
The evolution of the distribution of r
t
Given our results on the evolution of the mean and the variance of r
t
and the fact
that we assumed
t
to be normally distributed, we know that r
t
is normally distributed
at each point in time. We can therefore draw the evolution of the distribution of r
as in the following gure.
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19 1
3
5
7
91
11
31
51
71
9
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
A simple stochastic difference equation
initial distribution
limiting distribution
time
x ax
t t t
= +
1

x
t
Figure 32 Evolution of a distribution over time
Remember that we were able to plot a distribution for each t only because of
properties of the normal distribution. If we had assumed that
t
is lognormally or
equally distributed, we would not have been able to easily say something about the
distribution of r
t
. The means and variances in (7.10) and (7.11) would still have been
valid but the distribution of r
t
for future t is generally unknown for distributions of
t
other than the normal distribution.
An interpretation for many agents
Imagine the variable r
it
represents nancial wealth of an individual i. Equation
(7.7) can then be interpreted to describe the evolution of wealth r
it
over time,
r
it
= cr
it1
+
it
.
Given information in period t = 0, it predicts the density function ,(r
it
) for wealth
in all future periods, given some initial value r
i0
. Shocks are identically and indepen-
dently distributed (iid) over time and across individuals.
Assuming a large number of agents i, more precisely a continuum of agents with
mass :, a law of large numbers can be constructed along the lines of Judd (1985):
126
Let all agents start with the same wealth r
i0
= r
0
in t = 0. Then the density ,(r
it
)
for wealth of any individual i in t equals the realized wealth distribution in t for
the continuum of agents with mass :. Put dierently: when the probability for an
individual to hold wealth between some lower and upper bound in t is given by j, the
share of individuals that hold wealth between these two bounds in t is also given by
j. Total wealth in t is then deterministic (as are all other moments) and is given by
r
t
= :
_
1
1
,(r
it
)r
it
dr
it
= :j
t
. (7.12)
where j
t
is average wealth over individuals or expected wealth of any individual i.
The same argument can also be made with a discrete number of agents. Wealth
r
it
of individual i at t is a random variable. Let the probability that r
it
is smaller
than r be given by 1 (r
it
_ r) = j. Now assume there are : agents and therefore at
each point in time t there are : independent random variables r
it
. Denote by : the
number of random variables r
it
that have a realization smaller than r. The share of
random variables that have a realization smaller than r is denoted by = :,:. It is
then easy to show that 1 = j for all : and, more importantly, lim
a!1
c: ( ) = 0.
This equals in words the statement based on Judd above: The share in the total
population : is equal to the individual probability if the population is large. This
share becomes deterministic for large populations.
Here is now a formal proof (thanks are due to Christian Kleiber for discussions
of this issue, see Kleiber and Kotz (2003) for more on distributions): Dene 1
i
as
the number of realizations below r for one r
it
. i.e. 1
i
= 1 (r
it
_ r) where 1 is the
indicator function which is 1 if condition in parentheses holds and 0 if not. Clearly,
the probability that 1
i
= 1 is given by j. i.e. 1
i
is Bernoulli( j) distributed, with
11
i
= j and var1
i
= j(1 j). (The 1
i
are iid if the r
it
are iid.) Then the share
is given by =
a
i=1
1
i
,:. Its moments are 1 = j and var = j(1 j),:. Hence,
the variance tends to zero for : going to innity. (More technically, tends to j in
quadratic mean and therefore in probability. The latter means we have a weak
law of large numbers.)
7.4.2 A more general case
Let us now consider the more general dierence equation
r
t
= c
t
r
t1
+
t
.
where the coecient c is now also stochastic
c
t
~
_
c. o
2
o
_
.
t
~
_
j. o
2
.
_
.
Analyzing the properties of this process is pretty complex. For details see Vervaat
(1979).
What can be easily done, however, is to analyze the evolution of moments. As
Vervaat has shown, the limiting distribution has the following moments
1r
)
=
)
I=0
_
)
I
_
1
_
c
I
)I
_
1r
I
.
127
Assuming we are interested in the expected value, we would obtain
1r
1
=
1
I=0
_
1
I
_
1
_
c
I
1I
_
1r
I
=
_
1
0
_
1
_
c
0
1
_
1r
0
+
_
1
1
_
1
_
c
1
0
_
1r
1
= 1 +1c1r.
Solving for 1r yields 1r =
1.
11c
=
j
1o
.
7.5 A simple OLG model
Let us now return to maximization issues. We do so in the context of the most simple
example of a dynamic stochastic general equilibrium model. It is a straightforward
extension of the deterministic model analyzed in section 2.4.
The structure of the maximization problem of individuals, the timing of when
uncertainty reveals and what is uncertain depends on the fundamental and exogenous
sources of uncertainty. As the fundamental source of uncertainty results here from
the technology used by rms, technologies will be presented rst. Once this is done,
we can derive the properties of the maximization problem households or rms face.
7.5.1 Technology
Let there be an aggregate technology
1
t
=
t
1
c
t
1
1c
t
(7.13)
The total factor productivity level
t
is uncertain. We assume that
t
is identically
and independently distributed (iid) in each period. The random variable
t
is pos-
itive, has a mean and a variance o
2
. No further information is needed about its
distribution function at this point,
t
~
_
. o
2
_
.
t
0. (7.14)
Assuming that total factor productivity is iid means, inter alia, that there is no
technological progress. One can imagine many dierent distributions for
t
. In
principle, all distributions presented in the last section are viable candidates. Hence,
we can work with discrete distributions or continuous distributions.
7.5.2 Timing
The sequence of events is as follows. At the beginning of period t, the capital stock
1
t
is inherited from the last period, given decisions of the last period. Then, total
factor productivity is revealed. With this knowledge, rms choose factor employment
and households choose consumption (and thereby savings).
c
t
w
t
r
t
K
t
A
t
128
Figure 33 Timing of events
Hence, only at the end of the day one really knows how much one has produced.
This implies that wages and interest payments are known with certainty at the end
of the period only as well.
The state of the economy in t is completely described by 1
t
and the realization
of
t
. All variables of the model are contingent on the state.
7.5.3 Firms
As a consequence of this timing of events, rms do not bear any risk and they pay
the marginal product of labour and capital to workers and capital owners at the end
of the period,
n
t
= j
t
J1
t
J1
t
. (7.15)
:
t
= j
t
J1
t
J1
t
= j
t
t
c
_
1
t
1
t
_
1c
. (7.16)
All risk is therefore born by households through labour and capital income.
7.5.4 Individual decision problem
General approach
We consider an agent that lives for two periods and works and consumes in a
world as just described. Agents consume in both periods and choose consumption
such that they maximize expected utility. In all generality concerning the uncertainty,
she maximizes
max 1
t
n(c
t
) +,n(c
t1
) . (7.17)
where , is the subjective discount factor that measures the agents impatience to
consume. The expectations operator has an index t to indicate, similar to 1
0
in
(7.9), that expectations are based on the knowledge about random variables which
is available in period t. We will see in an instant whether the expectations operator
1
t
is put at a meaningful place in this objective function. Placing it in front of both
instantaneous consumption from c
t
and from c
t1
is the most general way of handling
it. We will also have to specify later what the control variable of the household is.
The agent has to choose consumption for the rst and the second period. When
consumption is chosen and given wage income n
t
. savings adjust such that the budget
constraint
n
t
= c
t
+:
t
(7.18)
for the rst period holds. Note that this constraint always holds, despite the uncer-
tainty about the wage. It holds in realizations, not in expected terms. In the second
period, the household receives interest payments on savings made in the rst period
and uses savings plus interests for consumption,
129
(1 +:
t1
) :
t
= c
t1
. (7.19)
One way of solving this problem (for an alternative, see the exercise) is to insert
consumption levels from these two constraints into the objective function (7.17). This
gives
max
c
I
1
t
n(n
t
:
t
) +,n((1 + :
t1
) :
t
) .
This nicely shows that the household inuences consumption in both periods by
choosing savings :
t
in the rst period. In fact, the only control variable the household
can choose is :
t
.
Let us now take into consideration, as drawn in the above gure, that consumption
takes place at the end of the period after revelation of productivity
t
in that period.
Hence, the consumption level in the rst period is determined by savings only and
thereby certain.
7
The consumption level in the second period is uncertain, however, as
the next period interest rate :
t1
depends on the realization of
t1
which is unknown
in t when decisions about savings :
t
are made. The objective function can therefore
be written as
max
c
I
n(n
t
:
t
) +,1
t
n((1 + :
t1
) :
t
) .
Let us now, for illustration purposes, assume a discrete random variable
t
with
a nite number : of possible realizations. Then, this maximization problem can be
written as
max
c
I
n(n
t
:
t
) +,
a
i=1
j
i
n((1 + :
i,t1
) :
t
)
where j
i
is the probability that the interest rate :
i,t1
is in state i in t + 1. This is
the same probability as the probability that the underlying source of uncertainty
t
is in state i. The rst-order condition then reads
n
0
(n
t
:
t
) = ,
a
i=1
j
i
n
0
((1 + :
i,t1
) :
t
) (1 +:
i,t1
) = ,1
t
n
0
((1 + :
t1
) :
t
) (1 +:
t1
)
(7.20)
Marginal utility of consumption today must equal expected marginal utility of con-
sumption tomorrow corrected by interest and time preference rate. Optimal behav-
iour in an uncertain world therefore means ex ante optimal behaviour, i.e. before
random events are revealed. Ex post, i.e. after resolution of uncertainty, behaviour
is suboptimal as compared to the case where the realization is known in advance:
Marginal utility in t will (with high probability) not equal marginal utility (corrected
by interest and time preference rate) in t + 1. This reects a simple fact of life: If
I had known before what would happen, I would have behaved dierently. Ex ante,
behaviour is optimal, ex post, probably not. Clearly, if there was only one realization
for
t
, i.e. j
1
= 1 and j
i
= 0 \ i 1. we would have the deterministic rst-order
condition we had in exercise 1 in ch. 2.
The rst-order condition also shows that closed form solutions are possible if
marginal utility is of a multiplicative type. As savings :
t
are known, the only quantity
7
If consumption c
t
(or savings) was chosen before revelation of total factor productivity, house-
holds would want to consume a dierent level of consumption c
t
after A
t
is known.
130
which is uncertain is the interest rate :
t1
. If the instantaneous utility function n(.)
allows to separate the interest rate from savings, i.e. the argument (1 +:
i,t1
) :
t
in
n
0
((1 + :
i,t1
) :
t
) in (7.20), an explicit expression for :
t
and thereby consumption can
be computed. This will be shown in the example following right now and in exercise
5.
An example - Cobb-Douglas preferences
Assume now the household maximizes a Cobb-Douglas utility function as in (2.5).
In contrast to the deterministic setup in (2.5), however, expectations about consump-
tion levels need to be formed. Preferences are therefore captured by
1
t
ln c
t
+ (1 ) ln c
t1
. (7.21)
When we inserting savings, we can express the maximization problem by
max
c
I
1
t
ln (n
t
:
t
) + (1 ) ln ((1 + :
t1
) :
t
)
= max
c
I
ln (n
t
:
t
) + (1 ) [ln :
t
+1
t
ln (1 +:
t1
)] . (7.22)
The rst-order condition with respect to savings reads
n
t
:
t
=
1
:
t
(7.23)
and the optimal consumption and saving levels are
c
t
= n
t
. c
t1
= (1 ) (1 +:
t1
) n
t
. :
t
= (1 ) n
t
. (7.24)
just as in the deterministic case (2.24).
Note that we can compute a closed form solution of the same structure as in the
deterministic solutions (2.7) and (2.8), despite the setup with uncertainty. What is
further peculiar about the solutions and also about the rst-order condition is the fact
that the expectations operator is no longer visible. One could get the impressions that
households do not form expectations when computing optimal consumption paths.
The expectations operator got lost in the rst-order condition only because of the
logarithm, i.e. the Cobb-Douglas nature of preferences. Nevertheless, there is still
uncertainty for an individual being in t : consumption in t + 1 is unknown as it is a
function of the interest-rate in t + 1.
7.5.5 Aggregation and the reduced form for the CD case
We now aggregate over all individuals. Consumption of all young individuals in period
t is given from (7.24),
C
j
t
= `n
t
= (1 c)1
t
.
The second equality used (7.15), the fact that with a Cobb-Douglas technology (7.15)
can be written as n
t
1
t
= j
t
(1 c) 1
t
. the normalization of j
t
to unity and the
131
identity between number of workers and number of young, 1
t
= `. Note that this
expression would hold identically in a deterministic model. With (7.16), consumption
of old individuals amounts to
C
c
t
= `(1 )(1 + :
t
)n
t1
= (1 )(1 + :
t
)(1 c)1
t1
.
The capital stock in period t +1 is given by savings of the young. One could show
this as we did in the deterministic case in ch. 2.4.2. In fact, adding uncertainty would
change nothing to the fundamental relationships. We can therefore directly write
1
t1
= `:
t
= `(1 )n
t
= (1 ) (1 c) 1
t
.
where we used the expression for savings for the Cobb-Douglas utility case from (7.24).
Again, we succeeded in reducing the presentation of the model to one equation in one
unknown. This allows us to illustrate the economy by using the simplest possible
phase diagram.
N
N
1(t) 1
0
1
1
1
2
1
S
1
t1
1
2
= (1 )(1 c)
1
1
c
1
1
1c
1
S
= (1 )(1 c)
2
1
c
2
1
1c
1
1
= (1 )(1 c)
0
1
c
0
1
1c
Figure 34 Convergence towards a stochastic steady state
This phase diagram looks in principle identical to the deterministic case. There
is of course a fundamental dierence. The 1
t1
loci are at a dierent point in each
period. In t = 0, 1
0
and
0
are known. Hence, by looking at the 1
1
loci, we can
compute 1
1
as in a deterministic setup. Then, however, TFP changes and, in the
example plotted above,
1
is larger than
0
. Once
1
is revealed in t = 1. the new
capital stock for period 2 can be graphically derived. With
2
revealed in t = 2. 1
S
can be derived an so on. It is clear that this economy will never end up in one steady
state as in the deterministic case as
t
is dierent for each t. As illustrated before,
however, the economy can converge to a stationary stable distribution for 1
t
.
7.5.6 Some analytical results
The basic dierence equation
132
In order to better understand the evolution of this economy, let us now look at
some analytical results. The logarithm of the capital stock evolves according to
ln 1
t1
= ln ((1 ) (1 c)) + ln
t
+cln 1
t
+ (1 c) ln 1
t
.
Assuming a constant population size 1
t
= 1, we can rewrite this equation as
i
t1
= :
0
+ci
t
+
t
.
t
~ `(j
i
. o
2
u
). (7.25)
where we used
i
t
= ln 1
t
(7.26)
:
0
= ln [(1 ) (1 c)] + (1 c) ln 1 (7.27)
and
t
= ln
t
captures the uncertainty stemming from the TFP level
t
. As TFP is
iid, so is its logarithm
t
. Since we assumed the TFP to be lognormally distributed,
its logarithm
t
is normally distributed. When we remove the mean from the random
variable by replacing according to
t
=
t
+j
i
. where
t
~ `(0. o
2
u
). we obtain
i
t1
= :
0
+j
i
+ci
t
+
t
.
The expected value and the variance
We can now compute the expected value of i
t
by following the same steps as in
ch. 7.4.1, noting that the only additional parameter is :
0
. Starting in t = 0 with
some initial value i
0
. and solving this equation recursively gives
i
t
= c
t
i
0
+ (:
0
+j
i
)
1 c
t
1 c
+
t1
c=0
c
t1c
c
. (7.28)
Computing the expected value from the perspective of t = 0 gives
1
0
i
t
= c
t
i
0
+ (:
0
+j
i
)
1 c
t
1 c
. (7.29)
where we used 1
0
c
= 0 for all : 0 (and we set
0
= 0 or assumed that expectations
in 0 are formed before the realization of
0
is known). For t going to innity, we
obtain
lim
t!1
1
0
i
t
=
:
0
+j
i
1 c
. (7.30)
Note that the solution in (7.28) is neither dierence- nor trend stationary. Only in
the limit, we have a (pure) random walk. For a very large t. c
t
in (7.28) is zero and
(7.28) implies
i
t
i
t1
=
t1
.
The variance of i
t
can be computed with (7.28), where again independence of all
terms is used as in (7.11),
\ c: (i
t
) = 0 + 0 + \ c:
_
t1
c=0
c
t1c
c
_
=
t1
c=0
c
2(t1c)
o
2
i
=
1 c
2t
1 c
2
o
2
i
(7.31)
In the limit, we obtain
lim
t!1
\ c: (i
t
) =
o
2
i
1 c
2
. (7.32)
133
Relation to fundamental uncertainty
For a more convincing economic interpretation of the mean and the variance, it
is useful to express some of the equations not as a function of properties of the log
of
t
. i.e. properties of i
t
. but directly of the level of
t
. As (7.6) implies, the mean
and variances of these two variables are related in the following fashion
o
2
i
= ln
_
1 +
_
o
_
2
_
(7.33)
j
i
= ln j
1
2
o
2
i
(7.SS)
= ln j
1
2
ln
_
1 +
_
o
_
2
_
(7.34)
We can insert these expressions into (7.29) and (7.31) and study the evolution over
time or directly focus on the long-run distribution of i
t
= ln 1
t
. Doing so by inserting
in (7.30) and (7.32) gives
lim
t!1
1
0
i
t
=
:
0
+ ln j
1
2
ln
_
1 +
_
o
/
j
/
_
2
_
1 c
. lim
t!1
\ c: (i
t
) =
ln
_
1 +
_
o
/
j
/
_
2
_
1 c
2
.
These equations tell us that uncertainty in TFP as captured by o
not only aects

the spread of the long-run distribution of the capital stock but also its mean. More
uncertainty leads to a lower long-run mean of the capital stock. This is interesting
given the standard result of precautionary saving and the portfolio eect (in setups
with more than one asset). (t.b.c.)
7.6 Risk-averse and risk-neutral households
Previous analyses in this book have worked with the represantitive-agent assumption.
This means that we neglected all potential dierences accross individuals and assumed
that they are all the same (especially identical preferences, labour income and initial
wealth). We now extend the analysis of the last section by (i) allowing individuals
to give loans to each other. These loans have a riskless endogenous interest rates
:. As before, there is a normal asset which pays an uncertain return :
t1
. We
also (ii) assume there are two types of individuals, the risk-neutral ones and the
risk-averse individuals, denoted by i = c. :.
8
We want to understand in this world
with heterogeneous agents who owns which assets. We keep the analysis simple by
analysing a partial equilibrium setup.
Households
8
If individuals were identical, i.e. if they had identical preferences and experienced the same
income streams, no loans would be given in equilibrium.
134
The budget constraints (7.18) and (7.19) of all individuals i now read
n
t
= c
i
t
+:
i
t
. (7.35)
c
i
t1
= :
i
t
_
1 +o
i
:
t1
+
_
1 o
i
_
:
. (7.36)
Here and in subsequent chapters, o
i
denotes the share of wealth held by individual i
in the risky asset. A household i with time preference rate j and thereby discount
factor , = 1, (1 +j) maximizes
l
i
= n
_
n
t
:
i
t
_
+,1n
__
1 +o
i
:
t1
+
_
1 o
i
_
:
:
i
t
_
max
c
.
I
,0
.
(7.37)
by now choosing two control variables: The amount of resources not used in the rst
period for consumption, i.e. savings :
i
t
. and the share o
i
of savings held in the risky
asset.
First-order conditions for :
i
t
ando
i
, respectively, are
n
0
_
c
i
t
_
= ,1
t
_
n
0
_
c
i
t1
_ _
1 +o
i
:
t1
+
_
1 o
i
_
:
_
. (7.38)
1n
0
_
c
i
t1
_
[:
t1
:] = 0. (7.39)
Note that the rst-order condition for consumption (7.38) has the same interpre-
tation, once slightly rewritten, as the interpretation in deterministic two-period or
innite horizon models (see (2.9) in ch. 2.2.2 and (3.5) in ch. 3.2). When rewriting
it as
1
t
_
,n
0
_
c
i
t1
_
n
0
(c
i
t
)
1
1,
_
1 +o
i
:
t1
+
_
1 o
i
_
:
_
_
= 1. (7.40)
we see that optimal behaviour again requires to equate marginal utilities today and
tomorrow (the latter in its present value) with relative prices today and tomorrow
(the latter also in its present value). Of course, in this stochastic environment, we
need to express everything in expected terms. As the interest rates and consumption
tomorrow are jointly uncertain, we can not bring it exactly in the form as known from
above in (2.9) and (3.5). This will be possible, however, further below in (8.8) in ch.
8.2.
The rst-order condition for o says that expected returns from giving a loan and
holding the risky asset must be identical. Returns consist of the interest rate times
marginal utility. This condition can best be understood when rst thinking of a
certain environment. In this case, (7.39) would read :
t1
= : : agents would be
indierent between holding two assets only if they receive the same interest rate on
both assets. Under uncertainty and with risk-neutrality of agents, i.e. n
0
= co::t.,
we get 1
t
:
t1
= : : Agents hold both assets only if the expected return from the risky
assets equals the certain return from the riskless asset.
Under risk-aversion, we can write this condition as 1
t
n
0
_
c
i
t1
_
:
t1
= 1
t
n
0
_
c
i
t1
_
:
(or, given that : is non-stochastic, as 1
t
n
0
_
c
i
t1
_
:
t1
= :1
t
n
0
_
c
i
t1
_
). This says
that agents do not value the interest rate per se but rather the extra utility gained
from holding an asset: An asset provides interest of :
t1
which increases utility
135
by n
0
_
c
i
t1
_
:
t1
. The share of wealth held in the risky asset then depends on the
increase in utility from realizations of :
t1
accross various states and the expecta-
tions operator computes a weighted sum of theses utility increases: 1
t
n
0
_
c
i
t1
_
:
t1
=
a
)=1
n
0
_
c
i
)t1
_
:
)t1
j
)
. where , is the state, :
)t1
the interest rate in this state and j
)
the probability for state , to occur. An agent is then indierent between holding two
assets when expected utility-weighted returns are identical.
Risk-neutral and risk-averse behaviour
For risk-neutral individuals, the rst-order conditions become
1 = ,1 [1 +o
a
:
t1
+ (1 o
a
) :] . (7.41)
1:
t1
= :. (7.42)
The rst-order condition on how to invest implies together with (7.41) that the en-
dogenous interest rate for loans is pinned down by the time preference rate,
1 = , [1 +:] =: = j (7.43)
as the discount factor is given by , = (1 + j)
1
as stated before (7.37). Reinserting
this result into (7.42) shows that we need to assume that an interior solution for
1:
t1
= j exists. This is not obvious for an exogenously given distribution for the
interest rate :
t
but it is more plausible for a situation where :
t
is stochastic but
endogenous as in the analysis of the OLG model in the previous chapter.
For risk-averse individuals with n(c
o
t
) = ln c
o
t
. the rst-order condition for con-
sumption reads
1
c
o
t
= ,1
1
c
o
t1
[1 +o
o
:
t1
+ (1 o
o
) :] = ,1
1
:
o
t
= ,
1
:
o
t
(7.44)
where we used (7.36) for the second equality and the fact that :
t
as control variable
is deterministic for the third. Hence, as in the last section, we can derive explicit
expressions. Use (7.44) and (7.35) and nd
:
o
t
= ,c
o
t
=n
t
c
o
t
= ,c
o
t
=c
o
t
=
1
1 +,
n
t
.
This gives with (7.35) again and with (7.36)
:
o
t
=
,
1 +,
n
t
. c
o
t1
=
,
1 +,
[1 +o
o
:
t1
+ (1 o
o
) :] n
t
.
This is our closed-form solution for risk-averse individuals in our heterogeneous-agent
economy.
Let us now look at the investment problem of risk-averse households. The deriva-
tive of their objective function is given by the LHS of the rst-order condition (7.39).
136
Expressed for logarithmic utility function, and inserting the optimal consumption
result yields
1
1
c
o
t1
[:
t1
:] =
1 +,
,
1
:
t1
:
1 +o
o
:
t1
+ (1 o
o
) :
=
1 +,
,
1
:
t1
:
1 +o
o
(:
t1
:) +:
=
1 +,
,
1
A
c +oA
. (7.45)
The last step dened A = :
t1
: as a RV and c as a constant.
Who owns what?
It can now easily be shown that (7.45) implies that risk-averse individuals will
not allocate any of their savings to the risky asset, i.e. o
o
_ 0: The derivative of the
expression 1A, (c +o
o
A) with respect to o
o
is negative
d
do
o
1
A
c +o
o
A
= 1
_
A
2
(c +o
o
A)
2
_
< 0 \o
o
.
The sign of this derivative can also easily be seen from (7.45) as an increase in o
o
implies a larger denominator. Hence, when plotted, the rst-order condition is down-
ward sloping in o
o
. By guessing we nd that, with (7.42), o
o
= 0 satises the rst-order
condition for investment,
o
o
= 0 =1
A
c +o
o
A
= 1A = 0.
Hence, the rst-order condition is zero for o
o
= 0. As the rst-order condition is
monotonously decreasing, o
o
= 0 is the only value for which it is zero,
1
A
c +o
o
A
= 0 =o
o
= 0.
This is illustrated in the following gure.
N
N
0 o
o
1
a
o0
a
a
137
Figure 35 The rst-order condition for the share o
o
of savings held in the risky asset
The gure shows that expected utility of a risk-averse individual is the higher,
the lower the share of savings held in the risky asset. If we exclude short-selling,
risk-averse individuals will allocate all of their savings to loans, i.e. o
o
= 0. They
give loans to risk-neutral individuals who in turn pay a certain interest rate equal to
the expected interest rate. All risk is born by risk-neutral individuals.
7.7 Pricing of contingent claims and assets
7.7.1 The value of an asset
The question is what an individual is willing to pay in t = 0 for an asset that is
worth j
)
(which is uncertain) in t = 1. This is inspired by Brennan (1979, JOF). As
the assumption underlying the pricing of assets is that individuals behave optimally,
the answer is found by considering a maximization problem where individuals solve
a saving and a portfolio problem.
The individual
Let preferences of an individual be given by
1 n(c
0
) +,n(c
1
) .
This individual can invest in the risky and in a riskless investment form. It earns
labour income n in the rst period and does not have any other source of income.
This income n is used for consumption and savings. Savings are allocated to several
risky assets , and a riskless bond. The rst period budget constraint therefore reads
n = c
0
+
a
)=1
:
)
j
)0
+/
where the number of shares , are denoted by :
)
. their price by j
)0
and the amount
of wealth held in the riskless asset is /. The individual consumes all income in the
second period which implies the budget constraint
c
1
=
a
)=1
:
)
j
)1
+ (1 + :) /.
where : is the interest rate on the riskless bond.
We can summarize this maximization problem as
max
c
0
,n
n(c
0
) +,1n((1 + :) (n c
0
)
:
)
j
)0
) +
)
:
)
j
)1
) .
This expression illustrates that consumption in period 1 is given by capital income
from the riskless interest rate : plus income from assets ,. It also shows that there is
uncertainty only with respect to period one consumption. This is because consump-
tion in period zero is a control variable. The consumption in period 1 is uncertain as
prices of assets are uncertain.
138
Solution and asset pricing
Computing rst-order conditions gives
n
0
(c
0
) = (1 + :) ,1n
0
(c
1
)
This is the standard rst-order condition for consumption. The rst-order condition
for the number of assets to buy reads for any asset ,
,1 [n
0
(c
1
) (1 +:) (j
)0
) +j
)1
] = 0 =1 [n
0
(c
1
) j
)1
] = (1 +:) j
)0
1n
0
(c
1
) .
The last step used the fact that the interest rate and j
)0
are known in 0 and can there-
fore be pulled out of the expectations operatior on the right-hand side. Reformulating
this condition gives
j
)0
=
1
1 +:
1
n
0
(c
1
)
1n
0
(c
1
)
j
)1
(7.46)
which is an expression that can be given an interpretation as a pricing equation. The
price of an asset , in period zero is given by the discounted expected price in period
one. The price in period one is weighted by marginal utilities.
7.7.2 The value of a contingent claim
What we are now interested in are prices of contingent claims. Generally speaking,
contingent claims are claims that can be made only if certain outcomes occur. The
simplest example of the contingent claim is an option. An option gives the buyer the
right to buy or sell an asset at a set price on or before a given date. We will consider
current contingent claims whose value can be expressed as a function of the price of
the underlying asset. Denote by q (j
)1
) the value of the claim as a function of the
price j
)1
of asset , in period 1. The price of the claim in 0 is denoted by (j
)0
) .
When we add an additional asset, i.e. the contingent claim under consideration,
to the set of assets considered in the previous section, then optimal behaviour of
households implies
(j
)0
) =
1
1 +:
1
n
0
(c
1
)
1n
0
(c
1
)
q (j
)1
) . (7.47)
If all agents were risk neutral, we would know that the price of such a claim would
be given by
(j
)0
) =
1
1 +:
1q (j
)1
) (7.48)
Both equations directly follow from the rst-order condition for assets. Under risk
neutrality, a households utility function is linear in consumption and the rst deriv-
ative is therefore a constant. Including the contingent claim into the set of assets
available to households, exactly the same rst-order condition for their pricing would
hold.
139
7.7.3 Risk-neutral valuation
There is a strand in the nance literature, see e.g. Brennan (1979), that asks under
what conditions a risk neutral valuation of contingent claims holds (risk neutral val-
uation relationship, RNVR) even when households are risk averse. This is identical
to asking under which conditions marginal utilities in (7.46) do not show up. We will
now briey illustrate this approach and drop the index ,.
Assume the distribution of the price j
1
can be characterized by a density , (j
1
. j) .
where j = 1j
1
. Then, if a risk neutral valuation relationship exists, the price of the
contingent claim in zero is given by
(j
0
) =
1
1 +:
_
q (j
1
) , (j
1
. j) dj
1
. with j = (1 +:) j
0
.
This is (7.48) with the expectations operator being replaced by the integral over
realizations q (j
1
) times the density , (j
1
). With risk averse households, the pricing
relationship would read under this distribution
(j
0
) =
1
1 +:
_
n
0
(c
1
)
1n
0
(c
1
)
q (j
1
) , (j
1
. j) dj
1
.
This is (7.47) expressed without the expectations operator. The expected value j
is left unspecied here as it is not a priori clear whether this expected value equals
(1 +:) j
0
also under risk aversion. It is then easy to see that a RNVR holds if
n
0
(c
1
) ,1n
0
(c
1
) = , (j
1
) ,, (j
1
. j) . Similar conditions are derived in that paper for
other distributions and for longer time horizons.
7.8 Natural volatility I (*)
Natural volatility is a view about why growing economies experience phases of high
and phases of low growth. The central belief is that both long-run growth and short-
run uctuations are jointly determined by economic forces that are inherent to any real
world economy. Long-run growth and short-run uctuations are both endogenous and
two sides of the same coin: They both stem from the introduction of new technologies.
It is important to note that no exogenous shocks occur according to this approach.
In this sense, it diers from RBC and sunspot models and also from endogenous
growth models with exogenous disturbances (endogenous RBC models).
There are various models that analyse this view in more details and an overview
is provided at http://www.wifak.uni-wuerzburg.de/vwl2/nv. This section will look
at a simple model that provides the basic intuition. More on natural volatility will
follow in ch. ?? and ch. 10.5.
7.8.1 The basic idea
The basic mechanism of the natural volatility literature (and this is probably a nec-
essary property of any model that wants to explain both short-run uctuations and
long-run growth) is that some measure of productivity (this could be labour or total
140
factor productivity) does not grow smoothly over time as in most models of exogenous
or endogenous long-run growth but that productivity follows a step function.
N
N
time
productivity step function
log of
productivity
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
smooth productivity growth
h
h
x
h x
h
x h
x
x h
x
Figure 36 Smooth productivity growth in balanced growth models (dashed line) and
step-wise productivity growth in models of natural volatility
With time on the horizontal and the log of productivity on the vertical axis, this
gure shows as the dashed line a smooth productivity growth path. This is the smooth
growth path that induces balanced growth. In models of natural volatility, the growth
path of productivity has periods of no change at all and points in time of discrete
jumps. After a discrete jump, returns to investment go up and an upward jump in
growth rates results. Growth rates gradually fall over time as long as productivity
remains constant. With the next jump, growth rates jump up again. While this step
function implies long-run growth as productivity on average grows over time, it also
implies short-run uctuations.
The precise economic reasons given for this step function - which is not simply
imposed but always follows from some deeper mechanisms - dier from one approach
to the other. A crucial implication of this step-function is the implicit belief that
economically relevant technological jumps take place once every 4-5 years. Each
cycle of an economy and thereby also long-run growth go back to relatively rare
events. Fluctuations in time series that are of higher frequency than these 4-5 years
either go back to exogenous shocks, to measurement error or other disturbances of
the economy.
The step function sometimes captures jumps in total factor productivity, some-
times only in labour productivity of a most recent vintage of a technology. This dier-
ence is important for the economic plausibility of the models. Clearly, one should not
build a theory on large aggregate shocks to TFP as those are not easily observable.
Some papers in the literature show indeed how small changes in technology can have
large eects. See Francois and Lloyd-Ellis (2003) or Wlde (2002, 2005). See also
Matsuyama (1999).
This section presents the simples stochastic natural volatility model which allows
to show most easily the dierence to exogenous shock models of the business cycle.
141
7.8.2 A simple stochastic model
This section presents the simplest possible model that allows to understand the dif-
ference between the stochastic natural volatility and the RBC approach.
Technologies
Let the technology be described by a Cobb-Douglas specication,
1
t
=
t
1
c
t
1
1c
t
.
where
t
represents total factor productivity, 1
t
is the capital stock and 1
t
are hours
worked. Capital can be accumulated according to
1
t1
= (1 o) 1
t
+1
t
The technological level follows
t1
= (1 +
t
)
t
where
t
=
_
0
1
_
with probability
_
j
t
1 j
t
. (7.49)
The probability depends on resources 1
t
invested into R&D,
j
t
= j (1
t
) .
Clearly, the function j (1
t
) in this discrete time setup must be such that 0 _ j (1
t
) _
1.
The specication of technological progress in (7.49) is probably best suited to point
out the dierences to RBC type approaches: The probability that a new technology
occurs is endogenous. This shows both the new growth literature tradition and
the dierences to endogenous growth type RBC models. In the latter approach, the
growth rate is endogenous but shocks are still exogenously imposed. Here, the source
of growth and of uctuations all stem from one and the same source, the jumps in
t
in (7.49).
Optimal behaviour of households
The resource constraint the economy needs to obey in each period is given by
C
t
+1
t
+1
t
= 1.
where C
t
. 1
t
and 1
t
are aggregate consumption, investment and R&D expenditure,
respectively. Assume that optimal behaviour of households implies consumption and
investment into R&D amounting to
C
t
= :
1
1
t
. 1
t
= :
1
1
t
.
where :
1
and :
1
are two constant saving rates. This would be the outcome of a two-
period maximization problem or an innite horizon maximization problem with some
parameter restriction similar to e.g. Benhabib and Rustichini (1994) or, in continuous
time, Xie (1991,) or Wlde (2005). As the natural volatility literature has various
papers where the saving rate is not constant (Matsuyama, 1999, assumes a constant
saving rate, but see Matsuyama, 2001), it seems reasonable not to fully develop an
optimal saving approach here as it is not central to the natural volatility view.
142
7.8.3 Equilibrium
Equilibrium is determined by
1
t1
= (1 o) 1
t
+1
t
1
t
C
t
= (1 o) 1
t
+ (1 :
1
:
1
) 1
t
.
plus the random realization of technology jumps, where the probability of a jump
depends on investment in R&D,
j
t
= j (:
1
1
t
) .
When we use as auxiliary variable capital per eective labour, /
t
= 1
t
, (
t
1) .
we obtain with
1
I+1
I
1
= /
t1
I+1
I
= /
t1
(1 +
t
)
/
t1
=
(1 o) /
t
+ (1 :
1
:
1
) /
c
t
1 +
t
. (7.50)
The convergence behaviour is illustrated in the following gure.
Figure 37 The Sisyphus economy - convergence to a temporary steady state (phasendi-
agram.nb)
The gure plots the current capital stock per eective labour /
t
on the horizontal
and /
t1
on the vertical axis. As long as there is no technology jump, i.e. as long
as
t
= 0. the capital stock /
t
converges to its temporary steady state /
q
just as in
deterministic models. It follows by setting /
t1
= /
t
= /
q
in (7.50) and is given by
/
q
=
(1 o) /
q
+ (1 :
1
:
1
)
_
/
q
_
c
1 +
t
=(
t
+o) /
q
= (1 :
1
:
1
)
_
/
q
_
c
=/
q
=
_
1 :
1
:
1
t
+o
_
1(1c)
. (7.51)
143
With a new technology, /
t
= 1
t
, (
t
1) decreases and the /
t1
line increases, as
shown by the dashed line in g. 37. As a consequence, as can also be seen in (7.51),
the steady state increases. Subsequently, the economy approaches the steady state
again. As growth is higher immediately after a new technology is introduced (growth
is high when the economy is far away from its steady state), growth rates after the
introduction of a new technology is high and then gradually falls. Cyclical growth
is therefore characterized by a Sisyphus-type behaviour: /
t
permanently approaches
the steady state but eventually is thrown back due to the arrival of a new technology.
t.b.c.
More on basic concepts of random variables than in ch. 7.1 and 7.2 can be found
in Evans, Hastings and Peacock (2000). For a more advanced treatment, see John-
son, Kotz and Balakrishnan (1995). The results on distributions here are used and
extended in Kleiber, Sexauer and Wlde (2006).
Some references on stochastic dierence equations include Ptzelberger, Babillot
et al, Vervaat (1979) and Wolfe (1982).
144
Exercises Chapter 7
Stochastic dierence equations and
applications
1. Properties of random variables
Is the following function a density function? Draw this function.
,(r) = c`c
jAaj
. ` 0. (7.52)
State possible assumptions about the range of r and values for c.
2. The properties of uncertain technological change
Assume that total factor productivity in (7.14) is given by
(a)
t
=
_

with p
A
with 1-p
_
.
(b) the density function
,(
t
).
t

_
A
A
_
.
(c) the probability function q(
t
).
t

1
.
2
. ....
a
.
(d)
t
=
i
with probability j
i
and
(e)
t1
=
t
+
t1
. Make assumptions for and
t1
and discuss them.
What is the expected total factor productivity and what is its variance? What
is the expected output level and what is its variance, given a technology as in
(7.13)?
3. Stochastic dierence equations
Consider the following stochastic dierence equation
t1
= /
t
+:+
t1
.
t
~`(0. o
2
)
(a) Describe the limiting distribution of
t
.
(b) Does the expected value of
t
converge to its xpoint monotonically? How
does the variance of
t
evolve over time?
145
4. Saving under uncertainty
Consider an individuals maximization problem
max 1
t
n(c
t
) +,n(c
t1
)
subject to n
t
= c
t
+:
t
. (1 +:
t1
) :
t
= c
t1
(a) Solve this problem by replacing her second period consumption by an ex-
pression that depends on rst period consumption.
(b) Consider now the individual decision problem given the utility function
n(c
t
) = c
o
t
. Should you assume a parameter restriction on o?
(c) What is your implicit assumption about ,? Can it be negative or larger
than one? Can the time preference rate j. where , = (1 + j)
1
. be nega-
tive?
5. Closed-form solution for CES utility function
Let households maximize a CES-utility function
l
t
= 1
t
_
c
1o
t
+ (1 ) c
1o
t1
= c
1o
t
+ (1 ) 1
t
c
1o
t1
subject to budget constraints (7.18) and (7.19).
(a) Show that an optimal consumption-saving decision given budget constraints
(7.18) and (7.19) implies savings of
:
t
=
n
t
1 +
_

(1)4
_
.
.
Show further that consumption when old is
c
t1
= (1 +:
t1
)
n
t
1 +
_

(1)4
_
.
and that consumption of the young is
c
t
=
_

(1)4
_
.
1 +
_

(1)4
_
.
n
t
. (7.53)
(b) Discuss the link between the savings expression :
t
and the one for the loga-
rithmic case in (7.24). Point out why c
t1
is uncertain from the perspective
of t. Is c
t
uncertain from the perspective of t - and of t 1?
6. OLG in general equilibrium
Build an OLG model in general equilibrium with capital accumulation and auto-
regressive total factor productivity, ln
t1
= ln
t
+
t
. What is the reduced
form? Can a phase-diagram be drawn?
146
7. Asset pricing
Under what conditions is there a risk neutral valuation formula for assets? In
other words, under which conditions does the following equation hold?
j
)c
=
1
1 +:
1j
)1
147
8 Multi-period models
Uncertainty in dynamic models is probably most often used in discrete time models.
Looking at time with a discrete perspective has the advantage that timing issues
are very intuitive: Something happens today, something tomorrow, something the
day before. Time in the real world, however, is continuous. Take any two points
in time, you will always nd a point in time in between. What is more, working
with continuous time models under uncertainty has quite some analytical advantages
which make certain insights - after an initial more heavy investment into techniques
- much simpler.
We will nevertheless follow the previous structure of this book and rst present
models in discrete time. They are also, as mentioned, probably the more widely used
models in the literature.
8.1 Dynamic programming
Maximization problems in discrete times can be solved by using many methods. One
particularly useful one is - again - dynamic programming. We therefore start with
this method and consider the stochastic sibling to the deterministic case in ch. 3.4.
Due to uncertainty, our objective function is slightly changed. In contrast to (3.1),
it now reads
l
t
= 1
t
1
t=t
,
tt
n(c
t
) . (8.1)
where the only dierence consists in the expectations operator 1
t
. As we are in
an environment with uncertainty, we do not know all consumption levels c
t
with
certainty. We therefore need to form expectations about the implied instantaneous
utility from consumption, n(c
t
). The budget constraint is given by
r
t1
= , (r
t
. c
t
.
t
) . (8.2)
This constraint shows why we need to form expectations about future utility: The
value of the state variable r
t1
depends on some random source, denoted by
t
. Think
of this
t
as uncertain TFP or uncertain returns to investment.
The optimal program is dened in analogy to (3.9) by
\ (r
t
.
t
) = max
fc
:
g
l
t
subject to r
t1
= , (r
t,
c
t
.
t
) .
The additional term in the value function is
t
. It is useful to treat
t
explicitly as a
state variable for reasons we will soon see.
In order to solve the maximization problem, we again follow the three step scheme.
The Bellman equation is
\ (r
t
.
t
) = max
c
I
n(c
t
) +,1
t
\ (r
t1
.
t1
) .
148
It again exploits the fact that the objective function (8.1) is additively separable,
despite the expectations operator, assumes optimal behaviour as of tommorrow and
shifts the expectations operator behind instantaneous utility of today as n(c
t
) is
certain given that c
t
is a control variable. The rst-order condition is
n
0
(c
t
) +,1
t
\
a
I+1
(r
t1
.
t1
)
Jr
t1
Jc
t
= n
0
(c
t
) +,1
t
\
a
I+1
(r
t1
.
t1
)
J, (.)
Jc
t
= 0. (8.3)
It corresponds to the rst-order condition (3.12) in a deterministic setup. The only
dierence consists in the expectations operator 1
t
. Equation (8.3) provides again
an (implicit) functional relationship between consumption and the state variable,
c
t
= c
t
(r
t
.
t
) .
The derivative of the maximized Bellman equation with respect to the state vari-
able is (using the envelope theorem)
\
a
I
(r
t
.
t
) = ,1
t
\
a
I+1
(r
t1
.
t1
)
Jr
t1
Jr
t
.
Observe that r
t1
by the constraint (8.2) is given by , (r
t
. c
t
.
t
) . i.e. by quantities
that are known in t. Hence, the derivative Jr
t1
,Jr
t
= ,
a
I
is non-stochastic and we
can write this expression as (note the similarity to (3.13))
\
a
I
(r
t
.
t
) = ,,
a
I
1
t
\
a
I+1
(r
t1
.
t1
) . (8.4)
This step makes clear why it is useful to include
t
as a state variable into the ar-
guments of the value function. If we had not done so, one could get the impression,
for the same argument that r
t1
is known in t due to (8.2), that the shadow price
\
a
I
(r
t1
.
t1
) is non-stochastic as well and the expectations operator would not be
needed. Given that the value of the optimal program depends on
t
. however, it is
clear that \
0
(r
t1
.
t1
) is random in t indeed. (Some of the subsequent applications
will treat
t
as an implicit state variable and not always be as explicit as here.)
Using that J, (.) ,Jc
t
= ,
c
I
is non-random in t. we can rewrite the rst-order con-
dition as n
0
(c
t
) + ,,
c
I
1
t
\
a
I+1
((r
t1
.
t1
)) = 0. Inserting it in (8.4) gives \
a
I
(r
t
.
t
) =
)
i
I
)
c
I
n
0
(c
t
) . Shifting this expression by one period yields \
a
I+1
(r
t1
.
t1
) =
)
i
I+1
)
c
I+1
n
0
(c
t1
) .
Inserting this into the costate equation (8.4) again, we obtain
n
0
(c
t
) = ,1
t
,
c
I
,
c
I+1
,
a
I+1
n
0
(c
t1
) .
149
8.2 Household utility maximization
Let us now look at a rst example - a household that maximizes utility. The objective
function is given by (8.1),
l
t
= 1
t
1
t=t
,
tt
n(c
t
) .
It is maximized subject to a budget constraint of which we know from (??) or (3.33)
that it ts well into a general equilibrium setup,
c
t1
= (1 +:
t
) c
t
+n
t
j
t
c
t
.
Note that the budget constraint must hold after realization of random variables, not
in expected terms. From the perspective of t. all prices (:
t
, n
t
, j
t
) in t are known,
prices in t + 1 are uncertain.
Having understood in the previous general chapter that uncertainty can explicitly
be treated in the form of a state variable, we limit our attention to the endogenous
state variable c
t
here. It will turn out that this keeps notation simpler. The value of
optimal behaviour is therefore expressed by \ (c
t
) and the Bellman equation can be
written as
\ (c
t
) = max
c
I
n(c
t
) +,1
t
\ (c
t1
) .
The rst-order condition for consumption is
n
0
(c
t
) +,1
t
\
0
(c
t1
)
dc
t1
dc
t
= n
0
(c
t
) ,1
t
\
0
(c
t1
) j
t
= 0.
It can be written as
n
0
(c
t
) = ,j
t
1
t
\
0
(c
t1
) (8.5)
as the price j
t
is known in t.
Dierentiating the maximized Bellman equation gives (using the envelope theo-
rem)
\
0
(c
t
) = ,1
t
\
0
(c
t1
)
dc
t1
dc
t
=\
0
(c
t
) = , [1 +:
t
] 1
t
\
0
(c
t1
) . (8.6)
Again, the term [1 +:
t
] was put in front of the expectations operator as :
t
is
known in t. This dierence equation describes the evolution of the shadow price of
wealth in the case of optimal consumption choices.
150
Inserting the rst-order condition (8.5) gives \
0
(c
t
) = [1 +:
t
]
&
0
(c
I
)
j
I
. Inserting this
expression twice into the dierentiated maximized Bellman equation (8.6) gives a nice
Euler equation,
[1 +:
t
]
n
0
(c
t
)
j
t
= , [1 +:
t
] 1
t
[1 +:
t1
]
n
0
(c
t1
)
j
t1
=
n
0
(c
t
)
j
t
= 1
t
,n
0
(c
t1
)
(1 +:
t1
)
1
j
t1
.
(8.7)
Rewriting it as we did before with (7.40), we get
1
t
_
,n
0
_
c
i
t1
_
n
0
(c
i
t
)
j
t
(1 +:
t1
)
1
j
t1
_
= 1
which allows to given the same interpretation as the rst-order condition in a two-
period model, both deterministic (as in eq. (2.9) in ch. 2.2.2) and stochastic (as in
eq. (7.38) in ch. 7.6) and as in deterministic innite horizon models (as in eq. (3.5) in
ch. 3.2): Relative marginal utility must be equal to relative marginal prices - taking
into account that marginal utility in t + 1 is discounted at , and the price in t + 1 is
discounted by using the interest rate.
Using two further assumption, the expression in (8.7) can be rewritten such that
we come even closer to the deterministic Euler equations: First, let us choose the
output good as numeraire and thereby set prices j
t
= j
t1
= j constant. This allows
us to remove j
t
and j
t1
from (8.7). Second, we assume that the interest rate is
known (say we are in a small open economy with international capital ows). Hence,
expectations are formed only with respect to consumption in t +1. Taking these two
aspects into account, we can write (8.7) as
n
0
(c
t
)
,1
t
n
0
(c
t1
)
=
1
1, (1 +:
t1
)
. (8.8)
This is now as close as possible to (2.9) and (3.5). The ratio of (expected discounted)
marginal utilities is identical to the ratio of relative prices.
8.3 A central planner in an oil-importing economy
Consider another economy where output is produced with oil,
1
t
=
t
1
c
t
C
o
t
1
1co
t
Again, total factor productivity
t
is stochastic. Now, the price of oil,
t
. is stochastic
as well. Let capital evolve according to
1
t1
= (1 o) 1
t
+1
t
t
C
t
C
t
which is a trade balance and good market clearing condition, all in one. The central
planner maximizes
max
fC
:
,O
:
g
1
t
1
t=t
,
tt
n(C
t
)
by choosing a path of aggregate consumption ows C
t
.and oil consumption C
t
. At
t, all variables indexed t are known. The only uncertainty consists about
t1
and
about the price of oil in future periods.
151
The Bellman equation reads \ (1
t
) = max
C
I
,O
I
n(C
t
) +,1
t
\ (1
t1
) . The rst-
order condition for consumption is
n
0
(C
t
) +,1
t
d\ (1
t1
)
d1
t1
[1] = 0 =n
0
(C
t
) = ,1
t
\
0
(1
t1
) (8.9)
For oil it reads
,1
t
\
0
(1
t1
)
J
JC
t
[1
t
t
C
t
] = 0 =
J1
t
JC
t
=
t
.
The derivative of the Bellman equation with respect to the capital stock 1
t
gives
(using the envelope theorem)
\
0
(1
t
) = ,1
t
J
J1
t
\ (1
t1
) = ,
_
1 o +
J1
t
J1
t
_
1
t
\
0
(1
t1
) (8.10)
just as in the economy without oil. The term
_
1 o +
0Y
I
01
I
_
can be pulled out of the
expectation operator as
t
and thereby
0Y
I
01
I
is known at the moment of the savings
decision.
Following the same steps as above without oil, we again end up with
n
0
(C
t
) = ,1
t
n
0
(C
t1
)
_
1 o +
J1
t1
J1
t1
_
The crucial dierence is that now expectations are formed with respect to technolog-
ical uncertainty and with respect to uncertainty about the price of oil.
8.4 Asset pricing in a one-asset economy
This section takes a simple stochastic model with one asset, physical capital. It then
derives an equation that expresses the price of capital in terms of income streams
from holding capital. In order to be as explicit as possible about the nature of this
(real) capital price, we do not choose a numeriare good in this section.
152
8.4.1 The model
Technologies
The technology used to produce a homogenous output good is of the simple Cobb-
Douglas form
1
t
=
t
1
c
t
1
1c
. (8.11)
where TFP
t
is stochastic. Labour supply 1 is exogenous and x, the capital stock
is denoted by 1
t
.
Households
Preferences of households are standard and given by
l
t
= 1
t
1
t=t
,
tt
n(c
t
) .
We start from a budget constraint that can be derived as the budget constraint
(2.42) in a deterministic world. Wealth is held in units of capital 1
t
where the price
of one unit is
t
. When we dene the interest rate as
:
t
=
n
1
t
t
o (8.12)
the budget constraint (2.42) reads
/
t1
= (1 +:
t
) /
t
+
n
t
t

j
t
t
c
t
. (8.13)
Goods market
Investment and consumption goods are traded on the same goods market. Total
supply is given by 1
t
. demand is given by gross investment 1
t1
1
t
+ o1
t
and
consumption C
t
. Expressed in a well-known way, goods market equilibrium yields
the resource constraint of the economy,
1
t1
= 1
t
+1
t
C
t
o1
t
. (8.14)
8.4.2 Optimal behaviour
Firms maximize instantaneous prots which implies rst-order conditions
n
t
= j
t
J1
t
,J1. n
1
t
= j
t
J1
t
,J1
t
. (8.15)
Factor rewards are give by their value marginal products.
Given the preferences of households and the contraint in (8.13), optimal behaviour
of households is described by (this follows identical steps as e.g. in ch. 8.2 and is
treated in ex. 2),
n
0
(C
t
)
j
t
,
t
= 1
t
,n
0
(C
t1
)
(1 +:
t1
)
1
j
t1
,
t1
. (8.16)
This is the standard Euler equation extended for prices, given that we have not chosen
a numeraire. We replaced c
t
by C
t
to indicate that this is the evolution of aggregate
(and not individual) consumption.
153
8.4.3 The pricing relationship
Let us now turn to the main objective of this section and derive an expression for the
real price of one unit of capital, i.e. the price of capital in units of the consumption
good. Starting from the Euler equation (8.16), we insert the interest rate (8.12) in
its general formulation, i.e. including all prices, and rearrange to nd
n
0
(C
t
)

t
j
t
= ,1
t
n
0
(C
t1
)
_
1 +
n
1
t1
t1
o
_
t1
j
t1
= ,1
t
n
0
(C
t1
)
_
(1 o)

t1
j
t1
+
J1
t1
J1
t1
_
.
(8.17)
Now dene a discount factor
t1
= n
0
(C
t1
) ,n
0
(C
t
) and d
t
as the net dividend
payments, i.e. payments to the owner of one unit of capital. Net dividend payments
per unit of capital amount to the marginal product of capital J1
t
,J1
t
minus the
share o of capital that depreciates - that goes kaputt - each period times the real
price
t
,j
t
of one unit of capital, d
t
=
0Y
I
01
I
o
I
j
I
. Inserting this yields
t
j
t
= ,1
t
t1
_
d
t1
+

t1
j
t1
_
. (8.18)
Note that all variables uncertain from the perspective of today in t appear behind
the expectations operator.
Now assume for a second we were in a deterministic world and the economy is in
a steady state. Equation (8.18) could then be written with
t1
= 1 and without the
expectations operator as

I
j
I
= ,
_
d
t1
+

I+1
j
I+1
_
. Solving this linear dierential equation
forward, starting in
t
and inserting repeatedly gives
t
j
t
= ,
_
d
t1
+,
_
d
t2
+,
_
d
tS
+,
_
d
t1
+

t1
j
t1
____
= ,d
t1
+,
2
d
t2
+,
S
d
tS
+,
1
d
t1
+,
1
t1
j
t1
.
Continuing to insert, one eventually and obviously ends up with
t
j
t
=
T
c=1
,
c
d
tc
+,
T

tT
j
tT
.
The price
t
of a unit of capital is equal to the discounted sum of future dividend pay-
ments plus its discounted price (once sold) in t+1. In an innite horizon perspective,
this becomes
t
j
t
=
1
c=1
,
c
d
tc
+ lim
T!1
,
T

tT
j
tT
.
In our stochastic setup, we can proceed according to the same principles as in
the deterministic world but need to take the expectations operator and the discount
factor
t
into account. We replace

I+1
j
I+1
in (8.18) by ,1
t1
t2
_
d
t2
+

I+2
j
I+2
_
and then
154
t2
,j
t2
and so on to nd
t
j
t
= ,1
t
t1
_
d
t1
+,1
t1
t2
_
d
t2
+,1
t2
tS
_
d
tS
+

tS
j
tS
___
= 1
t
_
,
t1
d
t1
+,
2
t1
t2
d
t2
+,
S
t1
t2
tS
d
tS
+,
S
t1
t2
tS
tS
j
tS
_
= 1
t
S
c=1
c
d
tc
+1
t
tS
j
tS
. (8.19)
where we dened the discount factor to be
c
= ,
c
c
a=1
ta
= ,
c
c
a=1
n
0
(C
ta
)
n
0
(C
ta1
)
= ,
c
n
0
(C
tc
)
n
0
(C
t
)
.
The discount factor adjusts discounting by the preference parameter ,. by relative
marginal consumption and by prices. Obviously, (8.19) implies for larger time hori-
zons

I
j
I
= 1
t
T
c=1
c
d
tc
+1
t
tT
. Again, with an innite horizon, this reads
t
j
t
= 1
t
1
c=1
c
d
tc
+ lim
T!1
1
t
tT
. (8.20)
The real price
t
,j
t
amounts to the discounted sum of future dividend payments d
tc
.
The discount factor is
c
which contains marginal utilities, relative prices and the
individuals discount factor ,. The term lim
T!1
1
t
tT
is a bubble term for the
price of capital and can usually be set equal to zero. As the derivation has shown, the
expression for the price
t
,j
t
is simply a rewritten version of the Euler equation.
8.4.4 More real results
The price of capital again
The result on the determinants of the price of capital is useful for economic in-
tuition and received a lot of attention in the literature. But can we say more about
the real price of capital? The answer is yes and it comes from the resource constraint
(8.14). This constraint can be understood as a goods market clearing condition. The
supply of goods 1
t
equals demand resulting from gross investment 1
t1
1
t
+ o1
t
and consumption. The price of one unit of the capital good therefore equals the price
of one unit of the consumption and the output good, provided that investment takes
place, i.e. 1
t
0. Hence,
t
= j
t.
. (8.21)
The real price of capital
t
,j
t
is just equal to one. Not surprisingly, after all, capital
goods and consumption goods are traded on the same market.
The evolution of consumption (and capital)
155
When we want to understand what this model tells us about the evolution of
consumption, we can look at a modied version of (8.16) by inserting the interest
rate (8.12) with the marginal product of capital from (8.15) and the price expression
(8.21),
n
0
(C
t
) = ,1
t
n
0
(C
t1
)
_
1 +
J1
t1
J1
t1
o
_
.
This is the standard Euler equation (see e.g. (8.8)) that predicts how real consumption
evolves over time, given the real interest rate and the discount factor ,.
Together with (8.14), we have a system in two equations that determine C
t
and
1
t
(given appropriate boundary conditions). The price j
t
and thereby the value
t
can not be determined (which is of course a consequence of Walras law). The relative
price is trivially unity from (8.21),
t
,j
t
= 1. Hence, the predictions concerning real
variables do not change when a numeraire good is not chosen.
9
An endowment economy
Consider an individual that can save in one asset and whose budget constraint is
given by (8.13). Let this household behave optimally such that optimal consumption
follows (8.17).
Now change the capital accumulation equation (8.14) such that - for whatever
reasons - 1 is constant and let also, for simplicity, depreciation be zero, o = 0. Then,
output is given according to (8.11) by 1
t
=
t
1
c
1
1c
. i.e. it follows some exogenous
stochastic process, depending on the realization of
t
. This is the exogenous endow-
ment of the economy for each period t. Further, consumption equals output in each
period, C
t
= 1
t
.
Inserting output into the Euler equation (8.17) gives
n
0
(1
t
)

t
j
t
= ,1
t
n
0
(1
t1
)
_
(1 o)

t1
j
t1
+
J1
t1
J1
t1
_
The equation shows that in an endowment economy where consumption is exoge-
nously given at each point in time and households save by holding capital (which
is constant on the aggregate level), the price
t
,j
t
of the asset changes over time
such that households want to consume optimally the exogenously given amount 1
t
.
This equation provides a description of the evolution of the price of the asset in an
endowment economy. These aspects were analysed e.g. by Lucas (1978) and many
others.
9
This statement is not as obvious as it might appear from remembering Walras law from under-
graduate micro courses. There is a literature that analyses the eects of the choice of numeraire for
real outcomes of the economy when there is imperfect competition. See e.g. Gabszewicz and Vial
(1972) or Dierker and Grodahl (1995).
156
8.5 Asset pricing with many assets (*)
8.5.1 A many-sector economy
Consider an economy consisiting of many sectors i. Each sector produces a good 1 (i)
using capital and labour,
1
t
(i) = (i) 1 (i)
c
1(i)
1o
.
Let TFP (i) be uncertain. Capital is sector specic and pays a sector specic return
:
t
(i) =

t1
(i)
t
(i) +
0Y
I
(i)
01
I
(i)
t
(i)
o. (8.22)
This return takes physical depreciation into account.
8.5.2 The households maximization problem (*)
Consider a household that has an investment and a consumption problem. This is a
typical CAP setup. She maximizes
max
fC
I
,0
I
(i)g
1
t
1
t=0
,
t
n(C
t
)
subject to
c
t1
= (1 +:
t
) c
t
+n
t
j
t
c
t
(8.23)
where wealth is given by c
t
=
a
i=1
it
/
it
. the interest rate is is the weighted sum of
asset i specic interest rates,
:
t
=
a
i=1
o
it
:
it
. (8.24)
The weights o
it
=
it
/
it
,c
t
are given by the share of wealth held in asset i at t. They
sum to unity,
a
i=1
o
it
= 1 =o
t
(1) = 1
a
i=2
o
it
.
In order to solve this maximization problem, we follow the three dynamic program-
ming steps.
The Bellman equation reads
\ (c
t
) = n(c
t
) +,1
t
\ (c
t1
) (8.25)
and rst-order conditions are for consumption
n
0
(c
t
) = ,1
t
\
0
(c
t1
) j
t
(8.26)
and for shares o
t
(i)
1
t
\
0
(c
t1
) c
t
d:
t
do
t
(i)
= 0 =1
t
\
0
(c
t1
) [:
t
(1) :
t
(i)] = 0 (8.27)
In order to get an expression that allows us to link the value of an asset to future
dividend payments as in (8.16), we rewrite the foc as follows.
157
First compute the derivative of the maximized Bellman equation with respect to
assets c
t
. With the envelope theorem, this gives
\
0
(c
t
) = ,1
t
\
0
(c
t1
) (1 +:
t
) = ,1
t
\
0
(c
t1
) +,1
t
\
0
(c
t1
) :
t
= ,1
t
\
0
(c
t1
) +,
o
t
(i) 1
t
\
0
(c
t1
) :
t
(i) (8.28)
As (8.27) is identical to
1
t
\
0
(c
t1
) :
t
(1) = 1
t
\
0
(c
t1
) :
t
(i) (8.29)
we can write (8.28) as
\
0
(c
t
) = ,1
t
\
0
(c
t1
) +,
a
i=1
o
t
(i) 1
t
\
0
(c
t1
) :
t
(1)
= ,1
t
\
0
(c
t1
) +,1
t
\
0
(c
t1
) :
t
(1) (8.30)
where the last step used the fact that only o
t
(i) depends on i and

o
t
(i) = 1. As
this last step can be undertaken with any sector i, we have
\
0
(c
t
) = ,1
t
\
0
(c
t1
) (1 +:
t
(i))
and also
1
t
\
0
(c
t1
) (1 +:
t
(i)) = 1
t
\
0
(c
t1
) (1 +:
t
(1)) . (8.31)
Hence, with the derivative of Bellman equation we obtained this alternative version
(8.31) for the rst-order condition (8.27) for shares.
Now lets assume, there is one riskless asset, and call this asset asset number 1.
Then (8.30) reads
\
0
(c
t
) = (1 +:
t
(1)) ,1
t
\
0
(c
t1
) (8.32)
where the interest rate was pulled in front of the expectation operator as :
t
(1) is
certain. This is the evolution of the shadow price
0
(c
t
) which we need (in addition
to (8.31)) in what follows.
With (8.26) and (8.32) we then get a relationship between marginal utility from
consumption and the marginal value of wealth,
10
\
0
(c
t
) = (1 + :
t
(1))
n
0
(c
t
)
j
t
. (8.33)
10
Without this relationship, a nice Euler equation as (8.34) could not be derived. Hence, the
assumption of a risk less asset is crucial.
158
Inserting this twice into (8.32) yields
(1 +:
t
(1))
n
0
(c
t
)
j
t
= (1 +:
t
(1)) ,1
t
(1 +:
t1
(1))
n
0
(c
t1
)
j
t1
=
n
0
(c
t
)
j
t
= 1
t
,n
0
(c
t1
)
(1 +:
t
(1))
1
j
t1
(8.34)
With the usual interpretation (compare (8.7) in ch. 8.2), this is the Euler equation
for savings in the risky asset.
The rst-order condition (8.31) for assets becomes with (8.33)
1
t
n
0
(c
t1
)
j
t1
(1 +:
t
(i)) = 1
t
n
0
(c
t1
)
j
t1
(1 +:
t
(1)) (8.35)
where we used again that :
t
(1) is certain t. This expression can also be written as
1
t
,n
0
(c
t1
)
(1 +:
t1
(1))
1
j
t1
= 1
t
,n
0
(c
t1
)
(1 +:
t1
(i))
1
j
t1
.
The Euler equation (8.34) can therefore alternatively be written as
n
0
(c
t
)
j
t
= 1
t
,n
0
(c
t1
)
(1 +:
t1
(i))
1
j
t1
(8.36)
This is the Euler equation expressing returns in terms of the risky asset i. Comparing
(8.34) and (8.36) shows the indierence between marginally saving in asset i or the
risken asset.
8.5.3 The fundamental pricing rule
Remember the denition of the interest rate in (8.22). Insert in (8.36) and nd
n
0
(c
t
)
j
t
= ,1
t
(1 +:
t1
(i))
n
0
(c
t1
)
j
t1
= ,1
t
n
1
t
+
t1
t
n
0
(c
t1
)
j
t1
where n
1
t
=
0Y
I
01
I
o
t
are dened as divided payments. Multiply by
t
(which is
known at t), divide by n
0
(c
t
) and nd
t
= ,1
t
_
_
n
1
t
+
t1
n
0
(c
t1
) ,j
t1
n
0
(c
t
) ,j
t
_
.
Now express this equation for the next period, i.e. replace t by t + 1 and get
t1
=
,1
t1
_
n
1
t1
+
t2
_
&
0
(c
I+2
)j
I+2
&
0
(c
I+1
)j
I+1
. Inserting this yields
t
= ,1
t
__
n
1
t
+,1
t1
_
_
n
1
t1
+
t2
n
0
(c
t2
) ,j
t2
n
0
(c
t1
) ,j
t1
__
n
0
(c
t1
) ,j
t1
n
0
(c
t
) ,j
t
_
= 1
t
1
t1
_
,n
1
t
&
0
(c
I+1
)j
I+1
&
0
(c
I
)j
I
+,
2
n
1
t1
&
0
(c
I+2
)j
I+2
&
0
(c
I+1
)j
I+1
&
0
(c
I+1
)j
I+1
&
0
(c
I
)j
I
+,
2
t2
&
0
(c
I+2
)j
I+2
&
0
(c
I+1
)j
I+1
&
0
(c
I+1
)j
I+1
&
0
(c
I
)j
I
_
.
159
Noting that 1
t
1
t1
= 1
t
and inserting very often, one obtains
t
= 1
t
ta
c=t
_
c
)=t
,
n
0
(c
)1
) ,j
)1
n
0
(c
)
) ,j
)
_
n
1
c
+
ta
_
c
)=t
,
n
0
(c
)1
) ,j
)1
n
0
(c
)
) ,j
)
_
.
Hence, the value of a rm is the discounted sum of future divided payments plus
a bubble term. The interesting aspect is that the discount factor
c
)=t
,
&
0
(c
+1
)j
+1
&
0
(c
)j
depends on preferences of households.

8.5.4 The CAP formula
The well-known CAP formula relates excess returns to volatility measures. This
relationship can be obtained here as well by starting from the rst-order condition
(8.27) and following steps similar to those undertaken further below in the continuous
time model in ch. 10.4.3.
8.6 Matching and unemployment
Consider a rm that can change the number of employees only slowly due to searching
and matching on the labour market.
- coming soon -
8.7 How to do without dynamic programming
8.7.1 Individual utility maximization
We will now get to know a method that allows us to solve stochastic intertemporal
problems in discrete time without dynamic programming. One could ask oneself after
this chapter why dynamic programming exists at all ...
The objective is again max
fc
I
g
1
0
1
t=0
,
t
n(c
t
) subject to the constraint c
t1
=
(1 +:
t
) c
t
+n
t
c
t
. Again, the control variables of the household are c
t
. the state
variable is c
t
. the interest rate :
t
and the wage n
t
are exogenously given. Now rewrite
the objective function and insert the constraint twice,
1
0
_
c1
t=0
,
t
n(c
t
) +,
c
n(c
c
) +,
c1
n(c
c1
) +
1
t=c2
,
t
n(c
t
)
_
= 1
0
c1
t=0
,
t
n(c
t
) +1
0
,
c
n((1 + :
c
)c
c
+n
c
c
c1
)
+1
0
,
c1
((1 + :
c1
)c
c1
+n
c1
c
c2
) +1
0
1
t=c2
,
t
n(c
t
) .
Note that the expectations operator always refers to knowledge available at the be-
ginning of the planning horizon, i.e. to t = 0.
Now compute the rst-order condition with respect to c
c1
. This is unusual as we
directly choose the state variable which is usually understood to be indirectly chosen
by the control variable. Clearly, however, this is just a convenient trick and we really,
by choosing c
c1
. a state variable, we choose c
c
. The derivative with respect to c
c1
yields
1
0
,
c
n
0
(c
c
) = 1
0
,
c1
n
0
(c
c1
) (1 +:
c1
).
160
This almost looks like the standard optimal consumption rule. The dierence consists
in the expectations operator being present on both sides. This is not surprising as we
optimally chose c
c1
(i.e. c
c
), knowing only the state of the system in t = 0. If we now
assume we are in :. our expectations would be based on knowledge in : and we could
replace 1
0
by 1
c
. We would then obtain 1
c
,
c
n
0
(c
c
) = ,
c
n
0
(c
c
) for the left-hand side
and our optimality rule reads
n
0
(c
c
) = ,1
c
n
0
(c
c1
) (1 +:
c1
).
This is the rule we know from Bellman approaches, provided e.g. in (8.7).
8.7.2 Contingent claims pricing
Let us now ask how assets would be priced that pay a return only in 1 periods.
Consider an economy with an asset that pays a return : in each period and one long-
term asset which can be sold only after 1 periods. Assuming that investors behave
rationally, i.e. they maximize an intertemporal utility function subject to constraints,
the price of the long-term asset can be most easily found by using a Lagrange approach
or by straightforward inserting.
We assume that an investor maximizes her expected utility 1
0
T
t=0
,
t
n(c
t
) subject
to the constraints
c
0
+:
0
j
0
+c
0
= n
0
c
t
+c
t
= (1 +:)c
t1
+n
t
. 1 _ t _ 1 1.
c
T
= :
0
j
T
+ (1 + :) c
T1
+n
T
.
In period 0. the individual uses labour income n
0
to pay for consumption goods c
0
. to
buy :
0
units of the long-term asset and for normal assets c
0
. In periods 1 to 1 1.
the individual uses her assets c
t1
plus returns : on assets and her wage income n
t
to
nance consumption and buy again assets c
t
(or keep those of the previous period).
In the nal period 1. the long-term asset has a price of j
T
and is sold. Wealth of the
previous period plus interest plus labour income n
T
are further sources of income to
pay for consumption c
T
.
This maximization problem can most easily be solved by inserting consumption
levels for each period into the objective function. The objective function then reads
n(n
0
:
0
j
0
c
0
) +1
0
T1
t=1
,
t
n(n
t
+ (1 + :)c
t1
c
t
)
+1
0
,
T
n(:
0
j
T
+ (1 + :)c
T1
+n
T
)
and the control variables are wealth holdings c
t
in periods t = 0. .... 1 1 and the
number of assets :
0
bought in period zero.
Let us now look at the rst-order conditions. The rst-order condition for wealth
in period zero is
n
0
(c
0
) = (1 + :) ,1
0
n
0
(c
1
) . (8.37)
Wealth holdings in any period t 0 are optimally chosen according to
1
0
,
t
n
0
(c
t
) = 1
0
,
t1
(1 +:)n
0
(c
t1
) =1
0
n
0
(c
t
) = , (1 +:) 1
0
n
0
(c
t1
) . (8.38)
161
We can insert (8.38) into the rst-period condition (8.37) suciently often and nd
n
0
(c
0
) = (1 +:)
2
,
2
1
0
n
0
(c
2
) = ... = (1 +:)
T
,
T
1
0
n
0
(c
T
) (8.39)
The rst-order condition for the number of assets is
j
0
n
0
(c
0
) = ,
T
1
0
n
0
(c
T
)j
T
. (8.40)
When we insert combined rst-order conditions (8.39) for wealth holdings into the
rst-order condition (8.40) for assets, we obtain
j
0
(1 +:)
T
,
T
1
0
n
0
(c
T
) = ,
T
1
0
n
0
(c
T
)j
T
=
j
0
= (1 +:)
T
1
0
n
0
(c
T
)
1
0
n
0
(c
T
)
j
T
.
which is an equation where we nicely see the analogy to the two period example in
ch. 7.7.1. Instead of j
)0
=
1
1v
1
&
0
(c
1
)
1&
0
(c
1
)
j
)1
in (7.46) where we discount by one period
only and evaluate returns at expected marginal utility in period 1, we discount by 1
periods and evaluate returns at marginal utility in period 1.
This equation also oers a lesson for life when we assume risk-neutrality for sim-
plicity: if the payo j
T
of a long-term asset is not high enough such that the current
price is higher than the present value of the payo, j
0
(1 +:)
T
j
T
, then the long-
term asset is simply dominated by short-term investments that pay a return of :
per period. Optimal behaviour would imply not to buy the long-term asset and just
put wealth in normal assets. This should be kept in mind when talking next time
to your insurance agent who tries to sell you life-insurance or private pension plans.
Just ask for the present value of the payos and compare them to the present value
of what you pay into the savings plan.
8.7.3 Sticky prices (*)
The setup
Sticky prices are a fact of life. In macro economic models, they are either assumed
or result from some adjustment-cost setup. Here is a simplied way to derive sluggish
price adjustment. This example is inspired by Ireland (2004 NBER WP), going back
to Rotemberg (1982 JPE).
The objective function of the rm is to maximize its present value dened by the
sum of discounted expected prots,
\
t
= 1
t
1
t=t
_
1
1 +:
_
tt
:
t
.
Prots at a point t _ t in time are given by
:
t
= j
t
r
t
n
t
|
t
(j
t
. j
t1
)
162
where (j
t
. j
t1
) are price adjustment costs. These are similar in spirit to the
adjustment costs presented in ch. 5.5.1. We will later use a specication given by
(j
t
. j
t1
) =
c
2
_
j
t
j
t1
j
t1
_
2
. (8.41)
This specication only captures the essential mechanism that is required to make
prices sticky, i.e. the fact that the price change is squared (in fact, as with all
adjustment cost mechanisms, any power larger than 1 should do the job). More care
about economic implications needs to be taken when a reasonable model is to be
specied.
The rm uses a technology
r
t
=
t
|
t
.
We assume that there is a certain demand elasticity for the rms output. This
can reect a monopolistic competition setup. The rm can choose its output r
t
at
each point in time freely by hiring the corresponding amount of labour |
t
. Labour
productivity
t
or other quantities can be uncertain.
A solution
Inserting everything into the objective function yields
\
t
= 1
t
1
t=t
_
1
1 +:
_
tt
_
j
t
r
t

n
t
t
r
t
(j
t
. j
t1
)
_
= 1
t
_
j
t
r
t
n
t
t
r
t
(j
t
. j
t1
)
_
+1
t
_
j
t1
r
t1
n
t1
t1
r
t1
(j
t1
. j
t
)
_
+1
t
1
t=t2
_
1
1 +:
_
tt
_
j
t
r
t

n
t
t
r
t
(j
t
. j
t1
)
_
.
The second and third line present a rewritten way that allows to better see the in-
tertemporal structure of the maximization problem. We now maximize this objective
function by choosing output r
t
for today. (Output levels in the future are chosen at
a later stage.)
The rst-order condition is
1
t
_
d [j
t
r
t
]
dr
t

n
t
t

d(j
t
. j
t1
)
dr
t
_
1
t
d(j
t1
. j
t
)
dr
t
= 0 =
d [j
t
r
t
]
dr
t
=
n
t
t
+
d(j
t
. j
t1
)
dr
t
+1
t
d(j
t1
. j
t
)
dr
t
.
It has certain well-known components and some new ones. If there were no adjustment
costs, i.e. (.) = 0. the intertemporal problem would become a static one and the
usual condition would equate marginal revenue d [j
t
r
t
] ,dr
t
with marginal cost n
t
,
t
.
With adjustment cost, however, a change in output today not only aects adjustment
cost today
o4(j
I
,j
I1
)
oa
I
but also (expected) adjustment cost 1
t
o4(j
I+1
,j
I
)
oa
I
tomorrow. As
163
all variables with index t are assumed to be known in t. expectations are formed only
with respect to adjustment cost tomorrow in t + 1.
Specifying the adjustment cost function as in (8.41) and computing marginal rev-
enue using the demand elasticity
t
gives
d [j
t
r
t
]
dr
t
=
n
t
t
+
d
dr
t
_
c
2
_
j
t
j
t1
j
t1
_
2
_
+1
t
d
dr
t
_
c
2
_
j
t1
j
t
j
t
_
2
_
=
dj
t
dr
t
r
t
+j
t
=
_
1 +j
1
t
_
j
t
=
n
t
t
+c
_
j
t
j
t1
j
t1
_
1
j
t1
dj
t
dr
t
+1
t
_
c
_
j
t1
j
t
j
t
_
d
dr
t
j
t1
j
t
_
=
n
t
t
+c
_
j
t
j
t1
j
t1
_
j
t
r
t
j
t1
1
t
1
t
_
c
_
j
t1
j
t
j
t
_
j
t1
j
2
t
dj
t
dr
t
_
=
_
1 +j
1
t
_
j
t
=
n
t
t
+c
_
j
t
j
t1
j
t1
_
j
t
r
t
j
t1
1
t
1
t
_
c
_
j
t1
j
t
j
t
_
j
t1
j
2
t
j
t
r
t
1
t
_
=
_
1 +j
1
t
_
j
t
=
n
t
t
+c
_
j
t
j
t1
j
t1
_
j
t
r
t
j
t1
1
t

1
j
t
r
t
1
t
c1
t
_
j
t1
j
t
j
t
j
t1
_
where
t
=
oa
I
oj
I
j
I
a
I
is the demand elasticity for the rms good. Again, c = 0 would
give the standard static optimality condition
_
1 +j
1
t
_
j
t
= n
t
,
t
where the price is
a markup over marginal cost. With adjustment cost, prices change only slowly.
8.7.4 Optimal employment with adjustment cost (*)
The setup
Consider a rm that maximizes prots as in ch. 5.5.1,
t
= 1
t
1
t=t
1
(1 +:)
tt
:
t
. (8.42)
We are today in t. time extends until innity and the time between today and innity
is denoted by t. There is no particular reason why the planning horizon is innity in
contrast to ch. 5.5.1. The optimality condition for employment we will stress here is
identical to a nite horizon problem.
Instantaneous prots are given by the dierence between revenue in t. which is
identical to output 1 (1
t
) with an output price normalized to unity, labour cost n
t
1
t
and adjustment cost (1
t
1
t1
) .
:
t
= 1 (1
t
) n
t
1
t
(1
t
1
t1
) . (8.43)
Costs induced by the adjustment of the number of employees between the previous
period and today are captured by (.). Usually, one assumes costs both from hiring
and from ring individuals, i.e. both for an increase in the labour force, 1
t
1
t1

0. and from a decrease, 1
t
1
t1
< 0. A simple functional form for (.) which
captures this idea is a quadratic form, i.e. (1
t
1
t1
) =

2
(1
t
1
t1
)
2
, where c is
a constant.
164
Uncertainty for a rm can come from many sources: Uncertain demand, uncer-
tainty about the production process, uncertainty in labour cost or other sources. As
we express prots in units of the output good and assume that the real wage n
t
,
i.e. the amount of output goods to be paid to labour, is uncertain. Adjustment cost
(1
t
1
t1
) are certain, i.e. the rm knows by how many units of output prots
reduce when emplyoment changes by 1
t
1
t1
.
As in static models of the rm, the control variable of the rm is employment 1
t
.
In contrast to static models, however, employment decisions today in t have not only
eects on employment today but also on employment tommorrow as the employment
decision in t aects adjustment costs in t +1. There is therefore an intertemporal link
the rm needs to take into account which is not present in static models of the rm.
Solution
This maximization problem can be solved directly by inserting (8.43) into the
objective function (8.42). One can then choose optimal employment for some point
in time t _ : < after having split the objective function into several subperiods -
as e.g. in the previous chapter 8.7.1. The solution reads (to be shown in exercise 6)
1
0
(1
t
) = n
t
+
0
(1
t
1
t1
) 1
t
0
(1
t1
1
t
)
1 +:
.
When employment 1
t
is chosen in t. there is only uncertainty about 1
t1
. The current
wage n
t
(and all other deterministic quantities as well) are known with certainty. 1
t1
is uncertain, however, from the perspective of today as the wage in t +1 is unknown
and 1
t1
will be have to be adjusted accordingly in t + 1. Hence, expectations ap-
ply only to the adjustment-cost term which refers to adjustment cost which occur in
period t + 1. Economically speaking, given employment 1
t1
in the previous period,
employment in t is chosen such that marginal productivity of labour equals labour
cost adjusted for current and expected future adjustment cost. Expected future ad-
justment cost are discounted by the interest rate : to obtain its present value.
When we specify the adjustment cost function as a quadratic function, (1
t
1
t1
) =
2
[1
t
1
t1
]
2
. we obtain
1
0
(1
t
) = n
t
+c[1
t
1
t1
] 1
t
c[1
t1
1
t
]
1 +:
.
If there were no adjustment cost, i.e. c = 0. we would have 1
0
(1
t
) = n
t
. Employment
would be chosen such that marginal productivity equals the real wage. This conrms
the initial statement that the intertemporal problem of the rm arises purely from
the adjustment costs. Without adjustment cost, i.e. with c = 0. the rm has the
standard instantaneous, period-specic optimality condition.
8.8 An explicit time path for a boundary condition
Sometimes, an explicit time path for optimal behaviour is required. The transversality
condition is then usually not very useful. A more pragmatic approach sets assets at
165
some future point in time at some exogenous level. This allows then to easily (at least
numerically) compute the optimal path for all points in time before this nal point.
Let 1 be the nal period of life in our model, i.e. set c
T1
= 0 (or some other
level e.g. the deterministic steady state level). Then, from the budget constraint, we
can deduce consumption in 1.
c
T1
= (1 +:
T
) c
T
+n
T
c
T
=c
T
= (1 +:
T
) c
T
+n
T
.
Optimal consumption in 1 1 still needs to obey the Keynes-Ramsey rule, i.e.
n
0
(c
T1
) = 1
T1
, [1 +:
T
] n
0
(c
T
) .
As the budget constraint requires
c
T
= (1 +:
T1
) c
T1
+n
T1
c
T1
.
optimal consumption in 1 1 is determined by
n
0
(c
T1
) = 1
T1
, [1 +:
T
] n
0
((1 + :
T
) [(1 +:
T1
) c
T1
+n
T1
c
T1
] +n
T
)
This is one equation in one unknown, c
T1
, where expectations need to be formed
about :
T
and n
T
and n
T1
are unknown. When we assume a probability distribu-
tion for :
T
and n
T
, we can replace 1
T1
by a summation over states and solve this
expression numerically in a straightforward way.
A recent introduction and detailed analysis of discrete time models with uncertainty
in the RBC tradition with homogenous and heterogenous agents is by Heer and
Mausner (2005). A more rigorous approach than here is taken by Stokey and Lucas
(1989). An almost comprehensive in-depth presentation of macroeconomic aspects
under uncertainty is provided by Ljungqvist and Sargent (2004).
On capital asset pricing in one-sector economies, references include Jermann (1998),
Danthine, Donaldson and Mehra (1992), Abel (1990), Rouwenhorst (1995), Stokey
and Lucas (1989, ch. 16.2) and Lucas (1978). An overview is in ch. 13 of Ljungqvist
and Sargent (2004).
166
Exercises Chapter 8
Discrete time innite horizon models
under uncertainty
1. Central planner
Consider an economy where output is produced by
1
t
=
t
1
c
t
1
1c
t
Again, as in the OLG example in equation (7.13), total factor productivity
t
is stochastic. Let capital evolve according to
1
t1
= (1 o) 1
t
+1
t
C
t
The central planner maximizes
max
fC
:
g
1
t
1
t=t
,
tt
n(C
t
)
by again choosing a path of aggregate consumption ows C
t
. At t, all variables
indexed t are known. The only uncertainty consists about
t1
. What are
optimality conditions?
2. A household maximization problem
Consider the optimal saving problem of the household in ch. 8.4. Derive Euler
equation (8.16).
3. Closed-form solution
Solve this model for the utility function n(C) =
C
1
1
1o
and for o = 1. Solve it
for a more general case (Benhabib and Rustichini, 1994).
4. Habit formation
Assume instantaneous utility depends not only on current consumption but also
on habits (see e.g. Abel, 1990). Let the utility function therefore look like
l
t
= 1
t
1
t=t
,
tt
n(c
t
.
t
) .
where
t
stands for habits like e.g. past consumption,
t
= (c
t1
. c
t2
. ...) .
Let such an individual maximize utility subject to the budget constraint
c
t1
= (1 +:
t
) c
t
+n
t
j
t
c
t
167
(a) Assume the individual lives in a deterministic world and derive a rule for
an optimal consumption path where the eect of habits are explicitly taken
into account. Specify habits by
t
= c
t1
.
(b) Let there be uncertainty with respect to future prices. At a point in time
t, all variables indexed by t are known. What is the optimal consumption
rule when habits are treated in a parametric way?
(c) Choose a plausible instantaneous utility function and discuss the implica-
tions for optimal consumption given habits
t
= c
t1
.
5. Risk-neutral valuation
Under which conditions is there a risk neutral valuation relationship for contin-
gent claims in models with many periods?
6. Labour demand under adjustment cost
Solve the maximization problem of the rm in ch. 8.7.4 by directly inserting
prots (8.43) into the objective function (8.42) and then choosing 1
t
.
7. Solving by inserting
Solve the problem from ch. 8.7 in a slightly extended version, i.e. with prices j
t
.
Maximize 1
0
1
t=0
,
t
n(c
t
) by choosing a time path c
t
for consumption subject
to c
t1
= (1 +:
t
) c
t
+n
t
j
t
c
t
.
8. Matching on labour markets (exam)
Let employment 1
t
in a rm follow
1
t1
= (1 :) 1
t
+j\
t
.
where : is a constant separation rate, j is a constant matching rate and \
t
denotes the number of jobs a rm currently oers. The rms prots :
t
in period
t are given by the dierence between revenue j
t
1 (1
t
) and cost, where cost stem
from wage payments and costs for vacancies \
t
captured by a parameter .
:
t
= j
t
1 (1
t
) n
t
1
t
\
t
.
The objective function of the rm is given by
t
= 1
t
1
t=t
,
tt
:
t
.
where , is a discount factor and 1
t
is the expectations operator.
(a) Assume a deterministic world. Let the rm choose the number of vacancies
optimally. Use a Lagrangian to derive the optimality condition. Assume
that there is an interior solution. Why is this an assumption that might
not always be satised from the perspective of a single rm?
168
(b) Let us now assume that there is uncertain demand which translates into
uncertain prices j
t
which are exogenous to the rm. Solve the optimal
choice of the rm by inserting all equations into the objective function.
Maximize by choosing the state variable and explain also in words what
you do. Give an interpretation of the optimality condition. What does it
imply for the optimal choice of \
t
?
9. Optimal training for a marathon (exam)
Imagine you want to participate in a marathon or any other sports event. It
will take place in : days, i.e. in t + : where t is today. You know that
taking part in this event requires training c
t
. t [t. t +:] . Unfortunately,
you dislike training, i.e. your instantaneous utility n(c
t
) decreases in eort,
n
0
(c
t
) < 0. On the other hand, training allows you to be successful in the
marathon: more eort increases your personal tness 1
t
. Assume that tness
follows 1
t1
= (1 o) 1
t
+ c
t
, with 0 < o < 1. and tness at t + : is good for
you yielding happiness of /(1
tn
) .
(a) Formulate formally an objective function which captures the trade-os in
such a training program.
(b) Assume that everything is deterministic. How would your training schedule
look like (the optimal path of c
t
)?
(c) In the real world, normal night life reduces tness in a random way, i.e. o
is stochastic. How does now your training schedule look like?
169
Part IV
Stochastic models in continuous
time
The choice between working in discrete or continuous time is partly driven by previ-
ous choices: If the literature is mainly in discrete time, students will nd it helpful
to work in discrete time as well. This dominance of discrete time seems to hold
for macroeconomics. On the other hand, there are very good books that can help
as introductions (and go well beyond an introduction) to work with uncertainty in
continuous time. An example is Turnovsky (1997, 2000).
Whatever the tradition in the literature, continuous time models have the huge
advantage that they are analytically generally more tractable, once some initial invest-
ment into new methods is digested. One example is, as some papers in the literature
have shown (e.g. Wlde, 2005 or Posch and Wlde, 2005), that continuous time mod-
els with uncertainty can be analyzed in simple phase diagrams as in deterministic
continuous time setups. In other elds, continuous time uncertainty is dominating,
as e.g. in nance. See ch. 9.5 and 10.6 on further reading for references from many
elds.
This nal part of the book presents tools that allow to work with uncertainty in
continuous time models. It is probably the most innovative part of this book as many
results from recent research directly ow into it.
9 Stochastic dierential equations and rules for
dierentials
When working in continuous time, uncertainty enters the economy usually in the
form of Brownian motion, Poisson processes or Levy processes. This uncertainty
is represented in economic models by stochastic dierential equations (SDEs) which
describe e.g. the evolution of prices or technology frontiers. This section will cover a
wide range of dierential equations (and show how to work with them) that appear
in economics and nance. It will also show how to work with functions of stochastic
variables, e.g. how output evolves given that TFP is stochastic or how wealth of a
household grows over time, given that the price of the asset held by the houshold is
random. The entire treatment here, as before in this book, will be non-rigorous and
focused on how to compute things.
9.1 Some basics
9.1.1 Stochastic processes
We got to know random variables in ch. 7.1. A random variable relates in some
loose sense to a stochastic process how (deterministic) static models relate to (deter-
170
ministic) dynamic models: Static models describe one equilibrium, dynamic models
describe a sequence of equilibria. A random variable has when looked at once (e.g.
when throwing a die once) one realization. A stochastic process describes a sequence
of random variables and therefore when looked at once, describes a sequence of
realizations. More formally, we have the following
Denition 8 (Ross, 1996) A stochastic process is a parameterized collection of ran-
dom variables
A (t)
t2[t
0
,t
1
[
.
Let us look at an example for a stochastic process. We start from the normal
distribution of ch. 7.2.2 whose mean and variance are given by j and o
2
and its
density function is , (.) =
_
_
2:o
2
_
1
c
1
2
(
:
)
2
. Now dene a normally distributed
random variable 2 (t) that has a mean j and a variance that is a function of some t,
i.e. instead of o
2
, write o
2
t. Hence, the random variables we just dened has as density
function , (.) =
_
_
2:o
2
t
_
1
c
1
2
p
I
2
. By having done so and by interpreting t as
time, 2 (t) is in fact a stochastic process: We have a collection of random variables,
all normally distributed, they are parameterized by time t.
Stochastic processes can be stationary, weakly stationary or non-stationary. They
are non-stationary if they are not weakly stationary. Stationarity is a more restrictive
concept than weak stationarity.
Denition 9 (Ross, 1996, ch. 8.8): A process A (t) is stationary if A (t
1
) . .... A (t
a
)
and A (t
1
+:) . .... A (t
a
+:) have the same joint distribution for all : and :.
An implication of this denition, which might help to get some feeling for this
denition, is that a stationary process A (t) implies that, being in t = 0, A (t
1
) and
A (t
2
) have the same distribution for all t
2
t
1
0. A weaker concept of stationarity
only requires the rst two moments of A (t
1
) and A (t
2
) (and a condition on the
covariance) to be satised.
Denition 10 (Ross, 1996) A process A (t) is weakly stationary if the rst two mo-
ments are the same for all t and the covariance between A (t
2
) and A (t
1
) depends
only on t
2
t
1
.
1
0
A (t) = j. \ c:A (t) = o
2
. Co (A (t
2
) . A (t
1
)) = , (t
2
t
1
) .
where j and o
2
are constants and , (.) is some function.
The probably best known stochastic process, especially from nance, is Brownian
motion. It is sometimes called Wiener process after the mathematician Wiener who
provided the following denition.
Denition 11 (Ross, 1996) Brownian motion
A stochastic process . (t) is a Brownian motion process if (i) . (0) = 0. (ii) the process
has stationary independent increments and (iii) for every t 0. . (t) is normally
distributed with mean 0 and variance o
2
t.
171
The rst condition . (0) = 0 is a normalization. Any . (t) that starts at, say, .
0
can be redened as . (t) .
0
. The second condition says that for t
1
t
S
_ t
2
t
1
the increment . (t
1
) . (t
S
) . which is a random variable, is independent of previous
increments, say . (t
2
). (t
1
). Independent increments implies that Brownian motion
is a Markov process: Assuming that we are in t
S
today, the distribution of . (t
1
)
depends only on . (t
S
) . i.e. on the current state, and not on previous states like
. (t
1
). Increments are said to be stationary if, according to the above denition of
stationarity, the stochastic process A (t) = . (t) . (t :) where : is a constant
has the same distribution for any t. The third condition is nally the heart of the
denition - . (t) is normally distributed. The variance increases linearly in time; the
Wiener process is therefore nonstationary.
Let us now dene a stochastic process which plays also a major role in economics.
Denition 12 Poisson process (adapted following Ross 1993, p. 210)
A stochastic process (t) is a Poisson process with arrival rate ` if (i) (0) = 0. (ii)
the process has independent increments and (iii) the increment (t) (t) in any
interval of length t t (the number of jumps) is Poisson distributed with mean
`[t t] . i.e. (t) (t) ~Poisson(`[t t]) .
A Poisson process (and other related processes) are also sometimes called count-
ing processes as (t) counts how often a jump has occured, i.e. how often something
has happend.
There is a close similarity in the rst two points of this denition with the denition
of Brownian motion. The third point here means more precisely that the probability
that the process increases : times between t and t t is given by
1 [ (t) (t) = :] = c
A[tt[
(`[t t])
a
:!
. : = 0. 1. ... (9.1)
We know this probability from the denition of the Poisson distribution in ch. 7.2.1.
This is probably where the Poisson process got its name from. Hence, one could think
of as many stochastic processes as there are distributions, dening each process by
the distribution of its increments.
The most common way how Poisson processes are presented is by looking at the
distribution of the increment (t) (t) over a very small time intervall [t. t] . The
increment (t) (t) for t very close to t is usually expressed by d (t) . A stochastic
process (t) is then a Poisson process if its increment d (t) is driven by
d (t) =
_
0 with prob. 1Aot
1 with prob. Aot
. (9.2)
where the parameter ` is again called the arrival rate. A high ` then means that the
process jumps more often than with a low `.
11
These stochastic processes (and other processes) can now be combined in various
ways to construct more complex processes. These more complex processes can nicely
be represented by stochastic dierential equations (SDEs).
11
Note that the probabilities given in (9.2) are an approximation to the ones in (9.1) for t = dt:
We will return to this in ch. 9.4.2, see Poisson process II.
172
9.1.2 Stochastic dierential equations
The most frequently used SDEs include Brownian motion as the source of uncertainty.
These SDEs are used to model e.g. the evolution of asset prices or budget constraints
of households. Other examples include SDEs with Poisson uncertainty used explicitly
e.g. in the natural volatility literature, in nance, international macro or in other
contexts mentioned above. Finally and more recently, Levy processes are used in
nance as they allow for a much wider choice of properties of distributions of asset
returns than e.g. Brownian motion. We will now get to know examples for each type.
For all Brownian motions that will follow, we will assume, unless explicitly stated
otherwise, that increments have a standard normal distribution, i.e. 1
t
[. (t) . (t)] =
0 and var
t
[. (t) . (t)] = t. It is therefore sucient, consistent with most papers in
the literature and many mathematical textbooks, to work with a normalization of o
in denition 11 of Brownian motion to 1.
Brownian motion with drift
This is one of the simplest SDEs. It reads
dr(t) = cdt +/d. (t) . (9.3)
The constant c can be called drift rate, /
2
is sometimes refered to as the variance rate
of r (t) . In fact, ch. 9.4.4 shows that the expected increase of r (t) is determined by
c only (and not by /). In contrast, the variance of r (t) for some future t t is only
determined by /. The drift rate c is multiplied by dt. a short time intervall, the
variance parameter / is multiplied by d. (t) . the increment of the Brownian motion
process . (t) over a small time intervall. This SDE (and all the others following later)
therefore consists of a deterministic part (the dt-term) and a stochastic part (the
d.-term).
An intuition for this dierential equation can be most easily gained by undertaking
a comparison with a deterministic dierential equation. If we neglected the Wiener
process for a moment (set / = 0), divide by dt and rename the variable to . we obtain
the simple ordinary dierential equation
_ (t) = c (9.4)
whose solution is (t) =
0
+ct. When we draw this solution and also the above SDE
for three dierent realizations of . (t), we obtain the following gure.
173
Figure 38 The solution of the deterministic dierential equation (9.4) and three re-
alizations of the related stochastic dierential equation (9.3) RealizationBM_multi.nb
Hence, intuitively speaking, adding a stochastic component to the dierential
equation leads to uctuations around the deterministic path. Clearly, how much
the solution of the SDE diers from the deterministic one is random, i.e. unknown.
Further below in ch. 9.4.4, we will understand that the solution of the deterministic
dierential equation (9.4) is identical to the evolution of the expected value of r (t) .
i.e. (t) = 1
0
r (t) for t 0.
Generalized Brownian motion (Ito processes)
A more general way to describe stochastic processes is the following SDE
dr(t) = c (r (t) . . (t) . t) dt +/ (r (t) . . (t) . t) d. (t) . (9.5)
Here, one also refers to c (.) as the drift rate and to /
2
(.) as the instantaneous variance
rate. Note that these functions can be stochastic themselves. In addition to arguments
r (t) and time, Brownian motion . (t) can be included in these arguments. Thinking
of (9.5) as a budget constraint of a household, an example could be that wage income
or the interest rate depend on the current realization of the economys fundamental
source of uncertainty, which is . (t) .
Stochastic dierential equations with Poisson processes
Dierential equations can of course also be constructed that are driven by a Pois-
son process. A very simple example is
dr(t) = cdt +/d (t) . (9.6)
A realization of this path for r (0) = r
0
is in the following gure and can be understood
very easily. As long as no jump occurs, i.e. as long as d = 0. the variable r (t) follows
dr(t) = cdt which means linear growth, r (t) = r
0
+ ct. This is plotted as the thin
line. When jumps, i.e. d = 1. r (t) increases by / : Writing dr(t) = ~ r (t) r (t) .
where ~ r (t) is the level of r immediately after the jump, and letting the jump be very
fast such that dt = 0 during the jump, we have ~ r (t) r (t) = / 1. where the 1 stems
from d (t) = 1. Hence,
~ r (t) = r (t) +/. (9.7)
Clearly, the point in times when a jump occurs are random. A tilde (~) will always
denote in what follows (and in various papers in the literature) the value of a quantity
immediately after a jump.
174
Figure 39 An example of a Poisson process with drift (thick line) and a deterministic
dierential equation (thin line)
In contrast to Brownian motion, a Poisson process contributes to the increase of
the variable of interest: Without the d (t) term (i.e. for / = 0), r (t) would follow the
thin line. With occasional jumps, r (t) jumps faster. In the Brownian motion case of
the gure before, realizations of r (t) remained close to the deterministic solution.
This is simply due to the fact that the expected increment of Brownian motion is
zero while the expected increment of a Poisson process is positive.
Note that in the more formal literature, the tilde is not used but a dierence is
made between r (t) and r (t
) where t
stands for the point in time an instant

before t. (This is probably easy to understand on an intuitive level, thinking about
it for too long might not be a good idea as time is continuous ...) One would then
express the change in r due to a jump by r (t) = r (t
) +/ as the value of r to which

/ is added is the value of r before the jump. As the tilde-notation turned out to be
relatively intuitive, we will follow it in what follows.
12
A geometric Poisson process
An further example would be the geometric Poisson process
dr(t) = c ( (t) . t) r (t) dt +/ ( (t) . t) r (t) d (t) . (9.8)
Processes are usually called geometric when they describe the rate of change of some
RV r (t) . i.e. dr(t) ,r(t) is not a function of r (t) . In this example, the deterministic
part shows that r (t) grows at the rate of c (.) in a deterministic way and jumps by
/ (.) percent, when (t) jumps. Note that in contrast to a Brownian motion SDE,
c (.) here is not the average growth rate of r (t) (see below on expectations).
Geometric Poisson processes as here are sometimes used to describe the evolution
of asset prices in a simple way. There is some deterministic growth component c (.)
12
The technical background for the t
notation is the fact that the process x(t) is a so called

cdlg process ("continu a droite, limites a gauche"). That is, the paths of x(t) are continuous from
the right with left limits. The left limit is denoted by x(t
) = lim
s"t
x(s). See Sennewald (2006) or
Sennewald and Wlde (2006) for further details and references to the mathematical literature.
175
and some stochastic component / (.) . When the latter is positive, this could reect
new technologies in the economy. When the latter is negative, this equation could be
used to model negative shocks like oil-price shocks or natural disasters.
Aggregate uncertainty and random jumps
An interesting extension of a Poisson dierential equation consists in making the
amplitude of the jump random. Taking a simple dierential equation with Poisson
uncertainty as starting point, d(t) = /(t) d (t) . where / is a constant, we can
now assume that / (t) is governed by some distribution, i.e.
d(t) = / (t) (t) d (t) . where / (t) ~
_
j. o
2
_
. (9.9)
Assume that (t) is total factor productivity in an economy. Then, (t) does not
change as long as d (t) = 0. When (t) jumps, (t) changes by / (t) . i.e. d(t) =
~
(t) (t) = / (t) (t) . which we can rewrite as
~
(t) = (1 +/ (t)) (t) . \t where (t) jumps.
This equation says that whenever a jump occurs, (t) increases by / (t) percent,
i.e. by the realization of the random variable / (t) . Obviously, the realization of / (t)
matters only for points in time where (t) jumps.
Note that (9.9) is the stochastic dierential equation representation of the evolu-
tion of the states of the economy in the Pissarides-type matching model of Shimer
(2005), where aggregate uncertainty, (t) here, follows from a Poisson process. The
presentation in Shimers paper is A shock hits the economy according to a Pois-
son process with arrival rate `, at which point a new pair (j
0
. :
0
) is drawn from a
state dependent distribution. (p. 34). Note also that using (9.9) and assuming large
families such that there is no uncertainty from labour income left on the household
level would allow to analyze the eects of saving and thereby capital accumulation
over the business cycle in a closed-economy model with risk-averse households. The
background for the saving decision would be ch. 10.1.
9.1.3 The integral representation of stochastic dierential equations
Stochastic dierential equations as presented here can also be represented by integral
versions. This is identical to the integral representations for deterministic dierential
equations in ch. 4.3.3. The integral representation will be used frequently when
computing moments of r (t): As an example, think of the expected value of r for
some future point in time t. when expectations are formed today in t. i.e. information
until t is available, 1
t
r (t) . See ch. 9.4.4 or 10.1.6.
Brownian motion
Consider a dierential equation as (9.5). It can more rigorously be represented by
its integral version,
r (t) r (t) =
_
t
t
c(r. :)d: +
_
t
t
/(r. :)d. (:) . (9.10)
176
This version is obtain by rst rewriting (9.5) as dr(:) = c (r. :) d: +/ (r. :) d. (:) .
i.e by simply changing the time index from t to : (and dropping . (:) and writting
r instead of r (:) to shorten notation). Applying then the integral
_
t
t
on both sides
gives (9.10).
This implies, inter alia, a dierentiation rule
d
__
t
t
c(r. :)d: +
_
t
t
/(r. :)d. (:)
_
= d [r (t) r (t)] = dr(t)
= c(r. t)dt +/(r. t)d. (t) .
Poisson processes
Now consider a generalized version of the SDE in (9.6), with again replacing t by
:, dr(:) = c (r (:) . (:)) d: +/ (r (:) . (:)) d (:) . The integral representation reads,
after applying
_
t
t
to both sides,
r (t) r (t) =
_
t
t
c (r (:) . (:)) d: +
_
t
t
/ (r (:) . (:)) d (:) .
This can be checked by computing the dierential with respect to time t.
9.2 Functions of stochastic processes
Possibly the most important aspect when working with stochastic processes in contin-
uous time is that rules for computing dierentials of functions of stochastic processes
are dierent from standard rules. These rules are provided by various forms of Itos
Lemma or change of variable formulas (CVF). Itos Lemma is a rule how to com-
pute dierentials when the basic source of uncertainty is Brownian motion. The CVF
provides corresponding rules when uncertainty stems from Poisson processes or Levy
processes.
9.2.1 Why all this?
Computing dierentials of functions of stochastic processes sounds pretty abstract.
Let us start with an example from deterministic continuous time setups which gives
an idea what the economic background for such dierentials are.
Imagine the capital stock of an economy follows
_
1 (t) = 1 (t)o1 (t) . an ordinary
dierential equation (ODE) known from ch. 4. Assume further that total factor
productivity grows at an exogenous rate of q.
_
(t) ,(t) = q. Let output be given
by 1 ((t) . 1 (t) . 1) and let us ask how output grows over time. The reply would
be provided by looking at the derivative of 1 (.) with respect to time,
d
dt
1 ((t) . 1 (t) . 1) = 1
d(t)
dt
+1
1
d1 (t)
dt
+1
1
d1
dt
.
Alternatively, written as a dierential, we would have
d1 ((t) . 1 (t) . 1) = 1
d(t) +1
1
d1 (t) +1
1
d1.
177
We can now insert equations describing the evolution of TFP and capital, d(t) and
d1 (t) . and take into account that employment 1 is constant. This gives
d1 ((t) . 1 (t) . 1) = 1
q(t) dt +1
1
[1 (t) o1 (t)] dt + 0.
Dividing by dt would give a dierential equation that describes the growth of 1. i.e.
_
1 (t) .
The objective of the subsequent sections is to provide rules on how to compute
dierentials, of which d1 ((t) . 1 (t) . 1) is an example, in setups where 1 (t) or
(t) are described by stochastic DEs and not ordinary DEs as just used in this
example.
9.2.2 Computing dierentials for Brownian motion
One stochastic process
Lemma 3 (ksendal, 2000, Theorem 4.1.2.) Consider a function 1 (t. r) of the
diusion process r R that is at least twice dierentiable in r and once in t. Itos
Lemma gives the dierential d1 as
d1 = 1
t
dt +1
a
dr +
1
2
1
aa
(dr)
2
(9.11)
where (dr)
2
is computed by using
dtdt = dtd. = d.dt = 0. d.d. = dt. (9.12)
Let us look at an example. Assume that r (t) is described by a generalized Brown-
ian motion as in (9.5). The square of dr is then given by
(dr)
2
= c
2
(.) (dt)
2
+ 2c (.) / (.) dtd. +/
2
(.) (d.)
2
= /
2
(.) dt.
where the last equality used the rules from (9.12). Unfortunately, these rules have
a pretty complex stochastic background and an explanation would go far beyond the
objective of these notes. For those interested in the details, see the further-reading
chapter 9.5 on general background. The dierential of 1 (t. r) then reads
d1 = 1
t
dt +1
a
c (.) dt +1
a
/ (.) d. +
1
2
1
aa
/
2
(.) dt
=
_
1
t
+1
a
c (.) +
1
2
1
aa
/
2
(.)
_
dt +1
a
/ (.) d.. (9.13)
When we compare this dierential with the normal one, we recognize familiar terms:
The partial derivatives times deterministic changes, 1
t
+1
a
c (.) . would appear also in
circumstances where r follows a deterministic evolution. Put dierently, for / (.) = 0
in (9.5), the dierential d1 reduces to 1
t
+1
a
c (.) dt. Brownian motion therefore
aects the dierential d1 in two ways: First, the stochastic term d. is added and sec-
ond, maybe more surprisingly, the deterministic part of d1 is also aected through
the quadratic term containing the second derivative 1
aa
.
178
The lemma for many stochastic processes
This was the simple case of one stochastic process. Now consider the case of many
stochastic processes. Think of the price of many stocks traded on the stock market.
We then have the following
Lemma 4 (slightly simplied version of ksendal, 2000, Theorem 4.2.1.) Consider
dr
1
= n
1
dt +
11
d.
1
+... +
1n
d.
n
.
.
.
dr
a
= n
a
dt +
a1
d.
1
+... +
an
d.
n
or in matrix notation
dr = ndt +d.(t)
where
r =
_
_
_
r
1
.
.
.
r
a
_
_
_
. n =
_
_
_
n
1
.
.
.
n
a
_
_
_
. =
_
_
_

11
...
1n
.
.
.
.
.
.
a1
...
an
_
_
_
. d. =
_
_
_
d.
1
.
.
.
d.
n
_
_
_
.
Consider further a function 1(t. r) from [0. [ 1
a
to 1 with time t and the :
processes in r as arguments. Then
d1(t. r) = 1
t
dt +
a
i=1
1
a
.
dr
i
+
1
2
a
i=1
a
)=1
1
a
.
a
[dr
i
dr
)
] (9.14)
where, as an extension to (9.12),
dtdt = dtd.
i
= d.
i
dt = 0 and d.
i
d.
)
= j
i)
dt. (9.15)
When all .
i
are mutually independent then j
i)
= 0 for i ,= , and j
i)
= 1 for i = ,.
When two Brownian motions .
i
and .
)
are correlated, j
i)
is the correlation coecient
between their increments d.
i
and d.
)
.
An example with two stochastic processes
Let us now consider an example for a function 1 (t. r. ) of two stochastic processes.
As an example, assume that r is described by a generalized Brownian motion similar
to (9.5), dr = c (r. . t) dt +/ (r. . t) d.
a
and the stochastic process is described by
d = c (r. . t) dt +q (r. . t) d.
j
. Itos Lemma (9.14) gives the dierential d1 as
d1 = 1
t
dt +1
a
dr +1
j
d +
1
2
_
1
aa
(dr)
2
+ 21
aj
drd +1
jj
(d)
2
(9.16)
Given the rule in (9.15), the squares and the product in (9.16) are
(dr)
2
= /
2
(t. r. ) dt. (d)
2
= q
2
(t. r. ) dt. drd = j
aj
/ (.) q (.) dt.
179
where j
aj
is the correlation coecient of the two processes. More precisely, it is the
correlation coecient of the two normally distributed random variables that underlie
the Wiener processes. The dierential (9.16) therefore reads
d1 = 1
t
dt +c (.) 1
a
dt +/ (.) 1
a
d.
a
+c (.) 1
j
dt +q (.) 1
j
d.
j
+
1
2
__
1
aa
/
2
(.) dt + 2j
aj
1
aj
/ (.) q (.) dt +1
jj
q
2
(.) dt
_
=
_
1
t
+c (.) 1
a
+c (.) 1
j
+
1
2
_
/
2
(.) 1
aa
+ 2j
aj
/ (.) q (.) 1
aj
+q
2
(.) 1
jj
_
dt
+/ (.) 1
a
d.
a
+q (.) 1
j
d.
j
(9.17)
Note that this dierential is almost simply the sum of the dierentials of each
stochastic process independently. The only term that is added is the term that
contains the correlation coecient. In other words, if the two stochastic processes
were independent, the dierential of a function of several stochastic processes equals
the sum of the dierential of each stochastic process individually.
An example with one stochastic process and many Brownian motions
A second example stipulates a stochastic process r (t) governed by dr = n
1
dt +
n
i=1
i
d.
i
. When we compute the square of dr. we obtain
(dr)
2
= (n
1
dt)
2
+ 2n
1
dt [
n
i=1
i
d.
i
] + (
n
i=1
i
d.
i
)
2
= 0 + 0 + (
n
i=1
i
d.
i
)
2
.
where the second equality used (9.12). The dierential of 1 (t. r) therefore reads from
(9.14)
d1(t. r) = 1
t
dt +1
a
[n
1
dt +
n
i=1
i
d.
i
] +
1
2
1
aa
[
n
i=1
i
d.
i
]
2
= 1
t
+1
a
n
1
dt +
1
2
1
aa
[
n
i=1
i
d.
i
]
2
+1
a
n
i=1
i
d.
i
.
Computing the [
n
i=1
i
d.
i
]
2
term requires to take potential correlations into account.
For any two uncorrelated increments d.
i
and d.
)
. d.
i
d.
)
would from (9.15) be zero.
When they are correlated, d.
i
d.
)
= j
i)
dt which includes the case of d.
i
d.
i
= dt.
9.2.3 Computing dierentials for Poisson processes
When we consider the dierential of a function of the variable that is driven by the
Poisson process, we need to take the following CVFs into consideration.
One stochastic process
Lemma 5 Let there be a stochastic process r (t) driven by Poisson uncertainty (t)
described by
dr(t) = c (.) dt +/ (.) d (t) .
180
What was stressed before for Brownian motion is valid here as well: The functions
c (.) and / (.) in the deterministic and stochastic part of this SDE can have as argu-
ments any combinations of (t) . r (t) and t or can be simple constants. Consider the
function 1 (t. r) . The dierential of this function is
d1(t. r) = 1
t
dt +1
a
c (.) dt +1 (t. r +/ (.)) 1 (t. r) d. (9.18)
This rule is very intuitive: The dierential of a function is given by the normal
terms and by a jump term. The normal terms include the partial derivatives
with respect to time t and r times changes per unit of time (1 for the rst argument
and c (.) for r) times dt. Whenever the process increases, r increases by the / (.) .
The jump term therefore captures that the function 1 (.) jumps from 1 (t. r) to
1 (t. ~ r) = 1 (t. r +/ (.)).
Two stochastic processes
Lemma 6 Let there be two independent Poisson processes
a
and
j
driving two
stochastic processes r (t) and (t) .
dr = c (.) dt +/ (.) d
a
. d = c (.) dt +q (.) d
j
and consider the function 1 (r. ) . The dierential of this function is
d1 (r. ) = 1
a
c (.) +1
j
c (.) dt +1 (r +/ (.) . ) 1 (r. ) d
a
+1 (r. +q (.)) 1 (r. ) d
j
. (9.19)
Again, this dierentiation rule consists of the normal terms and the jump
terms. As the function 1 (.) depends on two arguments, the normal term contains
two drift components, 1
a
c (.) and 1
j
c (.) and the jump term contains the eect of
jumps in
a
and in
j
. Note that the dt term does not contain the time derivative
1
t
(r. ) as in this example 1 (r. ) is assumed not to be a function of time and
therefore 1
t
(r. ) = 0. In applications where 1 (.) is a function of time, the 1
t
(.)
would of course have to be taken into consideration.
Let us now consider a case that is frequently encountered in economic models
when there is one economy-wide source of uncertainty, say new technologies arrive or
commodity price shocks occur according to some Poisson process, and many variables
in this economy (e.g. all relative prices) are aected simultaneously by this one shock.
The CVF in situations of this type reads
Lemma 7 Let there be two variables r and following
dr = c (.) dt +/ (.) d. d = c (.) dt +q (.) d.
where uncertainty stems from the same for both variables. Consider the function
1 (r. ) . The dierential of this function is
d1 (r. ) = 1
a
c (.) +1
j
c (.) dt +1 (r +/ (.) . +q (.)) 1 (r. ) d.
181
One nice feature about dierentiation rules for Poisson processes is their very
intuitive structure. When there are two independent Poisson processes as in (9.19),
the change in 1 is given by either 1 (r +/ (.) . ) 1 (r. ) or by 1 (r. +q (.))
1 (r. ) . depending on whether one or the other Poisson process jumps. When both
arguments r and are aected by the same Poisson process, the change in 1 is given
by 1 (r +/ (.) . +q (.)) 1 (r. ) . i.e. the level of 1 after a simultaneous change of
both r and minus the pre-jump level 1 (r. ).
Many stochastic processes
We now present the most general case. Let there be : stochastic processes r
i
(t)
and dene the vector r (t) = (r
1
(t) . .... r
a
(t))
T
. Let stochastic processes be described
by : SDEs
dr
i
(t) = c
i
(.) dt +,
i1
(.) d
1
+... +,
in
(.) d
n
. i = 1. . . . . :. (9.20)
where ,
i)
(.) stands for ,
i)
(t. r (t)) . Each stochastic process r
i
(t) is driven by the
same : Poisson processes. The impact of Poisson process
)
on r
i
(t) is captured by
,
i)
(.) . Note the similarity to the setup for the Brownian motion case in (9.14).
Proposition 8 (Sennewald 2006 and Sennewald and Wlde 2006) Let there be : sto-
chastic processes described by (9.20). For a once continuously dierentiable function
1 (t. r), the process 1 (t. r) obeys
d1 (t. r (t)) = 1
t
(.) +
a
i=1
1
a
.
(.) c
i
(.) dt
+
n
)=1
_
1
_
t. r (t) +,
)
(.)
_
1 (t. r (t))
_
d
)
, (9.21)
where ,
t
and ,
a
.
, i = 1. . . . . :, denote the partial derivatives of , with respect to t and
r
i
, respectively, and ,
)
stands for the :-dimensional vector function
_
,
1)
. . . . . ,
a)
_
T
.
The intuitive understanding is again simplied by focusing on normal continuous
terms and on jump terms. The continuous terms are as before and simply describe
the impact of the c
i
(.) in (9.20) on 1 (.) . The jump terms show how 1 (.) changes
from 1 (t. r (t)) to 1
_
t. r (t) +,
)
(.)
_
when Poisson process , jumps. The argument
r (t) +,
)
(.) after the jump of
)
is obtained by adding ,
i)
to component r
i
in r. i.e.
r (t) +,
)
(.) =
_
r
1
+,
1)
. r
2
+,
2)
. .... r
a
+,
a)
_
.
9.2.4 Brownian motion and a Poisson process
There are much more general stochastic processes in the literature than just Brownian
motion or Poisson processes. This section provides a CVF for a function of a vari-
able which is driven by both Brownian motion and a Poisson process. More general
processes than just additive combinations are so-called Levy processes which will be
analyzed in future editions of these notes.
182
Lemma 8 Let there be a variable r which is described by
dr = c (.) dt +/ (.) d. +q (.) d (9.22)
and where uncertainty stems from Brownian motion . and a Poisson process . Con-
sider the function 1 (t. r) . The dierential of this function is
d1 (t. r) =
_
1
t
+1
a
c (.) +
1
2
1
aa
/
2
(.)
_
dt+1
a
/ (.) d. +1 (t. r +q (.)) 1 (t. r) d.
(9.23)
Note that this lemma is just a combination of Itos Lemma (9.13) and the CVF
for a Poisson process from (9.18). For an arrival rate of zero, i.e. for d = 0 at all
times, (9.23) is identical to (9.13). For / (.) = 0. (9.23) is identical to (9.18).
9.2.5 Application - Option pricing
One of the most celebrated papers in economics is the paper by Black and Scholes
(1973) in which they derived a pricing formula for options. This section presents the
rst steps towards obtaining this pricing equation. The subsequent chapter 9.3.1 will
complete the analysis. This section presents a simplied version (by neglecting jumps
in the asset price) of the derivation of Merton (1976). The basic question is: what is
the price of an option on an asset if there is absence of arbitrage on capital markets?
The asset and option price
The starting point is the price o of an asset which evolves according to a geometric
process
do
o
= cdt +od.. (9.24)
Uncertainty is modelled by the increment d. of Brownian motion. We assume that
the economic environment is such (inter alia short selling is possible, there are no
transaction cost) that the price of the option is given by a function 1 (.) having as
arguments only the price of the asset and time, 1(t. o (t)). The dierential of the
price of the option is then given from (9.11) by
d1 = 1
t
dt +1
S
do +
1
2
1
SS
(do)
2
. (9.25)
As the square of do is given by (do)
2
= o
2
o
2
dt. the dierential reads
d1 =
_
1
t
+co1
S
+
1
2
o
2
o
2
1
SS
_
dt +oo1
S
d. =
d1
1
= c
1
dt +o
1
d. (9.26)
where the last step dened
c
1
=
1
t
+co1
S
+
1
2
o
2
o
2
1
SS
1
. o
1
=
oo1
S
1
. (9.27)
183
Absence of arbitrage
Now comes the trick - the no-arbitrage consideration. Consider a portfolio that
consists of `
1
units of the asset itself, `
2
options and `
S
units of some riskless assets,
say wealth in a savings account. The price of such a portfolio is then given by
1 = `
1
o +`
2
1 +`
S
`.
where ` is the price of one unit of the riskless asset. The proportional change of the
price of this portfolio can be expressed as (holding the `
i
s constant, otherwise Itos
Lemma would have to be used)
d1 = `
1
do +`
2
d1 +`
S
d` =
d1
1
=
`
1
o
1
do
o
+
`
2
1
1
d1
1
+
`
S
`
1
d`
`
.
Dening shares of the portfolio held in these three assets by o
1
= `
1
o,1 and o
2
=
`
2
o,1. inserting option and stock price evolutions from (9.24) and (9.25) and letting
the riskless asset ` pay a constant return of :, we obtain
d1,1 = o
1
cdt +o
1
od. +o
2
c
1
dt +o
2
o
1
d. + (1 o
1
o
2
) :dt
= o
1
[c :] +o
2
[c
1
:] +: dt +o
1
o +o
2
o
1
d.. (9.28)
Now assume someone chooses weights such that the portfolio no longer bears any risk
o
1
o +o
2
o
1
= 0. (9.29)
The return of such a portfolio with these weights must then of course be identical to
the return of the riskless interest asset, i.e. identical to :.
d1,dt
1
vicI|ccc
= o
1
[c :] +o
2
[c
1
:] +:[
0
1
o=0
2
o
T
= : =
c :
o
=
c
1
:
o
1
.
If the return of the riskless portfolio did not equal the return of the riskless interest
rates, there would be arbitrage possibilities. This approach is therefore called no-
arbitrage pricing.
The Black-Scholes formula
Finally, inserting c
1
and o
1
from (9.27) yields the celebrated dierential equation
that determines the evolution of the price of the option,
c :
o
=
1
t
+co1
S
+
1
2
o
2
o
2
1
SS
:1
oo1
S
=
1
2
o
2
o
2
1
SS
+:o1
S
:1 +1
t
= 0. (9.30)
Clearly, this equation does not to say what the price 1 of the option actually is. It
only says how it changes over time and in reaction to o. But as we will see in ch.
9.3.1, this equation can actually be solved explicitly for the price of the option. Note
also that we did not make any assumption so far about what type of option we talk
about.
184
9.2.6 Application - Deriving a budget constraint
Most maximization problems require a constraint. For a household, this is usually
the budget constraint. It is shown here how the structure of the budget constraint
depends on the economic environment the household nds itself in and how the CVF
needs to be applied here.
Let wealth at time t be given by the number :(t) of stocks a household owns times
their price (t), c (t) = :(t) (t). Let the price follow a process that is exogenous to
the household (but potentially endogenous in general equilibrium),
d (t) = c (t) dt +, (t) d (t) . (9.31)
where c and , are constants. For , we require , 1 to avoid that the price can
become zero or negative. Hence, the price grows with the continuous rate c and at
discrete random times it jumps by , percent. The random times are modeled by the
jump times of a Poisson process (t) with arrival rate `, which is the probability
that in the current period a price jump occurs. The expected (or average) growth
rate is then given by c +`, (see ch. 9.4.4).
Let the household earn dividend payments, : (t) per unit of asset it owns, and
labour income, n(t). Assume furthermore that it spends j (t) c (t) on consumption,
where c (t) denotes the consumption quantity and j (t) the price of one unit of the
consumption good. When buying stocks is the only way of saving, the number of
stocks held by the household changes in a deterministic way according to
d:(t) =
:(t) : (t) +n(t) j (t) c (t)
(t)
dt.
When savings :(t) : (t) +n(t) j (t) c (t) are positive, the number of stocks held by
the household increases by savings divided by the price of one stock. When savings
are negative, the number of stocks decreases.
The change of the households wealth, i.e. the households budget constraint, is
then given by applying the CVF to c (t) = :(t) (t). The appropriate CVF comes
from (9.19) where only one of the two dierential equations shows the increment of
the Poisson process explicitly. With 1 (r. ) = r, we obtain
dc (t) =
_
(t)
:(t) : (t) +n(t) j (t) c (t)
(t)
+:(t) c (t)
_
dt
+:(t) [ (t) +, (t)] :(t) (t) d (t)
= : (t) c (t) +n(t) j (t) c (t) dt +,c (t) d (t) . (9.32)
where the interest-rate is dened as
: (t) =
: (t)
(t)
+c.
This is a very intuitive budget constraint: As long as the asset price does not jump,
i.e., d (t) = 0, the households wealth increases by current savings, : (t) c (t)+n(t)
185
j (t) c (t), where the interest rate, : (t), consists of dividend payments in terms of the
asset price plus the deterministic growth rate of the asset price. If a price jump occurs,
i.e., d (t) = 1, wealth jumps, as the price, by , percent, which is the stochastic part
of the overall interest-rate. Altogether, the average interest rate amounts to : (t)+`,
(see ch. 9.4.4).
9.3 Solving stochastic dierential equations
Just as there are theorems on uniqueness and existence for deterministic dierential
equations, there are theorems for SDEs on these issues. There are also solution
methods for SDEs. We will consider here some examples for solutions of SDEs.
Just as for ordinary deterministic dierential equations in ch. 4.3.2, we will simply
present solutions and not show how they can be derived. Solutions of stochastic
dierential equations d (r (t)) are, in analogy to the denition for ODE, again time
paths r (t) that satisfy the dierential equation. Hence, by applying Itos Lemma or
the CVF, one can verify whether the solutions presented here are indeed solutions.
9.3.1 Some examples for Brownian motion
This section rst looks at SDEs with Brownian motion which are similar to the ones
that were presented when introducing SDEs in ch. 9.1.2: We start with Brownian
motion with drift as in (9.3) and then look at an example for generalized Brownian
motion in (9.5). In both cases, we work with SDEs which have an economic interpre-
tation and are not just SDEs. Finally, we complete the analysis of the Black-Scholes
option pricing approach.
Brownian motion with drift 1
As an example for Brownian motion with drift, consider a representation of a
production technology which could be called a dierential-representation for the
technology. This type of presenting technologies was dominant in early contributions
that used continuous time methods under uncertainty but is sometimes still used
today. A simple example is
d1 (t) = 1dt +o1d. (t) . (9.33)
where 1 (t) is output in t. is a (constant) measure of total factor productivity, 1 is
capital, o is some variance measure of output and . is Brownian motion. The change
of output at each instant is then given by d1 (t). See Further reading on references
to the literature.
What does such a representation of output imply? To see this, look at (9.33) as
Brownian motion with drift, i.e. consider . 1. and o to be a constant (which is
relaxed in ex. 12). The solution to this dierential equation starting in t = 0 with 1
0
and . (0) is
1 (t) = 1
0
+1t +o1. (t) .
186
To simplify an economic interpretation set 1
0
= . (0) = 0. Output is then given by
1 (t) = (t +o. (t)) 1. This says that with a constant factor input 1. output in t
is determined by a deterministic and a stochastic part. The deterministic part t
implies linear (i.e. not exponential as is usually assumed) growth, the stochastic part
o. (t) implies deviations from the trend. As . (t) is Brownian motion, the sum of
the deterministic and stochastic part can become negative. This is an undesirable
property of this approach.
To see that 1 (t) is in fact a solution of the above dierential-representation, just
apply Itos Lemma and recover (9.33).
Brownian motion with drift 2
As a second example and in an attempt to better understand why output can
become negative, consider a standard representation of a technology 1 (t) = (t) 1
and let TFP follow Brownian motion with drift,
d(t) = qdt +od. (t) .
where q and o are constants. What does this alternative specication imply?
Solving the SDE yields (t) =
0
+ qt + o. (t) (which can again be checked by
applying Itos lemma). Output is therefore given by
1 (t) = (
0
+qt +o. (t)) 1 =
0
1 +q1t +o1. (t)
and can again become negative.
Geometric Brownian motion
Let us now assume that TFP follows geometric Brownian motion process,
d(t) ,(t) = qdt +od. (t) . (9.34)
where again q and o are constants. Let output continue to be given by 1 (t) = (t) 1.
The solution for TFP, provided an initial condition (0) =
0
. is given by
(t) =
0
c
(
j
1
2
o
2
)
to:(t)
. (9.35)
At any point in time t. the TFP level depends on time t and the current level of the
stochastic process . (t) . This shows that TFP at each point t in time is random and
thereby unkown from the perspective of t = 0. Hence, a SDE and its solution describe
the deterministic evolution of a distribution over time. One could therefore plot a
picture of (t) which in principle would look like the evolution of the distribution in
ch. 7.4.1.
Interestingly, and this is due to the geometric specication in (9.34) and impor-
tant for representing technologies in general, TFP can not become negative. While
Brownian motion . (t) can take any value between minus and plus innity, the term
c
(
j
1
2
o
2
)
to:(t)
is always positive. With an 1 specication for output, output is
187
always positive, 1 (t) =
0
c
(
j
1
2
o
2
)
to:(t)
1. In fact, it can be shown that output and
TFP are lognormally distributed. Hence, the specication of TFP with geometric
Brownian motion provides an alternative to the dierential-representation in (9.33)
which avoids the possibility of negative output.
The level of TFP at some future point in time t is determined by a deterministic
part,
_
q
1
2
o
2
_
t. and by a stochastic part, o. (t) . Apparently, the stochastic nature
of TFP has an eect on the deterministic term. The structure (the factor 1,2 and the
quadratic termo
2
) reminds of the role the stochastic disturbance plays in Itos lemma.
There as well (see e.g. (9.13)), the stochastic disturbance aects the deterministic
component of the dierential. As we will see later, however, this does not aect
expected growth. As we will see in (9.53), expected output grows at the rate q (and
is thereby independent of the variance parameter o).
One can verify that (9.35) is a solution to (9.34) by using Itos lemma. To do so, we
bring (9.34) into a form which allows us to apply the formulas which we got to know
in ch. 9.2.2. Dene r (t) =
_
q
1
2
o
2
_
t + o. (t) and (t) = 1 (r (t)) =
0
c
a(t)
. As a
consequence, the dierential for r (t) is a nice SDE, dr(t) =
_
q
1
2
o
2
_
dt + od. (t) .
As this SDE is of the form as in (9.5), we can use Itos lemma from (9.13) and nd
d(t) = d1 (r (t)) =
_
1
a
(r (t))
_
q
1
2
o
2
_
+
1
2
1
aa
(r (t)) o
2
_
dt +1
a
(r (t)) od..
Inserting the rst and second derivatives of 1 (r (t)) yields
d(t) =
_
0
c
a(t)
_
q
1
2
o
2
_
+
1
2
0
c
a(t)
o
2
_
dt +
0
c
a(t)
od. =
d(t) ,(t) =
_
q
1
2
o
2
+
1
2
o
2
_
dt +od. = qdt +od..
where the i reinserted (t) =
0
c
a(t)
and divided by (t) . As (t) satises the
orignianl SDE (9.34), (t) is a solution of (9.34).
Option pricing
Let us come back to the Black-Scholes formula for option pricing. The SDE
derived above in (9.30) describes the evolution of the price 1 (t. o (t)) . where t is
time and o (t) the price of the underlying asset at t. We now look at a European
call option, i.e. an option which gives the right to buy an asset at some xed point
in time 1. the maturity date of the option. The xed exercise or strike price of the
option, i.e. the price at which the asset can be bought is denoted by 1.
Clearly, at any point in time t when the price of the asset is zero, the value of
the option is zero as well. This is the rst boundary condition for our SDE (9.30).
When the option can be exercised at 1 and the price of the asset is o. the value of
the option is zero if the exercise price 1 exceeds the price of the asset and o 1 if
not. This is the second boundary condition.
1 (t. 0) = 0 1 (1. o) = max 0. o 1 .
188
In the latter case where o 1 0, the owner of the option would in fact buy the
asset.
Given these two boundary conditions, the SDE has the solution (originating from
physics, see Black and Scholes, 1973, p. 644)
1 (t. o (t)) = o (t) c(d
1
) 1c
v[Tt[
c(d
2
)
where
c() =
1
_
2:
_
j
1
c
u
2
2
dn.
d
1
=
ln
S(t)
1
+
_
:
v
2
2
_
(1 t)
o
_
1 t
. d
2
= d
1
o
_
1 t.
The expression 1 (t. o (t)) gives the price of an option at a point in time t _ 1 where
the price of the asset is o (t) . It is a function of the cumulative standard normal
distribution c() . For any path of o (t) . time up to maturity 1 aects the option
price through d
1
. d
2
and directly in the second term of the above dierence. More
interpretation is oered by many nance textbooks.
9.3.2 A general solution for Brownian motions
Consider the dierential equation
dr(t) = (t) r (t) +c (t) dt +
n
i=1
1
i
(t) r (t) +/
i
(t) d\
i
(t) (9.36)
with boundary value r (0) = r
0
. \
i
(t) is a Brownian motion. The correlation coe-
cient of its increments with the increments of \
)
(t) is j
i)
. Its solution is
r (t) = c(t)
_
_
_
_
r
0
+
t
_
0
c(:)
1
c (:)
1b
(:) d:
+
n
i=1
t
_
0
c(:)
1
/
i
(:) d\
i
(:)
_
_
_
_
(9.37)
where
c(t) = c
I
R
0
(
(c)
1
2
Y
TT
(c)
)
oc
r
.=1
I
R
0
1
.
(c)oW
.
(c)
.
11
(:) =
n
i=1
n
)=1
j
i)
1
i
(:) 1
)
(:) .
1b
(:) =
n
i=1
n
)=1
j
i)
1
i
(:) /
i
(:) .
Arnold (1974, ch. 8.4) provides the solution for independent Brownian motions.
This solution is an educated guess. To obtain some intuition for (9.37), we can
consider the case of certainty. For 1
i
(t) = /
i
(t) = 0. this is an linear ODE and the
solution should correspond to the results we know from ch. 4.3.2. For the general
case, we now show that (9.37) indeed satises (9.36).
189
Proof. In order to use Itos Lemma, dene
(t) =
_
t
0
_
(:)
1
2
11
(:)
_
d: +
n
i=1
_
t
0
1
i
(:) d\
i
(:)
such that
c(t) = c
j(t)
(9.38)
and dene
.(t) = r
0
+
t
_
0
c(:)
1
c (:)
1b
(:) d: +
n
i=1
t
_
0
c(:)
1
/
i
(:) d\
i
(:)
Then the solution (9.37) reads
r(t) = c
j(t)
.(t) = ,((t). .(t)) (9.39)
where
d(t) =
_
(t)
1
2
11
(t)
_
dt +
n
i=1
1
i
(t) d\
i
(t)
d.(t) = c(t)
1
c (t)
1b
(t) dt +
n
i=1
c(t)
1
/
i
(t) d\
i
(t)
In order to compute the dierential dr(t), we have to apply the multidimensional
Ito-Formula (9.16) where time is not an argument of ,(.). This gives
dr(t) = c
j(t)
.(t)d(t) +c
j(t)
d.(t) +
1
2
[c
j(t)
.(t)[d]
2
+ 2c
j(t)
[dd.] + 0 + [d.]
2
] (9.40)
As [d]
2
=
_
n
i=1
1
i
(t) d\
i
(t)
_
2
by (9.15) - all terms multiplied by dt equal zero - we
obtain
[d]
2
=
n
i=1
n
)=1
j
i)
1
i
(t) 1
)
(t) dt (9.41)
Further, again by (9.15),
dd. =
_
n
i=1
1
i
(t) d\
i
(t)
_ _
n
i=1
c(t)
1
/
i
(t) d\
i
(t)
_
= c(t)
1
n
i=1
n
)=1
j
i)
1
i
(t) /
i
(t) dt. (9.42)
Hence, reinserting (9.38) and (9.39) and inserting in (9.40) gives
dr(t) = r(t)
__
(t)
1
2
11
(t)
_
dt +
n
i=1
1
i
(t) d\
i
(t)
_
+c (t)
1b
(t) dt +
n
i=1
/
i
(t) d\
i
(t)
+
1
2
_
r(t)
_
n
i=1
n
)=1
j
i)
1
i
(t) 1
)
(t) dt
+ 2
n
i=1
n
)=1
j
i)
1
i
(t) /
i
(t) dt
190
Rearranging gives
dr(t) = r(t)(t) +c (t) dt +
n
i=1
r(t)1
i
(t) +/
i
(t) d\
i
(t)
_
1
2
r(t)
11
(t) +
1b
(t)
_
dt
+
_
1
2
r(t)
n
i=1
n
)=1
j
i)
1
i
(t) 1
)
(t) +
n
i=1
n
)=1
j
i)
1
i
(t) /
i
(t)
_
dt
= r(t)(t) +c (t) dt +
n
i=1
r(t)1
i
(t) +/
i
(t) d\
i
(t) .
9.3.3 Dierential equations with Poisson processes
The presentation of solutions and their verifycation for Poisson processes follows a
similar structure as for Brownian motion. We start here with a geometric Poisson
process and compare properties of the solution with the TFP Brownian motion case.
We then look at a more general process - the description of a budget constraint -
which extends the geometric Poisson process. One should keep in mind, as stressed
already in ch. 4.3.3, that solutions to dierential equations are dierent from the
integral representation of e.g. ch. 9.1.3.
A geometric Poisson process
Imagine that TFP follows a deterministic trend and occasionally makes a discrete
jump. This is capture by a geometric description as in (9.34) only that Brownian
motion is replaced by a Poisson process,
d(t) ,(t) = qdt +od (t) . (9.43)
Again, q and o are constant.
The solution to this SDE is given by
(t) =
0
c
jt[q(t)q(0)[ ln(1o)
. (9.44)
In contrast to the solution (9.35) for the Brownian motion case, uncertainty does not
aect the deterministic part here. As before, TFP follows a deterministic growth
component and a stochastic component, [ (t) (0)] ln (1 +o). The latter makes
future TFP uncertaint from the perspective of today.
The claim that (t) is a solution can be proven by applying the appropriate CVF.
This will be done for the next, more general, example.
A budget constraint
As a second example, we look at a dynamic budget constraint,
dc (t) = : (t) c (t) +n(t) c (t) dt +,c (t) d. (9.45)
191
Dening (:) = n(:) c (:) . the backward solution of (9.45) with initial condition
c (0) = c
0
reads
c (t) = c
j(t)
_
c
0
+
_
t
0
c
j(c)
(:) d:
_
(9.46)
where (t) is
(t) =
_
t
0
: (n) dn + [ (t) (0)] ln (1 +,) . (9.47)
Note that the solution in (9.46) has the same structure as the solution to a determin-
istic version of the dierential equation (9.45) (which we would obtain for , = 0).
Put dierently, the stochastic component in (9.45) only aects the discount factor
(t) . This is not surprising - in a way - as uncertainty is proportional to c (t) and the
factor , can be seen as the stochastic component of the interest payments on c (t) .
We have just stated that (9.46) is a solution. This should therefore be verifyed.
To this end, dene
.(t) = c
0
+
_
t
0
c
j(c)
(:)d:. (9.48)
and write the solution (9.46) as c(t) = c
j(t)
.(t) where from (9.47) and (9.48),
d(t) = :(t)dt + ln(1 + ,)d(t). d.(t) = c
j(t)
(t)dt. (9.49)
We have thereby dened a function c (t) = 1 ( (t) . . (t)) where the SDEs describing
the evolution of (t) and . (t) are given in (9.49). The CVF (9.19) then says
dc(t) = c
j(t)
.(t):(t)dt +c
j(t)
c
j(t)
(t)dt +
_
c
j(t)ln(1o)
.(t) c
j(t)
.(t)
_
d
= : (t) c (t) + (t) dt +,c (t) d.
This is the original dierential equation. Hence, c (t) is a solution for (9.45).
The intertemporal budget constraint
In stochastic worlds, there is also a link between dynamic and intertemporal bud-
get constraints, just as in deterministic setups as e.g. in ch. 4.3.4. We rst present
here a budget constraint for a nite planning horizon and then generalize the result.
For the nite horizon case, we can rewrite (9.46) as
_
t
0
c
j(c)
c (:) d: +c
j(t)
c (t) = c
0
+
_
t
0
c
j(c)
n(:) d:.
This formulation suggests a standard economic interpretation. Total expenditure over
the planning horizon from 0 to t on the left-hand side must equal total wealth on the
right-hand side. Total expenditure consists of the present value of the consumption
expenditure path c (:) and the present value of assets c (t) the household wants to
hold at the end of the planning period, i.e. at t. Total wealth is given by initial
nancial wealth c
0
and the present value of current and future wage income n(:) . All
192
discounting takes place at the realized stochastic interest rate (:) - no expectations
are formed.
In order to obtain an innite horizon intertemporal budget constraint, the solution
(9.46) should be written more generally - after replacing t by t and 0 by t - as
c (t) = c
j(t)
c
t
+
_
t
t
c
j(t)j(c)
(n(:) c (:)) d: (9.50)
where (t) is
(t) =
_
t
t
: (n) dn + ln (1 +,) [ (t) (t)] . (9.51)
Multiplying (9.50) by c
j(t)
and rearranging gives
c (t) c
j(t)
+
_
t
t
c
j(c)
(c (:) n(:))d: = c(t).
Letting t go to innity, assuming a no-Ponzi game condition lim
t!1
c (t) c
j(t)
= 0
and rearranging again yields
_
1
t
c
j(c)
c (:) d: = c(t) +
_
1
t
c
j(c)
n(:) d:.
The present value of consumption expenditure needs to equal current nancial
wealth c (t) plus human wealth, i.e. the present value of current and future labour
income. Here as well, discounting takes place at the risk-corrected interest rate
as captured by (t) in (9.51). Note also that this intertemporal budget constraint
requires equality in realizations, not in expected terms.
A switch process
Here are two funny processes which have an interesting simple solution. Consider
the initial value problem,
dr(t) = 2r (t) d (t) . r(0) = r
0
.
The solution is r (t) = (1)
q(t)
r
0
. i.e. r (t) oscillates between r
0
and r
0
.
Now consider the transformation = +r. It evolves according to
d = 2 ( ) d.
0
= +r
0
.
Its solution is (t) = + (1)
q(t)
r
0
and (t) oscillates now between r (t) oscillates
between r
0
and +r
0
.
This could be nicely used for models with wage uncertainty, where wages are
sometimes high and sometimes low. An example would be a matching model where
labour income is n (i.e. +r
0
) when employed and / (i.e. r
0
) when unemployed.
The dierence between labour income levels is then 2r
0
. The drawback is that the
probability of nding a job is identical to the probability of losing it.
193
9.4 Expectation values
9.4.1 The idea
What is the idea behind expectations of stochastic processes? When thinking of a
stochastic process A (t), either a simple one like Brownian motion or a Poisson process
or more complex ones described by SDEs, it is useful to keep in mind that one can
think of the value A (t) the stochastic process takes at some future point t in time as
a normal random variable. At each future point in time A (t) is characterized by
some mean, some variance, by a range etc. This is illustrated in the following gure.
Figure 40 The distribution of A (t) at t = .4 t = 0
The gure shows four realizations of the stochastic process A (t) . The starting
point is today in t = 0 and the mean growth of this process is illustrated by the
dotted line. When we think of possible realizations of A (t) for t = .4 from the
perspective of t = 0. we can imagine a vertical line that cuts through possible paths
at t = .4. With suciently many realizations of these paths, we would be able to
make inferences about the distributional properties of A (.4) . As the process used
for this simulation is a geometric Brownian motion process as in (9.34), we would
nd that A (.4) is lognormally distributed as depicted above. Clearly, given that we
have a precise mathematical description of our process, we do not need to estimate
distributional properties for A (t) . as suggested by this gure, but we can explicitly
compute them.
What this section therefore does is to provide tools that allow to determine proper-
ties of the distribution of a stochastic process for future points in time. Put dierently,
understanding a stochastic process means understanding the evolution of its distri-
butional properties. We will rst start by looking at relatively straightforward ways
to compute means. Subsequently, we provide some martingale results which allow
us to then understand a more general approach to computing means and also higher
moments.
194
9.4.2 Simple results
Brownian motion I
Consider a Brownian motion process 2 (t) . t 0. Let 2 (0) = 0. From the
denition of Brownian motion in ch. 9.1.1, we know that 2 (t) is normally distributed
with mean 0 and variance o
2
t. Let us assume we are in t today and 2 (t) ,= 0. What
is the mean and variance of 2 (t) for t t?
We construct a stochastic process 2 (t)2 (t) which at t = t is equal to zero. Now
imagining that t is equal to zero, 2 (t)2 (t) is a Brownian motion process as dened
in ch. 9.1.1. Therefore, 1
t
[2 (t) 2 (t)] = 0 and var
t
[2 (t) 2 (t)] = o
2
[t t] .
This says that the increments of Brownian motion are normally distributed with mean
zero and variance o
2
[t t] . Hence, the reply to our question is
1
t
2 (t) = 2 (t) . var
t
2 (t) = o
2
[t t] .
For our convention o = 1 which we follow here as stated at the beginning of ch. 9.1.2,
the variance would equal var
t
2 (t) = t t.
Brownian motion II
Consider again the geometric Brownian process d(t) ,(t) = qdt + od. (t) de-
scribing the growth of TFP in (9.34). Given the solution (t) =
0
c
(
j
1
2
o
2
)
to:(t)
from (9.35), we would now like to know what the expected TFP level (t) in t from
the perspective of t = 0 is. To this end, apply the expectations operator 1
0
and nd
1
0
(t) =
0
1
0
c
(
j
1
2
o
2
)
to:(t)
=
0
c
(
j
1
2
o
2
)
t
1
0
c
o:(t)
. (9.52)
where the second equality exploited the fact that c
(
j
1
2
o
2
)
t
is non-random. As . (t)
is standard normally distributed, . (t) ~` (0. t), o. (t) is normally distributed with
` (0. o
2
t) . As a consequence (compare ch. 7.3.3, especially eq. (7.6)), c
o:(t)
is log-
normally distributed with mean c
1
2
o
2
t
. Hence,
1
0
(t) =
0
c
(
j
1
2
o
2
)
t
c
1
2
o
2
t
=
0
c
jt
. (9.53)
The expected level of TFP growing according to a geometric Brownian motion process
grows at the drift rate q of the stochastic process. The variance parameter o does
not aect expected growth.
Poisson processes I
The expected value in t of (t) . where t t. and its variance are (cf. e.g. Ross,
1993, p. 210 and 241)
1
t
(t) = (t) +`[t t] . t t (9.54)
var
t
(t) = `[t t] .
The expected value 1
t
(t) directly follows from the denition of a Poisson process
in ch. 9.1.1, see (9.1). There are simple generalizations of the Poisson process where
the variance dieres from the mean (see ch. 9.4.4).
195
Poisson processes II
Let (t) be a Poisson process with arrival rate `. What is the probability that it
jumped exactly once between t and t?
Given that by def. 12,
c
A[:I]
(A[tt[)
n
a!
is the probability that jumped : times by
t. i.e. (t) = :. this probability is given by 1 ( (t) = (t) + 1) = c
A[tt[
`[t t] .
Note that this is a function which is non-monotonic in time t.
Figure 41 The probability of one jump by time t
How can the expression 1 ( (t) = (t) + 1) = c
A[tt[
`[t t] be reconciled with
the usual description of a Poisson process, as e.g. in (9.2), where it says that the
probability of a jump is given by `dt? When we look at a small time intervall dt.
we can neglect c
Aot
as it is "very close to one" and we get 1 ( (t) (t) = 1) =
1 (d (t) = 1) = `dt. As over a very small instant dt a Poisson process can either jump
or not jump (it can not jump more than once in a small instant dt), the probability
of no jump is therefore 1 (d (t) = 0) = 1 `dt.
What is the length of time between two jumps? Clearly, we do not know as jumps
are random. What is the probability that the length of time between two jumps is
smaller or equal to ? The probability that there is no jump by t is given, following
(9.1), by 1 [ (t) = (t)] = c
A[tt[
. The probability that there is at least one jump
by t is therefore 1 [ (t) (t)] = 1 c
A[tt[
. The latter is the cumulative density
function of the exponential distribution. Hence, the length t t between two jumps
is exponentially distributed with density `c
A[tt[
.
Poisson process III
Following the same structure as with Brownian motion, we now look at the geo-
metric Poisson process describing TFP growth (9.43). The solution was given in
(9.44) which reads, slightly generalized with the perspective of t and initial condition
t
. (t) =
t
c
j[tt[[q(t)q(t)[ ln(1o)
. What is the expected TFP level in t?
196
Applying the expectation operator gives
1
t
(t) =
t
c
j[tt[q(t) ln(1o)
1
t
c
q(t) ln(1o)
(9.55)
where, as in (9.52), we split the exponential growth term into its deterministic and
stochastic part. To proceed, we nee the following
Lemma 9 (taken from Posch and Wlde, 2006) Assume that we are in t and form
expectations about future arrivals of the Poisson process. The expected value of c
Iq(t)
,
conditional on t where (t) is known, is
1
t
(c
Iq(t)
) = c
Iq(t)
c
(c
I
1)A(tt)
. t t. c. / = co::t.
Note that for integer /, these are the raw moments of c
q(t)
.
Proof. We can trivially rewrite c
Iq(t)
= c
Iq(t)
c
I[q(t)q(t)[
. At time t. we know
the realization of (t) and therefore 1
t
c
Iq(t)
= c
Iq(t)
1
t
c
I[q(t)q(t)[
. Computing this
expectation requires the probability that a Poisson process jumps : times between t
and t. Formally,
1
t
c
I[q(t)q(t)[
=
1
a=0
c
Ia
c
A(tt)
(`(t t))
a
:!
=
1
a=0
c
A(tt)
(c
I
`(t t))
a
:!
= c
(c
I
1)A(tt)
1
a=0
c
A(tt)(c
I
1)A(tt)
(c
I
`(t t))
a
:!
= c
(c
I
1)A(tt)
1
a=0
c
c
I
A(tt)
(c
I
`(t t))
a
:!
= c
(c
I
1)A(tt)
.
where
c
A:
(At)
n
a!
is the probability of (t) = : and
1
a=0
c
c
I
A(:I)
(c
I
A(tt))
n
a!
= 1 denotes
the summation of the probability function over the whole support of the Poisson
distribution which was used in the last step. For a generalization of this lemma, see
ex. 7.
To apply this lemma to our case 1
t
c
q(t) ln(1o)
. we set c = c and / = ln (1 +o) .
Hence, 1
t
c
q(t) ln(1o)
=c
ln(1o)q(t)
c
(c
ln(1+)
1)A[tt[
=c
q(t) ln(1o)
c
oA[tt[
=c
q(t) ln(1o)oA[tt[
and, inserted in (9.55), the expected TFP level is
1
t
(t) =
t
c
j[tt[q(t) ln(1o)
c
q(t) ln(1o)oA[tt[
=
t
c
(joA)[tt[
.
This is an intuitive result: The growth rate of the expected TFP level is driven both by
the deterministic growth component of the SDE (9.43) for TFP and by the stochastic
part. The growth rate of expected TFP is higher, the higher the determinist part,
the q. the more often the Poisson process jumps on average (a higher arrival rate `)
and the higher the jump (the higher o).
This conrms formally what was already visible in and informally discussed after
g. 39 of ch. 9.1.2: A Poisson process as a source of uncertainty in a SDE implies
that average growth is not just determined by the deterministic part of the SDE (as
is the case when Brownian motion constitutes the disturbance term) but also by the
Poisson process itself. For a positive o. average growth is higher, for a negative o. it
is lower.
197
9.4.3 Martingales
Martingale is an impressive word for a simple concept. Here is a simplied denition
which is sucient for our purposes. More complete denitions (in a technical sense)
are in ksendal (1998) or Ross (1996).
Denition 13 A stochastic process r (t) is a martingale if, being in t today, the
expected value of r at some future point t in time equals the current value of r.
1
t
r (t) = r (t) . t _ t. (9.56)
As the expected value of r (t) is r (t) . 1
t
r (t) = r (t) . this can easily be rewritten
as 1
t
[r (t) r (t)] = 0. This is identical to saying that r (t) is a martingale if the
expected value of its increments between now t and somewhere at t in the future
is zero. This denition and the denition of Brownian motion imply that Brownian
motion is a martingale.
In what follows, we will use the martingale properties of certain processes rela-
tively frequently, e.g. when computing expected growth rates. Here are now some
fundamental examples for martingales.
Brownian motion
First, look at Brownian motion where we have a central result useful for many
applications where expectations and other moments are computed. It states that
_
t
t
, (. (:) . :) d. (:), where , (.) is some function and . (:) is Brownian motion, is a
martingale (Corollary 3.2.6, Oksendal, 1998, p. 33),
1
t
_
t
t
, (. (:) . :) d. (:) = 0. (9.57)
Poisson uncertainty
A similar fundamental result for Poisson processes exists. We will use in what
follows the martingale property of various expressions containing Poisson uncer-
tainty. These expressions are identical to or special cases of
_
t
t
, ( (:) . :) d (:)
`
_
t
t
, ( (:) . :) d:. of which Garcia and Griego (1994, theorem 5.3) have shown that
it is a martingale indeed,
1
t
__
t
t
, ( (:) . :) d (:) `
_
t
t
, ( (:) . :) d:
_
= 0. (9.58)
As always, ` is the (constant) arrival rate of (:).
198
9.4.4 A more general approach to computing moments
When we want to understand moments of some stochastic process, we can proceed in
two ways. Either, a SDE is expressed in its integral version, expectations operators
are applied and the resulting deterministic dierential equation is solved. Or, the
SDE is solved directly and then expectation operators are applied. We already saw
examples for the second approach in ch. 9.4.2. We will now follow the rst way as
this is generally the more exible one. We rst start with examples for Brownian
motion processes and then look at cases with Poisson uncertainty.
The drift and variance rates for Brownian motion
We will now return to our rst SDE in (9.3), dr(t) = cdt + /d. (t) . and want to
understand why c is called the drift term and /
2
the variance term. To this end, we
compute the mean and variance of r (t) for t t.
We start by expressing (9.3) in its integral representation as in ch. 9.1.3. This
gives
r (t) r(t) =
_
t
t
cd: +
_
t
t
/d.(:) = c [t t] +/ [.(t) . (t)] . (9.59)
The expected value 1
t
r (t) is then simply
1
t
r(t) = r (t) +c [t t] +/1
t
[.(t) . (t)] = r (t) +c [t t] . (9.60)
where the second step used the fact that the expected increment of Brownian motion
is zero. As the expected value 1
t
r(t) is a deterministic variable, we can compute
the usual time derivative and nd how the expected value of r (t), being today in t,
changes the further the point t lies in the future, d1
t
r(t),dt = c. This makes clear
why c is referred to as the drift rate of the random variable r (.) .
Let us now analyse the variance of r (t) where we also start from (9.59). The
variance can from (7.4) be written as var
t
r(t) = 1
t
r
2
(t) [1
t
r(t)]
2
. In contrast
to the term in (7.4) we need to condition the variance on t: If we computed the
variance of r (t) from the perspective of some earlier t. the variance would dier -
as will become very clear from the expression we will see in a second. Applying the
expression from (7.4) also shows that we can look at any r (t) as a normal random
variable: Whether r (t) is described by a stochastic process or by some standard
description of a random variable, an r (t) for any x future point t in time has some
distribution with corresponding moments. This allows us to use standard rules for
computing moments. Computing rst 1
t
r
2
(t) gives by inserting (9.59)
1
t
r
2
(t) = 1
t
_
[r(t) +c [t t] +/ [.(t) . (t)]]
2
_
= 1
t
_
[r(t) +c [t t]]
2
+ 2 [r(t) +c (t t)] / [.(t) . (t)] + [/ (.(t) . (t))]
2
_
= [r(t) +c [t t]]
2
+ 2 [r(t) +c (t t)] /1
t
[.(t) . (t)] +1
t
_
[/ (.(t) . (t))]
2
_
= [r(t) +c [t t]]
2
+/
2
1
t
_
[.(t) . (t)]
2
_
. (9.61)
199
where the last equality used again that the expected increment of Brownian motion is
zero. As [1
t
r(t)]
2
= [r (t) +c [t t]]
2
from (9.60), inserting (9.61) into the variance
expression gives var
t
r(t) = /
2
1
t
_
[.(t) . (t)]
2
_
.
Computing the mean of the second moment gives
1
t
_
[.(t) . (t)]
2
_
= 1
t
_
.
2
(t) 2.(t). (t) +.
2
(t)
_
= 1
t
_
.
2
(t)
_
.
2
(t) = var
t
. (t) .
where we used that . (t) is known in t and therefore non-stochastic, that 1
t
.(t) =
.(t) and equation (7.4) again. We therefore found that var
t
r(t) = /
2
var
t
. (t) . the
variance of r (t) is /
2
times the variance of . (t) . As the latter variance is given by
var
t
. (t) = t t. given that we focus on Brownian motion with standard normally
distributed increments, we found
c:
t
r(t) = /
2
[t t] .
This equation shows why / is the variance parameter for r (.) and why /
2
is called
its variance rate. The expression also makes clear why it is so important to state the
current point in time, t in our case. If we were further in the past or t is further in
the future, the variance would be larger.
Expected returns - Brownian motion
Imagine an individual owns wealth c that is allocated to ` assets such that shares
in wealth are given by o
i
= c
i
,c. The price of each asset follows a certain pricing rule,
say geometric Brownian motion, and lets assume that total wealth of the household,
neglecting labour income and consumption expenditure, evolves according to
dc (t) = c (t)
_
:dt +
.
i=1
o
i
o
i
d.
i
(t)
.
where : =
.
i=1
o
i
:
i
. This is in fact the budget constraint (with n c = 0) which
will be derived and used in ch. 10.4 on capital asset pricing. Note that Brownian
motions .
i
are correlated, i.e. d.
i
d.
)
= j
i)
dt as in (9.15). What is the expected return
and the variance of holding such a portfolio, taking o
i
and interest rates and variance
parameters as given?
Using the same approach as in the previous example we nd that the expected
return is simply given by :. This is due to the fact that the mean of the BMs .
i
(t)
are zero. The variance of c (t) is - coming soon -
Expected returns - Poisson
Let us now compute the expected return of wealth when the evolution of wealth is
described by the budget constraint in (9.32), dc (t) =: (t) c (t) +n(t) j (t) c (t) dt+
,c (t) d (t). When we want to do so, we rst need to be precise about what we mean
by the expected return. We dene it as the growth rate of the mean of wealth when
consumption expenditure and labour income are identical, i.e. when total wealth
changes only due to capital income.
200
Using this denition, we rst compute the expected wealth level at some future
point in time t. Expressing this equation in its integral representation as in ch. 9.1.3
gives
c (t) c (t) =
_
t
t
: (:) c (:) +n(:) j (:) c (:) d: +
_
t
t
,c (:) d (:) .
Applying the expectations operator yields
1
t
c (t) c (t) = 1
t
_
t
t
: (:) c (:) +n(:) j (:) c (:) d: +1
t
_
t
t
,c (:) d (:)
=
_
t
t
1
t
: (:) 1
t
c (:) d: +,`
_
t
t
1
t
c (:) d:.
The second equality used an ingredient of the denition of the expected return, i.e.
n(:) = j (:) c (:) . that : (:) is independent of c (:) and the martingale result of (9.58).
When we dene the mean of c (:) from the perspective of t by j(:) = 1
t
c (:) . this
equation reads
j(t) c (t) =
_
t
t
1
t
: (:) j(:) d: +,`
_
t
t
j(:) d:.
Computing the derivative with respect to time t gives
_ j(t) = 1
t
: (t) j(t) +`,j(t) =
_ j(t)
j(t)
= 1
t
: (t) +,`. (9.62)
Hence, the expected return is given by 1
t
: (t) +,`.
Expected growth rate
Finally, we ask what the expected growth rate and the variance of the price (t) is
when it follows the geometric Poisson process known from (9.31), d (t) = c (t) dt +
, (t) d (t) .
The expected growth rate can be dened as 1
t
(t)(t)
(t)
. Given that (t) is known in
t. we can write this expected growth rate as
1
I
(t)(t)
(t)
where expectations are formed
only with respect to the future price (t) . Given this expression, we can follow the
usual approach. The integral version of the SDE, applying the expectations operator,
is
1
t
(t) (t) = c1
t
_
t
t
(:) d: +,1
t
_
t
t
(:) d (:) .
Pulling the expectations operator inside the integral, using the martingale result from
(9.58) and dening j(:) = 1
t
(:) gives
j(t) (t) = c
_
t
t
j(:) d: +,`
_
t
t
j(:) d:.
The time-t derivative gives _ j(t) = cj(t) + ,`j(t) which shows that the growth
rate of the mean of the price is given by c + ,`. This is, as just discussed, also the
expected growth rate of the price (t) .
We now turn to the variance of (t) . - coming soon -
201
Disentangling the mean and variance of a Poisson process
Consider the SDE d (t) , (t) = cdt + i,d (t) where the arrival rate of (t) is
given by `,i. The mean of (t) is 1
t
(t) = (t) exp
(cAo)(tt)
and thereby indepen-
dent of i. The variance, however, is given by var
t
1
(t) = [1
t
(t)]
2
_
exp
Aio
2
[tt[
1
_
.
A mean preserving spread can thus be achieved by an increase of i : This increases
the randomly occurring jump i, and reduces the arrival rate `,i - the mean remains
unchanged, the variance increases.
13
Computing the mean step-by-step for a Poisson example
Let us now consider some stochastic process A (t) described by a dierential equa-
tion and ask what we know about expected values of A (t) . where t lies in the future,
i.e. t t. We take as rst example a stochastic process similar to (9.8). We take as
an example the specication for total factor productivity ,
d(t)
(t)
= cdt +,
1
d
1
(t) ,
2
d
2
(t) . (9.63)
where c and ,
i
are positive constants and the arrival rates of the processes are given
by some constant `
i
0. This equation says that TFP increases in a deterministic
way at the constant rate c (note that the left hand side of this dierential equation
gives the growth rate of (t)) and jumps at random points in time. Jumps can be
positive when d
1
= 1 and TFP increases by the factor ,
1
. i.e. it increases by ,
1
%.
or negative when d
2
= 1. i.e. TFP decreases by ,
2
%.
The integral version of (9.63) reads (see ch. 9.1.3)
(t) (t) =
_
t
t
c(:) d: +
_
t
t
,
1
(:) d
1
(:)
_
t
t
,
2
(:) d
2
(:)
= c
_
t
t
(:) d: +,
1
_
t
t
(:) d
1
(:) ,
2
_
t
t
(:) d
2
(:) . (9.64)
When we form expectations, we obtain
1
t
(t) (t) = c1
t
_
t
t
(:) d: +,
1
1
t
_
t
t
(:) d
1
(:) ,
2
1
t
_
t
t
(:) d
2
(:)
= c1
t
_
t
t
(:) d: +,
1
`
1
1
t
_
t
t
(:) d: ,
2
`
2
1
t
_
t
t
(:) d:.
(9.65)
13
This is taken from Sennewald and Wlde (2006). An alternative is provided by Steger (2005)
who uses two symmetric Poisson processes instead of one here. He obtains higher risk at an invariant
mean by increasing the symmetric jump size.
202
where the second equality used the martingale result from ch. 9.4.3, i.e. the expression
in (9.58). Pulling the expectations operator into the integral gives
14
1
t
(t) (t) = c
_
t
t
1
t
(:) d: +,
1
`
1
_
t
t
1
t
(:) d: ,
2
`
2
_
t
t
1
t
(:) d:.
When we nally dene :
1
(t) = 1
t
(t) . we obtain a deterministic dierential equa-
tion that describes the evolution of the expected value of (t) from the perspective
of t. We rst have the integral equation
:
1
(t) (t) = c
_
t
t
:
1
(:) d: +,
1
`
1
_
t
t
:
1
(:) d: ,
2
`
2
_
t
t
:
1
(:) d:
which we can then dierentiate with respect to time t by applying the rule for dier-
entiating integrals from (4.9),
_ :
1
(t) = c:
1
(t) +,
1
`
1
:
1
(t) ,
2
`
2
:
1
(t)
= (c +,
1
`
1
,
2
`
2
) :
1
(t) . (9.66)
We now immediately see that TFP does not increase in expected terms, more
precisely 1
t
(t) = (t) . if c +,
1
`
1
= ,
2
`
2
. Economically speaking, if the increase
in TFP through the deterministic component c and the stochastic component ,
1
is destroyed on average by the second stochastic component ,
2
. TFP does not
increase.
9.5 Exercise to chapter 9 and further reading
Mathematical background
There are many textbooks on dierential equations, Ito calculus, change of vari-
able formulas and related aspects in mathematics. One that is widely used is k-
sendal (1998). A more technical approach is presented in Protter (1995). A classic
mathematical reference is Gihman and Skorohod (1972). A special focus with a de-
tailed formal analysis of SDEs with Poisson processes can be found in Garcia and
Griego (1994). They also provide solutions of SDEs and the background for com-
puting moments of stochastic processes. Further solutions, applied to option pricing,
are provided by Das and Foresi (1996). The CVF for the combined Poisson-diusion
setup in lemma 8 is a special case of the expression in Sennewald (2006) which in
turn is based on ksendal and Sulem (2005). ksendal and Sulem present CVFs for
more general Levyprocesses of which the SDE (9.22) is a very simple special case.
A less technical background
14
The integral and the expectations operator can be exchanged when, under a technical condition,
both the expected integral and the integral of the expected expression exist. The expected integral
exists by assumption as otherwise our dierential equation would be specied in a meaningless way.
The existence of the integral of the expected expression will be shown by computing it. The existence
proof is therefore an ex-post proof. We are grateful to Ken Sennewald for discussions of this issue.
203
A very readable introduction to stochastic processes is by Ross (1993, 1996). He
writes in the introduction to the rst edition of his 1996 book that his text is a
nonmeasure theoretic introduction to stochastic processes. This makes this book
highly accessible for economists.
An introduction with many examples from economics is Dixit and Pindyck (1994).
See also Turnovsky (1997, 2000) for many applications. Brownian motion is treated
extensively in Chang (2004).
The CVF for Poisson processes is most easily accessible in Sennewald (2006) or
Sennewald and Wlde (2006). Sennewald (2006) provides the mathematical proofs,
Sennewald and Wlde (2006) has a focus on applications.
How to present technologies in continuous time
There is a tradition in economics starting with Eaton (1981) where output is rep-
resented by a stochastic dierential equation as presented in ch. 9.3.1. This and
similar representations of technologies are used by Epaulart and Pommeret (2003),
Pommeret and Smith (2005), Turnovsky and Smith (2006), Turnovsky (2000), Chat-
terjee, Giuliano and Turnovsky (2004), and many others. It is well known (see e.g.
footnote 4 in Grinols and Turnovsky (1998)) that this implies the possibility of nega-
tive 1 . An alternative where standard Cobb-Douglas technologies are used and TFP
is described by a SDE to this approach is presented in Wlde (2005) or Nolte and
Wlde (2007).
Application of Poisson processes in economics
The Poisson process is widely used in nance (early references are Merton, 1969,
1971) and labour economics (in matching and search models, see e.g. Pissarides,
2000, Burdett and Mortenson, 1998 or Moscarini, 2005). See also the literature
on the real options approach to investment (McDonald and Siegel, 1986, Dixit and
Pindyck, 1994, Chen and Funke 2005 or Guo et al., 2005). It is also used in growth
models (e.g. quality ladder models la Aghion and Howitt, 1992 or Grossman and
Helpman, 1991), in analyzes of business cycles in the natural volatility tradition (e.g.
Wlde, 2005), contract theory (e.g. Guriev and Kvasov, 2005), in the search approach
to monetary economics (e.g. Kiyotaki and Wright, 1993 and subsequent work) and
many other areas. Further examples include Toche (2005), Steger (2005), Laitner and
Stolyarov (2004), Farzin et al. (1998), Hassett and Metcalf (1999), Thompson and
Waldo (1994), Palokangas (2003, 2005) and Venegas-Martnez (2001).
One Poisson shock aecting many variables (as stressed in lemma 7) was used by
Aghion and Howitt (1992) in their famous growth model. When deriving the budget
constraint in Appendix 1 of Wlde (1999a), it is taken into consideration that a jump
in the technology level aects both the capital stock directly as well as its price. Other
examples include the natural volatility papers by Wlde (2002, 2005) and Posch and
Wlde (2006).
204
Exercises Chapter 9
Stochastic dierential equations and rules
for dierentiating
1. Dierentials of functions of stochastic processes I
Assume r (t) and (t) are two correlated Wiener processes. Compute d [r (t) (t)],
d [r (t) , (t)] and d ln r (t) .
2. Dierentials of functions of stochastic processes II
(a) Show that with 1 = 1 (r. ) = r and dr = ,
a
(r. ) dt + q
a
(r. ) d
a
and d = ,
j
(r. ) dt + q
j
(r. ) d
j
, where
a
and
j
are two independent
Poisson processes, d1 = rd +dr.
(b) Does this also hold for two Wiener processes
a
and
j
?
3. Correlated jump processes (exam)
Let
i
(t) for i = 1. 2. 3 be three independent Poisson processes. Dene two
jump processes by
a
(t) =
1
(t) +
2
(t) and
j
(t) =
1
(t) +
S
(t) . Let labour
productivity in sector A and 1 be driven by
d(t) = cdt +,d
a
(t) .
d1(t) = dt +od
j
(t) .
Let GDP in a small open economy (with internationally given constant prices
j
a
and j
j
) be given by
1 (t) = j
a
(t) 1
a
+j
j
1(t) 1
j
.
(a) Given an economic interpretation to equations d(t) and d1(t), based on
i
(t) . i = 1. 2. 3.
(b) How does GDP evolve over time in this economy if labour allocation 1
a
and 1
j
is invariant? Express the dierential d1 (t) by using d
a
and d
j
if possible.
4. Deriving a budget constraint
Consider a household who owns some wealth c = :
1
1
+:
2
2
, where :
i
denotes
205
the number of shares held by the household and
i
is the value of the share.
Assume that the value of the share evolves according to
d
i
= c
i
i
dt +,
i
i
d
i
.
Assume further that the number of shares held by the household changes ac-
cording to
d:
i
=

i
(:
1
:
1
+:
2
:
2
+n c)
i
dt.
where
i
is the share of savings used for buying stock i.
(a) Give an interpretation (in words) to the last equation.
(b) Derive the households budget constraint.
5. Option pricing
Assume the price of an asset follows do,o = cdt + od. + ,d (as in Merton,
1976), where . is Brownian motion and is a Poisson process. This is a gen-
eralization of (9.24) where , = 0. How does the dierential equation look like
that determines the price of an option on this asset?
6. Martingales
(a) The weather tommorow will be just the same as today. Is this a martin-
gale?
(b) Let . (:) be Brownian motion. Show that 1 (t) dened by
1 (t) = exp
_
_
t
0
, (:) d. (:)
1
2
_
t
0
,
2
(:) d:
_
(9.67)
is a martingale.
(c) Show that A (t) dened by
A (t) = exp
__
t
0
, (:) d. (:)
1
2
_
t
0
,
2
(:) d:
_
is also a martingale.
7. Expectations - Poisson
Assume that you are in t and form expectations about future arrivals of the
Poisson process (t). Prove the following statement by using lemma 9: The
number of expected arrivals in the time interval [t. :] equals the number of
expected arrivals in a time interval of the length t : for any : t,
1
t
(c
I[q(t)q(c)[
) = 1(c
I[q(t)q(c)[
) = c
(c
I
1)A(tc)
. t : t. c. / = const.
Hint: If you want to cheat, look at the appendix to Posch and Wlde (2006). It
is available e.g. at www.waelde.com/publications.
206
8. Expected values
Show that the growth rate of the mean of r (t) described by the geometric
Brownian motion
dr(t) = cr (t) dt +/r (t) d. (t)
is given by c.
(a) Do so by using the integral version of this SDE and compute the increase
of the expected value of r (t).
(b) Do the same as in (a) but solve rst the SDE and compute expectations
by using this solution.
(c) Compute the covariance of r (t) and r (:) for t : _ t.
9. Expected returns
Consider the budget constraint
dc (t) = :c (t) +n c (t) dt +,c (t) d. (t) .
(a) What is the expected return for wealth? Why does this expression dier
from (9.62)?
(b) What is the variance of wealth?
10. Dierential-representation of a technology
(a) Consider the dierential-representation of a technology, d1 (t) = 1dt+
o1d. (t) . as presented in (9.33). Compute expected output of 1 (t) for
t t and the variance of 1 (t) .
(b) Consider a more standard representation by 1 (t) = (t) 1 and let TFP
follow d(t) ,(t) = qdt + od. (t) as in (9.34). What is the expected
output level 1
t
1 (t) and what is its variance?
11. Solving stochastic dierential equations I
Consider the dierential equation
dr(t) = [c (t) r (t)] dt +1
1
(t) r (t) d\
1
(t) +/
2
(t) d\
2
(t)
where \
1
and \
2
are two correlated Brownian motions.
(a) What is the solution of this dierential equation?
(b) Use Itos Lemma to show that your solution is a solution.
12. Solving stochastic dierential equations II
Look at the generalization of (9.33) where now the capital stock can change
over time, i.e. d1 (t) = 1 (t) dt +o1 (t) d. (t) . Proof that 1 (t) = ... coming
soon ... is a solution of this SDE.
207
13. Dynamic and intertemporal budget constraints - Brownian motion
Consider the dynamic budget constraint
dc (t) = (: (t) c (t) +n(t) j (t) c (t)) dt +oc (t) d. (t) .
where . (t) is Brownian motion.
(a) Show that the intertemporal version of this budget constraint, after making
a standard economic assumption, can be written as
_
1
t
c
,(t)
j (t) c (t) dt = c
t
+
_
1
t
c
,(t)
n(t) dt
where the discount factor ,(t) is given by
,(t) =
_
t
t
: (:)
1
2
o
2
d: +
_
t
t
od. (t) .
(b) Now assume we are willing to assume that intertemporal budget constraints
need to hold in expectations only and not in realizations: we require only
that agents balance at each instant expected consumption expenditure to
current nancial wealth plus expected labour income. Show that the in-
tertemporal budget constraint then simplies to
1
t
_
1
t
c
R
:
I
v(c)oc
j (t) c (t) dt = c
t
+1
t
_
1
t
c
R
:
I
v(c)oc
n(t) dt.
i.e. all variance terms drop out of the discount factor.
208
10 Dynamic programming
We now return to our main concern: How to solve maximization problems. We
rst look at optimal behaviour under Poisson uncertainty where we analyse cases for
uncertain asset prices and labour income. We then switch to Brownian motion and
look at capital asset pricing as an application.
10.1 Optimal saving under Poisson uncertainty
10.1.1 The setup
Let us consider an individual that tries to maximize her objective function that is
given by
l (t) = 1
t
_
1
t
c
j[tt[
n(c (t)) dt. (10.1)
The structure of this objective function is identical to the one we know from de-
terministic continuous time models in e.g. (5.1) or (5.28): We are in t today, the
time preference rate is j 0. instantaneous utility is given by n(c (t)) . Given the
uncertain environment the individual lives in, we now need to form expectations as
consumption in future points t in time is unknown.
When formulating the budget constraint of a household, we have now seen at
various occasions that it is a good idea to derive it from the denition of wealth
of a household. We did so in discrete time models in ch. 2.5.5 and in continuous
time models in ch. 4.3.4. Deriving the budget constraint in stochastic continuous
time models is especially important as a budget constraint in an economy where the
fundamental source of uncertainty is Brownian motion looks very dierent from one
where uncertainty stems from Poisson or Levy processes. For this rst example, we
use the budget constraint (9.32) derived in ch. 9.2.6,
dc (t) = : (t) c (t) +n(t) jc (t) dt +,c (t) d (t) . (10.2)
where the interest rate : (t) = c +: (t) , (t) was dened as the deterministic rate of
change c of the price of the asset (compare the equation for the evolution of assets
in (9.31)) plus dividend payments : (t) , (t) . We treat the price j here as a constant
(see the exercises for an extension). Following the tilde notation from (9.7), we can
express wealth ~ c (t) after a jump by
~ c (t) = (1 +,) c (t) . (10.3)
The budget constraint of this individual reects standard economic ideas about
budget constraints under uncertainty. As visible in the derivation in ch. 9.2.6, the
uncertainty for this households stems from uncertainty about the evolution of the
price (reected in ,) of the asset she saves in. No statement was made about the
evolution of the wage n(t) . Hence, we take n(t) here as parametric, i.e. if there are
stochastic changes, they all come as a surprise and are therefore not anticipated. The
household does take into consideration, however, the uncertainty resulting from the
209
evolution of the price (t) of the asset. In addition to the deterministic growth rate c
of (t) . (t) changes in a stochastic way by jumping occasional by , percent (again,
see (9.31)). The returns to wealth c (t) are therefore uncertain and are composed of
the usual : (t) c (t) term and the stochastic ,c (t) term.
10.1.2 Three dynamic programming steps
We will solve this problem as before by dynamic programming methods. Note, how-
ever, that it is not obvious whether the above problem can be solved by dynamic
programming methods. In principle, a proof is required that dynamic programming
indeed yields necessary and sucient conditions for an optimum. While proofs exist
for bounded instantaneous utility function n(t) . such a proof did not exist until re-
cently for unbounded utility functions. Sennewald (2006) extends the standard proofs
and shows that dynamic programming can also be used for unbounded utility func-
tions. We can therefore follow the usual three-step approach to dynamic programming
here as well. The result of this section combine
The rst tool we need to derive rules for optimal behavior is the Bellman equation.
Dening the optimal program as \ (c) = max
fc(t)g
l (t) subject to a constraint, this
equation is given by (see Sennewald and Wlde, 2006, or Sennewald, 2006)
j\ (c (t)) = max
c(t)
_
n(c (t)) +
1
dt
1
t
d\ (c (t))
_
. (10.4)
The Bellman equation has this basic form for most maximization problems in con-
tinuous time. It can therefore be taken as the starting point for other maximization
problems as well, independently e.g. of whether uncertainty is driven by Poisson
processes, Brownian motion or Levy processes. We will see examples for related
problems later (see ch. 10.1.4, ch. 10.2.2 or ch. 10.3.2) and discuss then how to ad-
just certain features of this general Bellman equation. In this equation, the variable
c (t) represents the state variable, in our case wealth of the individual.
Given the general form of the Bellman equation in (10.4), we need to compute the
dierential d\ (c (t)) . Given the evolution of c (t) in (10.2) and the CVF from (9.18),
we nd
d\ (c (t)) = \
0
(c) :c +n jc dt +\ (c +,c) \ (c) d.
In contrast to the CVF notation in e.g. (9.18), we use here and in what follows
simple derivative signs like \
0
(c) as often as possible in contrast to e.g. \
o
(c) .
This is possible as long as the functions, like the value function \ (c) here, have one
argument only. Forming expectations about d\ (c (t)) is easy and they are given by
1
t
d\ (c (t)) = \
0
(c) :c +n jc dt +\ (~ c) \ (c) 1
t
d.
The rst term, the dt-term is known in t: The current state c (t) and all prices
are known and the shadow price \
0
(c) is therefore also known. As a consequence,
210
expectations need to be applied only to the d-term. The rst part of the d-term,
the expression \ ((1 + ,) c) \ (c) is also known in t as again c (t), parameters and
the function \ are all non-stochastic. We therefore only have to compute expectations
about d. From (9.54), we know that 1
t
[ (t) (t)] = `[t t] . Now replace (t)
(t) by d and t t by dt and nd 1
t
d = `dt. The Bellman equation therefore reads
j\ (c) = max
c(t)
n(c (t)) +\
0
(c) [:c +n jc] +`[\ ((1 + ,) c) \ (c)] . (10.5)
Note that forming expectations the way just used is somewhat oppy. Doing it
in the more stringent way introduced in ch. 9.4.4 would, however, lead to identical
results.
The rst-order condition is
n
0
(c) = \
0
(c) j. (10.6)
As always, (current) utility from an additional unit of consumption n
0
(c) must equal
(future) utility from an additional unit of wealth \
0
(c), multiplied by the price j of
the consumption good, i.e. by the number of units of wealth for which one can buy
one unit of the consumption good.
In order to understand the evolution of the marginal value \
0
(c) of the optimal
program, i.e. the evolution of the costate variable, we need to (i) compute the partial
derivative of the maximized Bellman equation with respect to assets and (ii) compute
the dierential d\
0
(c) by using the CVF and insert the partial derivative into this
expression. These two steps corresponds to the two steps in DP2 in the deterministic
continuous time setup of ch. 6.1.
(i) In the rst step, we state the maximized Bellman equation from (10.5) as the
Bellman equation where controls are replaced by their optimal values,
j\ (c) = n(c (c)) +\
0
(c) [:c +n jc (c)] +`[\ ((1 + ,) c) \ (c)] .
We then compute again the derivative with respect to the state - as in discrete time
and deterministic continuous time setups - as this gives us an expression for the
shadow price \
0
(c). In contrast to the previous emphasis on Itos Lemmas and CVFs,
we can use for this step standard rules from algebra as we compute the derivative for
a given state c - the state variable is held constant and we want to understand the
derivative of the function \ (c) with respect to c. We do not compute the dierential
of \ (c) and ask how the value function changes as a function of a change in c.
Therefore, using the envelope theorem,
j\
0
(c) = \
0
(c) : +\
00
(c) [:c +n jc] +`[\
0
(~ c) [1 + ,] \
0
(c)] . (10.7)
We used here the denition of ~ c given in (10.3).
211
(ii) In the second step, the dierential of the shadow price \
0
(c) is computed,
given again the evolution of c (t) in (10.2),
d\
0
(c) = \
00
(c) [:c +n jc] dt + [\
0
(~ c) \
0
(c)] d. (10.8)
Finally, replacing \
00
(c) [:c +n jc] in (10.8) by the same expression from (10.7)
gives
d\
0
(c) = (j :) \
0
(c) `[\
0
(~ c) [1 +,] \
0
(c)] dt +\
0
(~ c) \
0
(c) d.
Finally, we can replace the marginal value by marginal utility from the rst-
order condition (10.6). In this step, we employ that j is constant and therefore
d\
0
(c) = j
1
dn
0
(c). Hence, the Keynes-Ramsey rule describing the evolution of
marginal utility reads
dn
0
(c) = (j :) n
0
(c) `[n
0
(~ c) [1 +,] n
0
(c)] dt +n
0
(~ c) n
0
(c) d. (10.9)
Note that the constant price j dropped out. This rule shows how marginal utility
changes in a deterministic and stochastic way.
10.1.3 The solution
The dynamic programming approach provided us with an expression in (10.9) which
describes the evolution of marginal utility from consumption. While there is a one-to-
one mapping from marginal utility to consumption which would allow some inferences
about consumption from (10.9), it would nevertheless be more useful to have a Keynes-
Ramsey rule for optimal consumption itself.
The evolution of consumption
If we want to know more about the evolution of consumption, we can use the CVF
formula as follows. Let , (.) be the inverse function for n
0
. i.e. , (n
0
(c)) = c. and
apply the CVF to , (n
0
(c)) . This gives
d, (n
0
(c)) = ,
0
(n
0
(c)) (j :) n
0
(c) `[n
0
(~ c) [1 + ,] n
0
(c)] dt+, (n
0
(~ c)) , (n
0
(c)) d.
As , (n
0
(c)) = c. we know that , (n
0
(~ c)) = ~ c and ,
0
(n
0
(~ c)) =
o)(&
0
(c))
o&
0
(c)
=
oc
o&
0
(c)
=
1
&
00
(c)
.
Hence,
dc =
1
n
00
(c)
(j :) n
0
(c) `[n
0
(~ c) [1 + ,] n
0
(c)] dt +~ c c d =
n
00
(c)
n
0
(c)
dc =
_
: j +`
_
n
0
(~ c)
n
0
(c)
[1 +,] 1
__
dt
n
00
(c)
n
0
(c)
~ c c d. (10.10)
This is the Keynes-Ramsey rule that describes the evolution of consumption under
optimal behaviour for a household that faces stochastic interest rate uncertainty re-
sulting from Poisson processes. This equation is useful to understand e.g. economic
212
uctuations in a natural volatility setup (see e.g. Matsuyama, 1999, Francois and
Lloyd-Ellis, 2003 or Wlde, 1999, 2005). It corresponds to its deterministic pendant
in (5.30) in ch. 5.5.3: By setting ` = 0 here (implying d = 0), noting that we treated
the price as a constant and dividing by dt, we obtain (5.30).
A specic utility function
Let us now assume for the instantaneous utility function the widely used constant
relative risk aversion (CRRA) instantaneous utility function,
n(c (t)) =
c (t)
1o
1
1 o
. o 0. (10.11)
Then, the Keynes-Ramsey rule becomes
o
dc
c
=
_
: j +`
_
(1 +,)
_
c
~ c
_
o
1
__
dt +o
_
~ c
c
1
_
d. (10.12)
The left-hand side gives the proportional change of consumption times o. the inverse
of the intertemporal elasticity of substitution o
1
. This corresponds to o_ c,c in the
deterministic Keynes-Ramsey rule in e.g. (5.31). Growth of consumption depends
on the right-hand side in a deterministic way on the usual dierence between the
interest rate and time preference rate plus the `term which captures the impact
of uncertainty. When we want to understand the meaning of this term, we need to
nd out whether consumption jumps up or down, following a jump of the Poisson
process. When , is positive, the household holds more wealth and consumption
should increase. Hence, the ratio c,~ c is smaller than unity and the sign of the bracket
term (1 +,)
_
c
` c
_
o
1 is qualitatively unclear. If it is positive, consumption growth is
faster in a world where wealth occasionally jumps by , percent.
The d-term gives discrete changes in the case of a jump in . It is tautological,
however: When jumps and d = 1 and dt = 0 for this small instant of the jump,
(10.12) says that odc,c on the left-hand side is given by o ~ c,c 1 on the right hand
side. As the left hand side is by denition of dc given by o [~ c c] ,c. both sides are
identical. Hence, the level of ~ c after the jump needs to be determined in an alternative
way.
10.1.4 Optimal consumption and portfolio choice
This section analyses a more complex maximization problem than the one presented
in ch. 10.1.1. In addition to the consumption-saving trade-o, it includes a portfolio
choice problem. Interestingly, the solution is much simpler to work with as ~ c can
explicitly be computed and closed form solutions can easily be found.
Consider a household that is endowed with some initial wealth c
0
0. At each
instant, the household can invest its wealth c (t) in both a risky and a safe asset. The
213
share of wealth the household holds in the risky asset is denoted by o (t). The price
1
(t) of one unit of the risky asset obeys the SDE
d
1
(t) = :
1
1
(t) dt +,
1
(t) d (t) . (10.13)
where :
1
R and , 0. That is, the price of the risky asset grows at each instant with
a xed rate :
1
and at random points in time it jumps by , percent. The randomness
comes from the well-known Poisson process (t) with arrival rate `. The price
2
(t)
of one unit of the safe asset is assumed to follow
d
2
(t) = :
2
2
(t) dt. (10.14)
where :
2
_ 0. Let the household receive a xed wage income n and spend c (t) _ 0
on consumption. Then, in analogy to ch. 9.2.6, the households budget constraint
reads
dc (t) = [o (t) :
1
+ (1 o (t)) :
2
] c (t) +n c (t) dt +,o (t) c (t) d (t) . (10.15)
We allow wealth to become negative but we could assume that debt is always covered
by the households lifetime labour income discounted with the safe interest rate :
2
,
i.e. c (t) n,:
2
.
Let intertemporal preferences of households be identical to the previous maximiza-
tion problem - see (10.1). The instantaneous utility function is again characterized
by CRRA as in (10.11), n(c(t)) = (c
1o
1) , (1 o) . The control variables of the
household are the nonnegative consumption stream c (t) and the share o (t) held
in the risky asset. To avoid a trivial investment problem, we assume
:
1
< :
2
< :
1
+`,. (10.16)
That is, the guaranteed return of the risky asset, :
1
, is lower than the return of the
riskless asset, :
2
, whereas, on the other hand, the expected return of the risky asset,
:
1
+ `,, shall be greater than :
2
. If :
1
was larger than :
2
, the risky asset would
dominate the riskless one and noone would want to hold positive amounts of the
riskless asset. If :
2
exceeded :
1
+`,. the riskless asset would dominate.
Again, the rst step of the solution of this maximization problem requires a Bell-
man equation. Dene the value function \ again as
15
\ (c (t)) = max
fc(t)g
l (t) .
The basic Bellman equation is taken from (10.4). When computing the dieren-
tial d\ (c (t)) and taking into account that there are now two control variables, the
Bellman equation reads
j\ (c) = max
c(t),0(t)
n(c) + [(o:
1
+ (1 o) :
2
) c +n c] \
0
(c) +`[\ (~ c) \ (c)] .
(10.17)
15
One can show that the value function in this example does not depend on initial time but on
initial wealth only. An ex-post proof is given in Sennewald and Wlde (2006) where an explicit
expression for the value function is provided.
214
where ~ c = (1 +o,) c denotes the post-jump wealth if at wealth c a jump in the risky
asset price occurs. The rst-order conditions which any optimal paths must satisfy
are given by
n
0
(c) = \
0
(c) (10.18)
and
\
0
(c) (:
1
:
2
) c +`\
0
(~ c) ,c = 0. (10.19)
While the rst rst-order condition equates as always marginal utility with the shadow
price, the second rst-order condition determines optimal investment of wealth into
assets 1 and 2. i.e. the optimal share o. The latter rst-order condition contains a
deterministic and a stochastic term and households hold their optimal share if these
two components just add up to zero. Assume, consistent with (10.16), that :
1
< :
2
. If
we were in a deterministic world, i.e. ` = 0. households would then only hold asset 2
as its return is higher. In a stochastic world, the lower instantaneous return on asset
1 is compensated by the fact that, as (10.13) shows, the price of this asset jumps up
occasionally by the percentage ,. Lower instantaneous returns :
1
paid at each instant
are therefore compensated for by large occasional positive jumps.
As this rst-order condition also shows, returns and jumps per se do not matter:
The dierence :
1
:
2
in returns is multiplied by the shadow price \
0
(c) of capital and
the eect of the jump size times its frequence, `,. is multiplied by the shadow price
\
0
(~ c) of capital after a jump. What matters for the household decision is therefore
the impact of holding wealth in one or the other asset on the overall value from
behaving optimally, i.e. on the value function \ (c). The channels through which
asset returns aect the value function is rst the impact on wealth and second the
impact of wealth on the value function i.e. the shadow price of wealth.
We can now immediately see why this more complex maximization problem yields
simpler solutions: Replacing in equation (10.19) \
0
(c) with n
0
(c) according to (10.18)
yields for c ,= 0
n
0
(~ c)
n
0
(c)
=
:
2
:
1
`,
. (10.20)
where ~ c denotes the optimal consumption choice for ~ c. Hence, the ratio for optimal
consumption after and before a jump is constant. If we assume e.g. a CES utility
function as in (10.11), this jump is given by
~ c
c
=
_
`,
:
2
:
1
_
1o
. (10.21)
No such result on relative consumption before and after the jump is available for the
maximization problem without a choice between a risky and a riskless asset.
Since by assumption (10.16) the term on the right-hand side is greater than 1, this
equation shows that consumption jumps upwards if a jump in the risky asset price
occurs. This result is not surprising since, if the risky asset price jumps upwards, so
does the households wealth.
215
In the next step, we compute the evolution of \
0
(c (t)), the shadow price of wealth.
Assume that \ is twice continuously dierentiable. Then, due to budget constraint
(10.15), the CVF from (9.18) yields
d\
0
(c) = [o:
1
+ (1 o) :
2
] c +n c \
00
(c) dt
+\
0
(~ c) \
0
(c) d (t) . (10.22)
Dierentiating the maximized Bellman equation yields under application of the en-
velope theorem
j\
0
(c) = [o:
1
+ (1 o) :
2
] c +n c \
00
(c)
+o:
1
+ [1 o] :
2
\
0
(c) +`\
0
(~ c) [1 + o,] \
0
(c) .
Rearranging gives
[o:
1
+ (1 o) :
2
] c +n c \
00
(c)
= j [o:
1
+ (1 o) :
2
] \
0
(c) `\
0
(~ c) [1 +o,] \
0
(c) .
Inserting this into (10.22) yields
d\
0
(c) =
_
j [o:
1
+ (1 o) :
2
] \
0
(c)
`[1 +o,] \
0
(~ c) \
0
(c)
_
dt +\
0
(~ c) \
0
(c) d (t) .
Replacing \
0
(c) by n
0
(c) following the rst-order condition (10.18) for optimal
consumption, we obtain
dn
0
(c) =
_
j [o:
1
+ (1 o) :
2
] n
0
(c)
`[1 +o,] n
0
(~ c) n
0
(c)
_
dt +n
0
(~ c) n
0
(c) d (t) .
Now applying the CVF again to , (r) = (n
0
)
1
(r) and using (10.20) leads to the
Keynes-Ramsey rule for general utility functions n,
n
00
(c)
n
0
(c)
dc =
_
o:
1
+ [1 o] :
2
j +`
_
:
2
:
1
`,
[1 +o,] 1
__
dt
n
00
(c)
n
0
(c)
~ c c d (t) .
As ~ c is also implicity determined by (10.20), this Keynes-Ramsey rule describes the
evolution of consumption under Poisson uncertainty without the ~ c term. This is the
crucial modelling advantage from introducing an additional asset into a standard
consumption-saving problem. Apart from this simplication, the structure of this
Keynes-Ramsey rule is identical to the one in (10.10) without a second asset.
A specic utility function
216
For the CRRA utility function as in (10.11), the elimination of ~ c becomes even
simpler and we obtain with (10.21)
dc (t)
c (t)
=
1
o
_
:
2
`
_
1
:
2
:
1
`,
_
j
_
dt +
_
_
`,
:
2
:
1
_
1o
1
_
d (t) .
The optimal change in consumption can thus be expressed in terms of well-known pa-
rameters. As long as the price of the risky asset does not jump, optimal consumption
grows constantly by the rate
_
:
2
`
_
1
v
2
v
1
Ao
_
j
_
,o. The higher the risk-free
interest rate, :
2
, and the lower the guaranteed interest rate of the risky asset, :
1
, the
discrete growth rate, ,, the probability of a price jump, `, the time preference rate,
j, and the risk aversion parameter, o, the higher becomes the consumption growth
rate. If the risky asset price jumps, consumption jumps as well to its new higher level
c (t) = [(`,) , (:
2
:
1
)]
1o
c (t). Here the growth rate depends positively on `, ,, and
:
1
, whereas :
2
and o have negative inuence.
10.1.5 Other ways to determine ~ c
The question of how to determine ~ c without an additional asset is in principle identical
to determining the initial level of consumption, given a deterministic Keynes-Ramsey
rule as e.g. (5.5). Whenever a jump in c following (10.12) occurs, the household
faces the issue of how to choose the initial level of consumption after the jump. In
principle, the level ~ c is therefore pinned down by some transversality condition. In
practice, the literature oers two ways, how ~ c can be determined.
One asset and idiosyncratic risk
When households determine optimal savings only, as in our setup where the only
rst-order condition is (10.6), ~ c can be determined (in principle) if we assume that
the value function is a function of wealth only - which would be the case in our
household example if the interest rate and wages did not depend on . This would
naturally be the case in idiosyncratic risk models where aggregate variables do not
depend on individual uncertainty resulting from . The rst-order condition (10.6)
then reads n
0
(c) = \
0
(c) (with j = 1). This is equivalent to saying that consumption
is a function of the only state variable, i.e. wealth c, c = c (c) . An example for c (c)
is plotted in the following gure.
N
N
c
c(c)
c
0
c
1
c
2
c
c
c
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
d = 1
N
s
s
217
Figure 42 Consumption c as a function of wealth c
As consumption c does not depend on directly, we must be at the same con-
sumption level c (c) . no matter how we got there, i.e. no matter how many jumps
took place before. Hence, if we jump from some c
1
to c
2
because d = 1. we are at
the same consumption level c (c
2
) as if we had reached c
2
smoothly without jump.
The consumption level ~ c after a jump is therefore the consumption level that belongs
to the asset level after the jump according to the policy function c (c) plotted in the
above gure, ~ c = c (c
2
). See Schlegel (2004) for a numerical implementation of this
approach.
Finding value functions
A very long tradition exists in economics where value functions are found by an
educated guess. Experience tells us - based on rst examples by Merton (1969,
1971) - how value functions generally look like. It is then possible to nd, after some
attempts, the value function for some specic problem. This then implies explicit -
so called closed-form solutions - for consumption (or any other control variable). For
the saving and investment problem in ch. 10.1.4, Sennewald and Wlde (2006, sect.
3.4) presented a value function and closed-form solutions for c ,= 0 of the form
\ (c) =

o
_
c +
&
v
2
_
1o
j
1
1 o
. c =
_
c +
n
:
2
_
. o =
_
_
`,
:
2
:
1
_1
1
_
c +
&
v
2
,c
.
Consumption is a constant share ( is a collection of parameters) out of the total
wealth, i.e. nancial wealth c plus human wealth n,:
2
(the present value of current
and future labour income). The optimal share o depends on total wealth as well, but
also on interest rates, the degree of risk-aversion and the level , of the jump of the
risky price in (10.13). Hence, it is possible to work with complex stochastic models
that allow to analyse many interesting real-world features and nevertheless end up
with explicit closed-form solutions. Many further examples exist - see ch. 10.6 on
further reading.
Finding value functions for special cases
As we have just seen, value functions and closed-form solutions can be found
for some models which have nice features. For a much larger class of models -
which are then standard models - closed form solutions can not be found for general
parameter sets. Economists then either go for numerical solutions, which preserves a
certain generality as in principle the properties of the model can be analyzed for all
parameter values, or they restrict the parameter set in a useful way. Useful means that
with some parameter restriction, value functions can be found again and closed-form
solutions are again possible.
Doing more serious maths
- coming soon -
218
10.1.6 Expected growth
Let us now try to understand the impact of uncertainty on expected growth. In order
to compute expected growth of consumption from realized growth rates (10.12), we
rewrite this equation as
odc (t) =
_
: (t) j +`
_
(1 +,)
_
c (t)
~ c (t)
_
o
1
__
c (t) dt +o ~ c (t) c (t) d.
Expressing it in its integral version as in (9.64), we obtain for t t.
o [c (t) c (t)] =
_
t
t
_
: (:) j +`
_
(1 +,)
_
c (:)
~ c (:)
_
o
1
__
c (:) d:
+o
_
t
t
~ c (:) c (:) d (:) .
Applying the expectations operator, given knowledge in t. yields
1
t
c (t) c (t) =
1
o
1
t
_
t
t
_
: (:) j +`
_
(1 +,)
_
c (:)
~ c (:)
_
o
1
__
c (:) d:
+1
t
_
t
t
~ c (:) c (:) d (:) .
Using again the martingale result from ch. 9.4.3 as already in (9.65), i.e. the ex-
pression in (9.58), we replace 1
t
_
t
t
~ c (:) c (:) d (:) by `1
t
_
t
t
~ c (:) c (:) d:.
i.e.
1
t
c (t) c (t) =
1
o
1
t
_
t
t
_
: (:) j +`
_
(1 +,)
_
c (:)
~ c (:)
_
o
1
__
c (:) d:
+`1
t
_
t
t
~ c (:) c (:) d:.
Dierentiating with respect to time t yields
d1
t
c (t) ,dt =
1
o
1
t
__
: (t) j +`
_
(1 +,)
_
c (t)
~ c (t)
_
o
1
__
c (t)
_
+`1
t
~ c (t) c (t) .
Let us now put into the perspective of time t. i.e. lets move time from t to t and
lets ask what expected growth of consumption is. This shift in time means formally
that our expectations operator becomes 1
t
and we obtain
d1
t
c (t) ,dt
c (t)
=
1
o
1
t
_
: (t) j +`
_
(1 +,)
_
c (t)
~ c (t)
_
o
1
__
+`1
t
_
~ c (t)
c (t)
1
_
.
In this step we used the fact that, due to this shift in time, c (t) is now known and
we can pull it out of the expectations operator and divide by it. Combining brackets
yields
d1
t
c (t) ,dt
c (t)
=
1
o
1
t
_
: (t) j +`
_
(1 +,)
_
c (t)
~ c (t)
_
o
1
_
+o`
_
~ c (t)
c (t)
1
__
=
1
o
1
t
_
: (t) j +`
_
(1 +,)
_
c (t)
~ c (t)
_
o
+o
~ c (t)
c (t)
1 o
__
.
219
10.2 Matching on the labour market: where value functions
come from
Value functions are widely used in matching models. Examples are unemployment
with frictions models of the Mortensen-Pissarides type or shirking models of unem-
ployment (Shapiro and Stiglitz, 1984). These value functions can be understood very
easily on an intuitive level, but they really come from a maximization problem of
households. In order to understand when value functions as the ones used in the just
mentioned examples can be used (e.g. under the assumption of no saving, or being
in a steady state), we now derive value functions in a general way and then derive
special cases used in the literature.
10.2.1 A household
Let wealth c of a household evolve according to
dc = :c +. c dt.
Wealth increases per unit of time dt by the amount dc which depends on current
savings :c + . c. Labour income is denoted by . which includes income n when
employed and unemployment benets / when unemployed, . = n. /. Labour income
follows a stochastic Poisson dierential equation as there is job creation and job
destruction. In addition, we assume technological progress that implies a constant
growth rate q of labour income. Hence, we can write
d. = q.dt d
&
+ d
b
.
where = n /. Job destruction takes place at an exogenous state-dependent
arrival rate : (.) . The corresponding Poisson process counts how often our household
moved from employment into unemployment is
&
. Job creation takes place at an
exogenous rate `(.) which is related to the matching function presented in (5.42).
The Poisson process related to the matching process is denoted by
b
. It counts how
often a household leaves her /-status, i.e. how often she found a job. As an
individual cannot loose her job when she does not have one and as nding a job
makes (in this setup) no sense for someone who has a job, both arrival rates are state
dependent. As an example, when an individual is employed, `(n) = 0. when she is
unemployed, : (/) = 0.
. n /
`(.) 0 `
: (.) : 0
Table State dependend arrival rates
Let the individual maximize expected utility 1
t
_
1
t
c
j[tt[
n(c (t)) dt. where in-
stantaneous utility is of the CES type, n(c) =
c
1
1
1o
with o 0.
220
10.2.2 The Bellman equation and value functions
The state space is described by c and .. The Bellman equation has the same structure
as in (10.4). The adjustments that need to be made here follow from the fact that we
have two state variables instead of one. Hence, the basic structure from (10.4) adopted
to our problem reads j\ (c. .) = max
fc(t)g
_
n(c) +
1
ot
1
t
d\ (c. .)
_
. The change of
\ (c. .) is, given the evolution of c and . from above and the CVF from (9.21),
d\ (c. .) = \
o
[:c +. c] +\
:
q. dt
+\ (c. . ) \ (c. .) d
&
+\ (c. . + ) \ (c. .) d
b
.
Forming expectations, remembering that 1
t
d
&
= : (.) dt and 1
t
d
b
= `(.) dt. and
dividing by dt gives the Bellman equation
j\ (c. .) = max
c
_
n(c) +\
o
[:c +. c] +\
:
q.
+: (.) [\ (c. . ) \ (c. .)] +`(.) [\ (c. . + ) \ (c. .)]
_
.
(10.23)
The value functions in the literature are all special cases of this general Bellman
equation.
Denote by l = \ (c. /) the expected present value of (optimal behaviour of a
worker) being unemployed (as in Pissarides, 2000, ch. 1.3) and by \ = \ (c. n) the
expected present value of being employed. As the probability of loosing a job for an
unemployed worker is zero, : (/) = 0. and `(/) = `. the Bellman equation (10.23)
reads
jl = max
c
n(c) +l
o
[:c +/ c] +l
:
q. +`[\ l] .
where we used that \ = \ (c. / + ) . When we assume that agents behave optimally,
i.e. we replace control variables by their optimal values, we obtain the maximized
Bellman equation,
jl = n(c) +l
o
[:c +/ c] +l
:
q/ +`[\ l] .
When we now assume that households can not save, i.e. c = :c+/. and that there
is no technological progress, q = 0. we obtain
jl = n(:c +/) +`[\ l] .
Assuming further that households are risk-neutral, i.e. n(c) = c. and that they have
no capital income, i.e. c = 0, consumption is identical to unemployment benets
c = /. If the interest rate equals the time preference rate, we obtain eq. (1.10) in
Pissarides (2000),
:l = / +`[\ l] .
10.3 Optimal saving under Brownian motion
10.3.1 The setup
Consider an individual whose budget constraint is given by
dc = :c +n jc dt +,cd..
221
The notation is as always, uncertainty stems from Brownian motion .. The individual
maximizes a utility function as given in (10.1), l (t) = 1
t
_
1
t
c
j[tt[
n(c (t)) dt. The
value function is dened by \ (c) = max
fc(t)g
l (t) . We again follow the three step
scheme for dynamic programming.
10.3.2 Three dynamic programming steps
The Bellman equation is given for Brownian motion by (10.4) as well. When a
maximization problem other than one where (10.4) is suitable is to be formulated and
solved, ve adjustments are in principle possible for the Bellman equation. First, the
discount factor j might be given by some other factor - e.g. the interest rate : when
the present value of some rm is maximized. Second, the number of arguments of
the value function needs to be adjusted to the number of state variables. Third, the
number of control variables depends on the problem that is to be solved and, fourth,
the instantaneous utility function is replaced by what is found in the objective function
- which might be e.g. instantaneous prots. Finally and obviously, the dierential
d\ (.) needs to be computed according to the rules that are appropriate for the
stochastic processes which drive the state variables.
As the dierential of the value function, following Itos Lemma in (9.13), is given
by
d\ (c) =
_
\
0
(c) [:c +n jc] +
1
2
\
00
(c) ,
2
c
2
_
dt +\
0
(c) ,cd..
forming expectations 1
t
and dividing by dt yields the Bellman equation for our specic
problem
j\ (c) = max
c(t)
_
n(c (t)) +\
0
(c) [:c +n jc] +
1
2
\
00
(c) ,
2
c
2
_
and the rst-order condition is
n
0
(c (t)) = \
0
(c) j (t) . (10.24)
(i) The derivative of the maximized Bellman equation with respect to the state
variable is (using the envelope theorem)
j\
0
= \
00
[:c +n jc] +\
0
: +
1
2
\
000
,
2
c
2
+\
00
,
2
c =
(j :) \
0
= \
00
[:c +n jc] +
1
2
\
000
,
2
c
2
+\
00
,
2
c. (10.25)
Not surprisingly, given that the Bellman equation already contains the second deriva-
tive of the value function, the derivative of the maximized Bellman equation contains
its third derivative \
000
.
222
(ii) Computing the dierential of the shadow price of wealth \
0
(c) gives, using
Itos Lemma (9.13),
d\
0
= \
00
dc +
1
2
\
000
,
2
c
2
dt
= \
00
[:c +n jc] dt +
1
2
\
000
,
2
c
2
dt +\
00
,cd..
and inserting into the partial derivative (10.25) of the maximized Bellman equation
yields
d\
0
= (j :) \
0
dt \
00
,
2
cdt +\
00
,cd.
=
_
(j :) \
0
\
00
,
2
c
_
dt +\
00
,cd.. (10.26)
As always at the end of DP2, we have a dierential equation (or dierence equation in
discrete time) which determines the evolution of \
0
(c) . the shadow price of wealth.
Assuming that the evolution of aggregate prices is independent of the evolution of
the marginal value of wealth, we can write the rst-order condition for consumption
in (10.24) as dn
0
(c) = jd\
0
+ \
0
dj. This follows e.g. from Itos Lemma (9.16) with
j
j\
0 = 0. Using (10.26) to replace d\
0
. we obtain
dn
0
(c) = j
__
(j :) \
0
\
00
,
2
c
_
dt +\
00
,cd.
+\
0
dj
=
_
(j :)
n
0
(c)
j
n
00
(c) c
0
(c) ,
2
c
_
dt +n
00
(c) c
0
(c) ,cd. +n
0
(c) dj,j.
(10.27)
where the second equality used the rst order condition n
0
(c (t)) = \
0
j (t) to re-
place \
0
and the partial derivative of this rst-order condition with respect to assets,
n
00
(c) c
0
(c) = \
00
j. to replace \
00
.
When comparing this with the expression in (10.9) where uncertainty stems from
a Poisson process, we see two common features: First, both KRRs have a stochastic
term, the d.-term here and the d-term in the Poisson case. Second, uncertainty
aects the trend term for consumption in both terms. Here, this term contains the
second derivative of the instantaneous utility function and c
0
(c) . in the Poisson case,
we have the ~ c terms. The additional dj-term here stems from the assumption that
prices are not constant. Such a term would also be visible in the Poisson case with
exible prices.
10.3.3 The solution
Just as in the Poisson case, we want a rule for the evolution of consumption here as
well. We again dene an inverse function and end up in the general case with
n
00
n
0
dc =
_
: j +
n
00
(c)
n
0
(c)
c
o
,
2
c +
1
2
n
000
(c)
n
0
(c)
[c
o
,c]
2
_
dt (10.28)
n
00
(c)
n
0
(c)
c
o
,cd. +
dj
j
.
223
With a CRRA utility function, we can replace the rst, second and third derivative
of n(c) and nd the corresponding rule
o
dc
c
=
_
: j o
c
o
c
,
2
c +
1
2
o [o + 1]
_
c
o
c
,c
_
2
_
dt (10.29)
+o
c
o
c
,cd. +
dj
j
.
10.4 Capital asset pricing
Let us again consider a typical CAP problem. This follows and extends Merton (1990,
ch. 15; 1973). The presentation is in a simplied way.
10.4.1 The setup
The basic structure of the setup is identical to before. There is an objective function
and a constraint. The objective function captures the preferences of our agent and
is described by a utility function as in (10.1). The constraint is given by a budget
constraint which will now be derived, following the principles of ch. 9.2.6.
Wealth c of households consist of a portfolio of assets i
c =
.
i=1
j
i
:
i
.
where the price of an asset is denoted by j
i
and the number of shares held by :
i
. The
total number of assets is given by `. Let us assume that the price of an asset follows
geometric Brownian motion,
dj
i
j
i
= c
i
dt +o
i
d.
i
. (10.30)
where each price is driven by its own drift parameter c
i
and its own variance para-
meter o
i
. Uncertainty results from Brownian motion .
i
which is also specic for each
asset. These parameters are exogenously given to the household but would in a gen-
eral equilibrium setting be determined by properties of e.g. technologies, preferences
and other parameters of the economy.
Households can buy or sell assets by using a share
i
of their savings,
d:
i
=
1
j
i
i
_
.
i=1
:
i
:
i
+n c
_
dt. (10.31)
When savings are positive and a share
i
is used for asset i. the number of stocks held
in i increases. When savings are negative and
i
is positive, the number of stocks i
decreases.
The change in the households wealth is given by dc =
.
i=1
d (j
i
:
i
) . The wealth
held in one asset changes according to
d (j
i
:
i
) = j
i
d:
i
+:
i
dj
i
=
i
_
.
i=1
:
i
:
i
+n c
_
dt +:
i
dj
i
.
224
The rst equality uses Itos Lemma from (9.16), taking into account that second
derivatives of 1 (.) = j
i
:
i
are zero and that d:
i
in (10.31) is deterministic and there-
fore dj
i
d:
i
= 0. Using the pricing rule (10.30) and the fact that shares add to unity,
.
i=1
i
= 1, the budget constraint of a household therefore reads
dc =
_
.
i=1
:
i
:
i
+n c
_
dt +
.
i=1
:
i
j
i
[c
i
dt +o
i
d.
i
]
=
_
.
i=1
:
i
j
i
:
i
j
i
+:
i
j
i
c
i
+n c
_
dt +
.
i=1
:
i
j
i
o
i
d.
i
=
_
.
i=1
c
i
_
:
i
j
i
+c
i
_
+n c
_
dt +
.
i=1
c
i
o
i
d.
i
.
Now dene o
i
as always as the share of wealth held in asset i. o
i
= c
i
,c. Then, by
denition, c =
.
i=1
o
i
c and shares add up to unity,
.
i=1
o
i
= 1. We rewrite this for
later purposes as
o
.
= 1
.1
i=1
o
i
(10.32)
Dene further the interest rate for asset i and the interest rate of the market portfolio
by
:
i
=
:
i
j
i
+c
i
. : =
.
i=1
o
i
:
i
. (10.33)
This gives us the budget constraint,
dc =
_
c
.
i=1
o
i
:
i
+n c
_
dt +c
.
i=1
o
i
o
i
d.
i
= :c +n c dt +c
.
i=1
o
i
o
i
d.
i
. (10.34)
10.4.2 Optimal behaviour
Let us now consider an agent who behaves optimally when choosing her portfolio
and in making her consumption-saving decision. We will not go through all steps to
derive a Keynes-Ramsey rule as asset pricing requires only the Bellman equation and
rst-order conditions.
The Bellman equation
The Bellman equation is given by (10.4), i.e. j\ (c) = max
c(t),0
.
(t)
_
n(c (t)) +
1
ot
1
t
d\ (c)
_
.
Hence, we need again the expected change of the value of one unit of wealth. With
one state variable, we simply apply Itos Lemma from (9.11) and nd
1
dt
1
t
d\ (c) =
1
dt
1
t
_
\
0
(c) dc +
1
2
\
00
(c) [dc]
2
_
. (10.35)
In a rst step required to obtain the explicit version of the Bellman equation,
we compute the square of dc. It is given, taking (9.12) into account, by [dc]
2
=
c
2
_
.
i=1
o
i
o
i
d.
i
2
. When we compute the square of the sum, the expression for the
product of Brownian motions in (9.15) becomes important as correlation coecients
225
need to be taken into consideration. Denoting the covariances by o
i)
= o
i
o
)
j
i)
, we
get
[dc]
2
= c
2
[o
1
o
1
d.
1
+o
2
o
2
d.
2
+... +o
a
o
a
d.
a
]
2
= c
2
_
o
2
1
o
2
1
dt +o
1
o
1
o
2
o
2
j
12
dt +... +o
2
o
2
o
1
o
1
j
12
dt +o
2
2
o
2
2
dt +... +...
= c
2
.
)=1
.
i=1
o
i
o
)
o
i)
dt. (10.36)
Now rewrite the sum in (10.36) as follows
.
)=1
.
i=1
o
)
o
i
o
i)
=
.1
)=1

.
i=1
o
)
o
i
o
i)
+
.
i=1
o
i
o
.
o
i.
=
.1
)=1

.1
i=1
o
)
o
i
o
i)
+
.
i=1
o
i
o
.
o
i.
+
.1
)=1
o
.
o
)
o
.)
=
.1
)=1

.1
i=1
o
)
o
i
o
i)
+
.1
i=1
o
i
o
.
o
i.
+
.1
)=1
o
.
o
)
o
.)
+o
2
.
o
2
.
=
.1
)=1

.1
i=1
o
)
o
i
o
i)
+ 2
.1
i=1
o
i
o
.
o
i.
+o
2
.
o
2
.
As the second term, using (10.32), can be written as
.1
i=1
o
i
o
.
o
i.
=
_
1
.1
)=1
o
i
.1
i=1
o
i
o
i.
.
our (dc)
2
reads
(dc)
2
= c
2
_
.1
)=1

.1
i=1
o
)
o
i
o
i)
+ 2
_
1
.1
)=1
o
i
.1
i=1
o
i
o
i.
+
_
1
.1
)=1
o
i
2
o
2
.
_
dt
= c
2
_
.1
)=1

.1
i=1
o
)
o
i
(o
i)
2o
i.
) + 2
.1
i=1
o
i
o
i.
+
_
1
.1
i=1
o
i
2
o
2
.
_
dt. (10.37)
The second preliminary step for obtaining the Bellman equation uses (10.32) again
and expresses the interest rate from (10.33) as a sum of the interest rate of asset `
(which could but does not need to be a riskless asset) and weighted excess returns
:
i
:
.
.
: =
.1
i=1
o
i
:
i
+
_
1
.1
i=1
o
i
_
:
.
= :
.
+
.1
i=1
o
i
[:
i
:
.
] . (10.38)
The Bellman equation with (10.37) and (10.38) now nally reads
j\ (c) = max
c(t),0
.
(t)
_
n(c) +\
0
(c)
_
(:
.
+
.1
i=1
o
i
[:
i
:
.
])c +n c
+
1
2
\
00
(c) [dc]
2
_
.
where (dc)
2
should be thought of representing (10.37).
First-order conditions
The rst-order conditions are the rst-order condition for consumption,
n
0
(c) = \
0
(c) .
and the rst-order condition for assets. The rst-order condition for consumption has
the well-known form.
226
To compute rst-order conditions for shares o
i
, we compute d . ,do
i
for (10.37),
d .
do
i
=
.1
)=1
o
)
(o
i)
2o
i.
) +
.1
i=1
o
i
(o
i)
2o
i.
) + 2o
i.
2
_
1
.1
)=1
o
i
o
2
.
= 2
_
.1
)=1
o
)
(o
i)
2o
i.
) +o
i.

_
1
.1
)=1
o
i
o
2
.
_
= 2
_
.1
)=1
o
)
(o
i)
o
i.
) +
_
1
.1
)=1
o
i
_
o
i.
o
2
.
__
= 2
.
)=1
o
i
[o
i)
o
i.
] . (10.39)
Hence, the derivative of the Bellman equation with respect to o
i
is with (10.38) and
(10.39)
\
0
c[:
i
:
.
] +
1
2
\
00
2c
2
.
)=1
o
i
[o
i)
o
i.
] = 0 ==
:
i
:
.
=
\
00
\
0
c
.
)=1
o
i
[o
i)
o
i.
] . (10.40)
The interpretation of this optimality rule should take into account that we assumed
that an interior solution exists. This condition therefore says that agents are indier-
ent between the current portfolio and a marginal increase in a share o
i
if the dierence
in instantaneous returns, :
i
:
.
, is compensated by the covariances of of assets i and
`. Remember that from (10.33), instantaneous returns are certain at each instant.
10.4.3 Capital asset pricing
Given optimal behaviour of agents, we now derive the well-known capital asset pricing
equation. Start by assuming that asset ` is riskless. This implies that it has a
variance of zero and therefore a covariance o
.)
with any other asset of zero as well,
o
.)
= 0 \,. Denote the return for asset ` by :. Dene further = c
\
00
\
0
. the
covariance of asset i with the market portfolio as o
iA
=
.
)=1
o
)
o
i)
. the variance of
the market portfolio as o
2
A
=
.
)=1
o
i
o
iA
and the return of the market portfolio as
: =
.
i=1
o
i
:
i
as in (10.33).
We are now only few steps away from the CAP equation. Using the denition of
:. and o
iA
allows to rewrite the rst-order condition for shares (10.40) as
:
i
: = o
iA
. (10.41)
Multiplying this rst-order condition by the share o
i
gives o
i
[:
i
:] = o
i
o
iA
. Sum-
ming over all assets, i.e. applying
.
)=1
to both sides, and using the above denitions
yields
: : = o
2
A
.
Dividing this expression by version (10.41) of the rst-order condition yields the
capital asset pricing equation,
:
i
: =
o
iA
o
2
A
(: :) .
The ratio o
iA
,o
2
A
is what is usually called the ,-factor.
227
10.5 Natural volatility II (*)
Before this book comes to an end, the discussion of natural volatility models in ch.
7.8 is completed in this section. We will present a simplied version of those models
that appear in the literature which are presented in stochastic continuous time setups.
The usefulness of Poisson processes will become clear here. Again, more background
is available at www.wifak.uni-wuerzburg.de/vwl2/nv.
10.5.1 An RBC model
This section presents the simplest general equilibrium setup that allows to study
uctuations stemming from occasional jumps in technology. The basic belief is that
economically relevant changes in technologies are rare and occur every 5-8 years.
Jumps in technology means that the technological level, as captured by the TFP
level, increases. Growth cycles therefore result without any negative TFP shocks.
Technologies
The economy produces a nal good by using a Cobb-Douglas technology
1 = 1
c
1
1c
. (10.42)
Total factor productivitiy follows a geometric Poisson process with drift
d, = dt +,d. (10.43)
where q and / are positive constants and ` is the exogenous arrival rate of the Poisson
process . We know from (9.66) in ch. 9.4.4 that the growth rate of the expected
value of is given by +`,.
The nal good can be used for consumption and investment, 1 = C + 1. which
implies that the prices of all these goods are identical. Choosing 1 as the numeraire
good, the price is one for all these goods. Investment increases the stock of production
units 1 if investment is larger than depreciation, captured by a constant depreciation
rate o.
d1 = (1 C o1) dt. (10.44)
There are rms who maximize instantaneous prots. They do not bear any risk
and pay factors : and n. marginal productivities of capital and labour.
Households
Housholds maximize their objective function
l (t) = 1
t
_
1
t
c
j[tt[
n(c (t)) dt
by choosing the consumption path c (t) . Instantaneous utility can be specied by
n(c) =
c
1o
1
1 o
. (10.45)
228
Wealth of households consists in shares in rms which are denoted by /. This wealth
changes in a deterministic way (we do not derive it here but it could be done following
the steps in ch. 9.2.6), despite the presence of TFP uncertainty. This is due to two
facts: First, wealth is measured in units of physical capital, i.e. summing / over all
households gives 1. As the price of one unit of 1 equals the price of one unit of the
output good and the latter was chosen as numeraire, the price of one unit of wealth is
non-stochastic. This diers from (9.31) where the price jumps when jumps. Second,
a jump in does not aect / directly. This could be the case when new technologies
make part of the old capital stock obsolete. Hence, the constraint of housholds is a
budget constraint which reads
d/ = (:/ +n c) dt. (10.46)
Optimal behaviour
When computing optimal consumption levels, households take the capital stock
/ and the TFP level as their state variables into account. This setup is there-
fore similar to the deterministic two-state maximization problem in ch. 6.3. Going
through similar steps (concerning e.g. the substituting out of cross derivatives in step
DP2) and taking the specic aspects of this stochastic framework into account, yields
optimal consumption following (see exercise 8)
n
00
(c)
n
0
(c)
dc =
_
: j +`
_
n
0
(~ c)
n
0
(c)
1
__
dt
n
00
(c)
n
0
(c)
~ c c d. (10.47)
Despite the deterministic constraint (10.46) and due to TFP jumps in (10.43), con-
sumption jumps as well: a dterm shows up in this expression and marginal utility
levels before (n
0
(c)) and after the jump (~ n
c
), using the notation from (9.7) appear
as well. The reason is straightforward: Whenever there is a discrete increase in the
TFP level, the interest rate and wages jump. Hence, returns for savings or households
change and the household adjusts her consumption level. This is in principle iden-
tical to the behaviour in the deterministic case as illustrated in g. 27 in ch. 5.5.3.
(Undertaking this here for this stochastic case would be very useful.)
General equilibrium
We are now in the position to start thinking about the evolution of the economy
as a whole. It is described by a system in three equations. TFP follows (10.43). The
capital stock follows
d1 =
_
1
c
1
1c
C o1
_
dt
from (10.42) and (10.44). Aggregate consumption follows
o
dC
C
=
_
: j `
_
1
_
C
~
C
_
o
__
dt +
_
1
_
C
~
C
_
o
_
d
from (10.47) and (10.45) and from aggregating over households. This system looks
fairly familiar from deterministic models, the only substantial dierence lies in the d
term and the post-jump consumption levels
~
C.
229
The simplest way to get an intuition about how the economy evolves consists in
looking at an example, i.e. by looking at a solution of the above system that holds
for a certain parameter set. We choose as example the parameter set for which the
saving rate is constant, C = (1 :) 1.
t.b.c.
... , we can draw a phase diagram and illustrate cyclical growth.
.
Figure 43 Cyclical growth
10.5.2 A natural volatility model
t.b.c.
10.5.3 Summary
Mathematical background on dynamic programming
There are many, and in most cases much more technical, presentations of dynamic
programming in continuous time under uncertainty. A classic mathematical reference
is Gihman and Skorohod (1972(Gihman and Skorohod 1972)) and a widely used
mathematical textbook is ksendal (1998); see also Protter (1995). These books are
probably useful only for those wishing to work on the theory of optimization and
not on applications of optimization methods. Optimization with unbounded utility
functions by dynamic programming was studied by Sennewald (2006).
With SDEs we need boundary conditions as well. In the innite horizon case, we
would need a transversality condition (TVC). See Smith (1996) for a discussion of
a TVC in a setup with Epstein-Zinn preferences. Sennewald (2006) has TVCs for
Poisson uncertainty.
Applications
Books in nance that use dynamic programming methods include Due (1988,
2001) and Bjrk (2004). Stochastic optimization for Brownian motion is also covered
nicely by Chang (2004).
A maximization problem of the type presented in ch. 10.1 was rst analyzed in
Wlde (1999, 2005b). This chapter combines these two papers. These two papers
were also jointly used in the Keynes-Ramsey rule appendix to Wlde (2005). It is
also used in Posch and Wlde (2006), Sennewald and Wlde (2006) and elsewhere.
Optimal control in stochastic continuous time setups is used in many applications.
Examples include issues in international macro (Obstfeld, 1994, Turnovsky, 1997,
2000), international nance and debt crises (Stein, 2006) and many others (coming
soon). A rm maximization problem with risk-neutrality where R&D increases quality
230
of goods, modelled by a stochastic dierential equation with Poisson uncertainty, is
presented and solved by Dinopoulos and Thompson (1998, sect. 2.3).
The Keynes-Ramsey rule in (10.27) was derived in a more or less complex frame-
work by Breeden (1986) in a synthesis of his consumption based capital asset pricing
model, Cox, Ingersoll and Ross (1985) in their continuous time capital asset pricing
model, or Turnovsky (2000) in his textbook. Cf. also Obstfeld (1994).
Closed form solutions
Closed-form solutions and analytical expressions for value functions are derived
by many authors. This approach was pioneered by Merton (1969, 1971) for Brownian
motion. Chang (2004) devotes an entire chapter (ch. 5) on closed-form solutions for
Brownian motion. For setups with Poisson-uncertainty, Dinopoulos and Thompson
(1998, sect. 2.3), Wlde (1999b) or Sennewald and Wlde (2006, ch. 3.4) derive
closed-form solutions.
Closed-form solutions for Levy processes are available e.g. from Aase (1984),
Framstad, ksendal and Sulem (2001) and in the textbook by ksendal and Sulem
(2005).
Closed-form solutions with parameter restrictions
Sometimes, restricting the parameter set of the economy in some intelligent way al-
lows to provide closed-form solutions for very general models. These solutions provide
insights which can not that easily be obtained by numerical analysis. Early examples
are Long and Plosser (1983) and Benhabib and Rustichini (1994) who obtain closed-
form solutions for a discrete-time stochastic setup. In deterministic, continuous time,
Xie (1991, 1994) and Barro, Mankiw and Sala-i-Martin (1995) use this approach as
well. Wlde (2005) and Wlde and Posch (2006) derive closed form solutions for
business cycle models with Poisson uncertainty.
Continuous time methods
There are various recent papers which use continuous time methods under un-
certainty. For examples from nance and monetary economics, see DeMarzo and
Uroevic (2006), Gabaix et al. (2006), Maenhout (2006), Piazzesi (2005), examples
from risk theory and learning include Kyle et al. (2006) and Keller and Rady (1999),
for industrial organization see e.g. Murto (2004), in spatial economics there is e.g.
Gabaix (1999), the behaviour of households in the presence of durable consumption
goods is analyzed by Bertola et al. (2005). The real options approch to investment
under uncertainty is another larger area. A recent contribution is by Guo et al.
(2005). Further references to papers that use Poisson processes can be found in ch.
9.5.
231
Exercises Chapter 10
Dynamic Programming in continuous
time under uncertainty
1. Optimal saving under Poisson uncertainty with two state variables (exam)
Consider the objective function
l (t) = 1
t
_
1
t
c
j[tt[
n(c (t)) dt
and the budget constraint
dc (t) = :c (t) +n j (t) c (t) dt +,c (t) d (t) .
where : and n are constant interest and wage rates, (t) is a Poisson process
with an exogenous arrival rate ` and , is a constant as well. Letting q and o
denote constants, assume that the price j (t) of the consumption good follows
dj (t) = j (t) [qdt +od (t)] .
(a) Derive a rule which optimally describes the evolution of consumption. De-
riving this rule in the form of marginal utility, i.e. dn
0
(c (t)) is sucient.
(b) Derive a rule for consumption, i.e. dc (t) = ...
(c) Derive a rule for optimal consumption for , = 0 or o = 0.
2. Optimal saving under Brownian motion
Derive the Keynes-Ramsey rules in (10.28) and (10.29), starting from the rule
for marginal utility in (10.27).
3. Adjustment cost
Consider a rmthat maximizes its present value dened by = 1
t
_
1
t
c
v[tt[
: (t) dt.
The rms prot : is given by the dierence between revenues and costs,
: = jr ci
2
. where output is assumed to be a function of the current cap-
ital stock, r = /
c
. The rms control variable is investment i that determines
its capital stock,
d/ = (i o/) dt.
232
The rm operates in an uncertain environment. Output prices and costs for
investment evolve according to
dj,j = c
j
dt +o
j
d.
j
. dc,c = c
c
dt +o
c
d.
c
.
where .
j
and .
c
are two independent stochastic processes.
(a) Assume .
j
and .
c
are two independent Brownian motions. Set c
j
= o
j
= 0.
such that the price j is constant. What is the optimal investment behaviour
of the rm?
(b) Consider the alternative case where cost c are constant but prices j follow
the above SDE. How would much would the rm now invest?
(c) Provide an answer to the question in a) when .
c
is a Poisson process.
(d) Provide an answer to the question in b) when .
j
is a Poisson process.
4. Firm specic technological progress (exam)
Consider a rm facing a demand function with price elasticity . r = j
.
.
where j is the price and is a constant. Let the rms technology be given
by r = c
q
| where c 1. The rm can improve its technology by investing
into R&D. R&D is modelled by the Poisson process which jumps with arrival
rate `(|
q
) where |
q
is employment in the research department of the rm. The
exogenous wage rate the rm faces amounts to n.
(a) What is the optimal static employment level | of this rm for a given
technological level ?
(b) Formulate an intertemporal objective function given by the present value of
the rms prot ows over an innite time horizon. Continue to assume that
is constant and let the rm choose | optimally from this intertemporal
perspective. Does the result change with respect to (a)?
(c) Using the same objective function as in (b), let the rm now determine
both | and |
q
optimally. What are the rst order conditions? Give an
interpretation in words.
(d) Compute the expected output level for t t, given the optimal employ-
ment levels |
and |
q
. In other words, compute 1
t
r (t) . Hint: Derive rst
a stochastic dierential equation for output r (t) .
5. Budget constraints and optimal saving and nance decisions (exam)
Imagine an economy with two assets, physical capital 1 (t) and government
bonds 1(t). Let wealth c (t) of households be given by c (t) = (t) / (t) +/ (t)
where (t) is the price of one unit of capital, / (t) is the number of stocks and
/ (t) is the nominal value of government bonds held by the household. Assume
the price of stocks follows
d (t) = c (t) dt +, (t) d (t) .
where c and , are constants and (t) is a Poisson process with arrival rate `.
233
(a) Derive the budget constraint of the household. Use o (t) = (t) / (t) ,c (t)
as the share of wealth held in stocks.
(b) Derive the budget constraint of the household by assuming that (t) is
Brownian motion.
(c) Now let the household live in a world with three assets (in addition to the
two above, there are assets available on ,oreign markets). Assume that
the budget constraint of the household is given by
dc (t) = : (t) c (t) +n(t) j (t) c (t) dt+,
I
o
I
(t) c (t) d (t)+,
)
o
)
(t) c (t) d
)
(t) .
where
: (t) = o
I
(t) :
I
+o
)
(t) :
)
+ (1 o
I
(t) o
)
(t)) :
b
is the interest rate depending on weights o
i
(t) and constant instantaneous
interest rates :
I
. :
)
and :
b
. Let (t) and
)
(t) be two Poisson processes.
Given the usual objective function
l (t) = 1
t
_
1
t
c
j[tt[
n(c (t)) dt.
what is the optimal consumption rule? What can be said about optimal
shares o
I
and o
)
?
6. Capital asset pricing in ch. 10.4.3
The covariance of asset i with the market portfolio is denoted by o
iA
. the
variance of the market portfolio is o
2
A
and the return of the market portfolio is
:
A
=
i
o
i
:
i
.
(a) Show that the covariance of asset i with the market portfolio o
iA
is given
by
.
)=1
o
)
o
i)
.
(b) Show that
.
)=1
o
i
o
iA
is the variance of the market portfolio o
2
A
.
234
7. Standard and non-standard technologies (exam)
Let the social welfare function of a central planner be given by
l (t) = 1
t
_
1
t
c
j[tt[
n(C (t)) dt.
(a) Consider an economy where the capital stock follows d1 = 1
c
1
1c
[jdt +od.]
(o1 +C) dt where d. is the increment of Brownian motion and j and o
are constants. Derive the Keynes-Ramsey rule for this economy.
(b) Assume that d1 = [jdt +od.] and that is constant. What is the
expected level of 1 (t) for t t. i.e. 1
t
1 (t)?
(c) Consider an economy where the technology is given by 1 = 1
c
1
1c
with d = dt + ,d.. where . is Brownian motion. Let the capital
stock follow d1 = (1 o1 C) dt. Derive the Keynes-Ramsey rule for
this economy as well.
(d) Is there a parameter constellation under which the Keynes-Ramsey rules
are identical?
8. Standard and non-standard technologies II (exam)
Provide answers to the same questions as in "Standard and non-standard tech-
nologies" but assume that . is a Poisson process with arrival rate `. Compare
your result to (10.47).
235
11 Notation and variables, references and index
Notation and variables
The notation is as homogenous as possible. Here are the general rules but some
exceptions are possible.
Variables in capital, like capital 1 or consumption C denote aggregate quanti-
ties, lower-case letters pertain to the household level
A function , (.) is always presented by using parentheses (.) . where the paren-
theses give the arguments of functions. Brackets [.] always denote a multiplica-
tion operation
A variable r
t
denotes the value of r in period t in a discrete time setup. A
variable r (t) denotes its value at a point in time t in a continuous time setup. We
will stick to this distinction in this textbook both in deterministic and stochastic
setups. (Note that the mathematical literature on stochastic continuous time
processes, i.e. what is treated here in part IV, uses r
t
to express the value of r
at a point in time t.)
A dot indicates a time derivative, _ r (t) = dr,dt.
A derivative of a function , (r) where r is a scalar and not a vector is abreviated
by ,
0
(r) = d, (r) ,dr. Partial derivatives of a function , (r. ) are denoted by
e.g. ,
a
(r. ) = ,
a
(.) = ,
a
= J, (r. ) ,Jr.
Here is a list of variables and abbreviations which shows that some variables
have multi-tasking abilities, i.e. they have multiple meanings
Greek letters
c output elasticity of capital
, = 1, (1 +j) discount factor in discrete time
preference parameter on rst period utility in two-period models
o depreciation rate
o share of wealth held in the risky asset
o
i
share of wealth held in asset i
: instantaneous prots, i.e. prots :
t
in period t or : (t) at point in time t
j time preference rate, correlation coecient between random variables
j
i)
correlation coecient between two random variables i and ,
t some future point in time, t t
i
the share of savings used for buying stock i
adjustment cost function
236
Latin letters
CVF change of variable formula
c expenditure c = jc. eort, exponential function
, (r +) a function , (.) with argument r +
, [r +] some variable , times r +
(t) Poisson process
SDE stochastic dierential equation
t time t standing for today (see also t)
TVC transversality condition
n instantaneous utility (see also :)
r
xpoint of a dierence or dierential equation (system)

. (t) Brownian motion, Wiener process
237
References
Aase, K. K. (1984): Optimum Portfolio Diversication in a General Continuous-
Time Model, Stochastic Processes and their Applications, 18, 8198.
Abel, A. B. (1990): Aggregate Asset Pricing; Asset Prices under Habit Formation
and Catching up the Joneses, American Economic Review; AEA Papers and Pro-
ceedings, 80(2), 3842.
Aghion, P., and P. Howitt (1992): A Model of Growth Through Creative Destruc-
tion, Econometrica, 60, 323351.
Araujo, A., and J. A. Scheinkman (1983): Maximum principle and transversality con-
dition for concave innite horizon economic models, Journal of Economic Theory,
30, 1 16.
Arrow, K. J.and Kurz, M. (1970): Public investment, the rate of return and optimal
scal policy. Johns Hopkins University Press, Baltimore.
Azariadis, C. (1993): Intertemporal Macroeconomics. Basil Blackwell, Oxford.
Barro, R. J., N. G. Mankiw, and X. Sala-i Martin (1995): Capital Mobility in
Neoclassical Models of Growth, American Economic Review, 85, 10315.
Bellman, R. (1957): Dynamic Programming. Princeton University Press, Princeton.
Benhabib, J., and A. Rustichini (1994): A note on a new class of solutions to dy-
namic programming problems arising in economic growth, Journal of Economic
Dynamics and Control, 18, 807813.
Bertola, G., L. Guiso, and L. Pistaferri (2005): Uncertainty and Consumer Durables
Adjustment, Review of Economic Studies, 72, 973
U1007.
Bjrk, T. (2004): Arbitrage Theory in Continuous Time, 2nd ed. Oxford University
Press.
Black, F., and M. Scholes (1973): The Pricing of Options and Corporate Liabilites,
Journal of Political Economy, 81, 637659.
Blanchard, O., and S. Fischer (1989): Lectures on Macroeconomics. MIT Press.
Blanchard, O. J. (1985): Debt, Decits, and Finite Horizons, Journal of Political
Economy, 93, 22347.
Bossmann, M., C. Kleiber, and K. Wlde (2007): Bequests, taxation and the distri-
bution of wealth in a general equilibrium model, Forthcoming Journal of Public
Economics.
Braun, M. (1975): Dierential equations and their applications (short version).
Springer, Berlin.
238
Brock, W. A., and A. G. Malliaris (1989): Dierential Equations, Stability and
Chaos in Dynamic Economics, Advanced Textbooks in Economics.
Buiter, W. H. (1981): Time Preference and International Lending and Borrowing in
an Overlappig-Generations Model, Journal of Political Economy, 89, 769797.
Burdett, K., and D. T. Mortensen (1998): Wage Dierentials, Employer Size, and
Unemployment, International Economic Review, 39(2), 25773.
Chang, F.-R. (2004): Stochastic Optimization in Continuous Time. Cambridge Uni-
versity Press, Cambridge, UK.
Chen, Y.-F., and M. Funke (2005): Non-Wage Labour Costs, Policy Uncertainty
And Labour Demand - A Theoretical Assessment, Scottish Journal of Political
Economy, 52, 687709.
Chiang, A. C. (1984): Fundamental Methods of Mathematical Economics. McGraw-
Hill, Third Edition, New York.
(1992): Elements of dynamic optimization. McGraw-Hill, New York.
Chow, G. C. (1997): Dynamic Economics. Optimization by the Lagrange Method.
Oxford University Press, Oxford.
Cox, J. C., J. E. Ingersoll, and S. A. Ross (1985): An intertmporal general Equilib-
rium Model of asset prices, Econometria, 53(2), 363.
Danthine, Jean Pierre Donaldson, J. B., and R. Mehra (1992): the equity premium
and the allocation of income risk, Journal fo Economics and Control, 16, 509 532.
Das, S. R., and S. Foresi (1996): Exact Solutions for Bond and Option Prices with
Systematic Jump Risk, Review of Derivatives Research, 1, 724.
de la Croix, D., and P. Michel (2002): A theory of economic growth: dynamics and
policy in overlapping generations. Cambridge University Press.
Dierker, E., and B. Grodahl (1995): Prot maximization mitigates competition,
Economic Theory, 7, 139160.
Dinopoulos, E., and P. Thompson (1998): Schumpeterian Growth Without Scale
Eects, Journal of Economic Growth, 3, 313335.
Dixit, A. K. (1990, 2nd ed.): Optimization in economic theory. Oxford University
Press.
Dixit, A. K., and R. S. Pindyck (1994): Investment Under Uncertainty. Princeton
University Press.
Due, D. (1988): Security Markets - Stochastic Models. Academic Press.
239
(2001): Dynamic Asset Pricing Theory, 3rd ed. Princeton University Press.
Eaton, J. (1981): Fiscal Policy, Ination and the Accumulation of Risky Capital,
Review of Economic Studies, 48(153), 435445.
Epaulard, A., and A. Pommeret (2003): Recursive Utility, Endogenous Growth, and
the Welfare Cost of Volatility, Review of Economic Dynamics, 6(2), 672684.
Evans, Hastings, and Peacock (2000): Statistical Distributions, 3rd edition. Wiley.
Farzin, Y. H., K. Huisman, and P. Kort (1998): Optimal Timing of Technology
Adoption, Journal of Economic Dynamics and Control, 22, 779799.
Feichtinger, G., and R. F. Hartl (1986): Optimale Kontrolle konomischer Prozesse:
Anwendungen des Maximumprinzips in den Wirtschaftswissenschaften. de Gruyter,
Berlin.
Fichtenholz (1997): Dierential und Integralrechnung, 14. Auage. Harri Deutsch.
Framstad, N. C., B. ksendal, and A. Sulem (2001): Optimal Consumption and
Portfolio in a Jump Diusion Market with Proportional Transaction Costs, Jour-
nal of Mathematical Economics, 35, 233257.
Francois, P., and H. Lloyd-Ellis (2003): Animal Spirits Trough Creative Destruc-
tion, American Economic Review, 93, 530550.
Gabaix, X. (1999): Zipfs Law For Cities: An Explanation, The Quarterly Journal
of Economics, 114(3), 739767.
Gabaix, X., P. Gopikrishnan, V. Plerou, and H. E. Stanley (2006): Institutional
Investors and Stock Market Volatility, Quarterly Journal of Economics, 121, 461
504.
Gabszewicz, J., and J. P. Vial (1972): Oligopoly " la Cournot" in a general equi-
librium analysis, Journal of Economic Theory, 4, 381400.
Gandolfo, G. (1996): Economic Dynamics, 3rd edition. Springer, Berlin.
Garca, M. A., and R. J. Griego (1994): An Elementary Theory of Stochastic Dif-
ferential Equations Driven by A Poisson Process, Commun. Statist. - Stochastic
Models, 10(2), 335363.
Gihman, I. I., and A. V. Skorohod (1972): Stochastic Dierential Equations. Springer-
Verlag, New York.
Grinols, E. L., and S. J. Turnovsky (1998): Consequences of debt policy in a sto-
chastically growing monetary economy, International Economic Review, 39(2),
495521.
240
Grossman, G. M., and E. Helpman (1991): Innovation and Growth in the Global
Economy. The MIT Press, Cambridge, Massachusetts.
Grossman, G. M., and C. Shapiro (1986): Optimal Dynamic R&D Programs,
RAND Journal of Economics, 17, 581593.
Guo, X., J. Miao, and E. Morelle (2005): Irreversible investment with regime shifts,
Journal of Economic Theory, 122, 37
U59.
Guriev, S., and D. Kvasov (2005): Contracting on Time, American Economic Re-
view, 95(5), 13691385.
Hassett, K. A., and G. E. Metcalf (1999): Investment with Uncertain Tax Policy:
Does Random Tax Discourage Investment?, Economic Journal, 109, 372393.
Heer, B., and A. Mauner (2005): Dynamic General Equilibrium Modelling, Compu-
tational Methods and Applications. Springer, Berlin.
Intriligator, M. D. (1971): Mathematical optimization and economic theory. Engle-
wood Clis, New Jersey.
Jermann, U. J. (1998): Asset pricing in production economies, Journal of Monetary
Economics, 41, 257275.
Johnson, N. L., S. Kotz, and N. Balakrishnan (1995): Continuous Univariate Distri-
butions, Vol. 2, 2nd edition. Wiley.
Judd, K. L. (1985): The Law of Large Numbers with a Continuum of I.I.D. Random
Variables, Journal of Economic Theory, 35, 1925.
Kamien, M. I., and N. L. Schwartz (1991): Dynamic optimization. The calculus of
variations and optimal control in economics and management. 2nd edition. Ams-
terdam, North-Holland.
Kamihigashi, T. (2001): Necessity of Transversality Conditions for Innite Horizon
Problems, Econometrica, 69, 9951012.
Keller, G., and S. Rady (1999): Optimal Experimentation in a Changing Environ-
ment, Review of Economic Studies, 66, 475507.
Kiyotaki, N., and R. Wright (1993): A Search-Theoretic Approach to Monetary
Economics, American Economic Review, 83, 6377.
Kleiber, C., and S. Kotz (2003): Statistical Size Distributions in Economics and
Actuarial Sciences. Wiley.
Kyle, A. S., H. Ou-Yang, and W. Xiong (2006): Prospect theory and liquidation
decisions, Journal of Economic Theory, 129, 273
U288.
241
Laitner, J., and D. Stolyarov (2004): Aggregate returns to scale and embodied tech-
nical change: theory and measurement using stock market data, Journal of Mon-
etary Economics, 51, 191233, pdf.
Leonard, D., and N. V. Long (1992): Optimal Control Theory and Static Optimization
in Economics. Cambridge University Press, New York.
Ljungqvist, L., and T. J. Sargent (2004): Recursive Macroeconomic Theory 2nd ed.
MIT Press.
Long, J. B. Jr.., and C. I. Plosser (1983): Real Business Cycles, Journal of Political
Economy, 91, 3969.
Lucas, R. E. Jr.. (1978): Asset Prices in an Exchange Economy, Econometrica, 46,
14291445.
(1988): On the Mechanics of Economic Development, Journal of Monetary
Economics, 22, 342.
Maenhout, P. J. (2006): Robust portfolio rules and detection-error probabilities for
a mean-reverting risk premium, Journal of Economic Theory, 128, 136
U163.
Matsuyama, K. (1999): Growing Through Cycles, Econometrica, 67, 335347.
(2001): Growing through Cycles in an Innitely Lived Agent Economy,
Journal of Economic Theory, 100, 220234.
McDonald, R., and D. Siegel (1986): The Value of Waiting to Invest, Quarterly
Journal of Economics, 101, 70727.
Merton, R. C. (1969): Lifetime Portfolio Selection Under Uncertainty: The
Continuous-Time Case, Review of Economics and Statistics, 51, 247257.
(1971): Optimum Consumption and Portfolio Rules in a Continuous-Time
Model, Journal of Economic Theory, 3, 373413.
(1976): Option Pricing When Underlying Stock Returns are Discontinuous,
Journal of Financial Economics, 3, 125144.
Michel, P. (1982): On the Transversality Condition in Innite Horizon Optimal
Problems, Econometrica, 50, 975985.
Mitrinovic, D. (1970): Analytic Inequalities. Springer, Berlin.
Moscarini, G. (2005): Job Matching and The Wage Distribution, Econometrica,
73(2), 481
U516.
Murto, P. (2004): Exit in Duopoly under Uncertainty, RAND Journal of Economics,
35, 111127.
242
Obstfeld, M. (1994): Risk-Taking, Global Diversication, and Growth, American
Economic Review, 84, 131029.
ksendal, B. (1998): Stochastic Dierential Equations. Springer, Fifth Edition,
Berlin.
ksendal, B., and A. Sulem (2005): Applied Stochastic Control of Jump Diusions.
Springer-Verlag, Berlin.
Palokangas, T. (2003): Common Markets, Economic Growth, and Creative Destruc-
tion, Journal of Economics, 10, 5776.
(2005): International Labor Union policy and Growth with Creative De-
struction, Review of International Economics, 13, 90105.
Piazzesi, M. (2005): Bond Yields and the Federal Reserve, Journal of Political
Economy, 113(2), 311344.
Pissarides, Christopher, A. (2000): Equilibrium Unemployment Theory. MIT.
Pommeret, A., and W. T. Smith (2005): Fertility, volatility, and growth, Economics
Letters, 87(3), 347353.
Protter, P. (1995): Stochastic Integration and Dierential Equations. Springer-Verlag,
Berlin.
Rebelo, S. (1991): Long-Run Policy Analysis and Long-Run Growth, Journal of
Political Economy, 99(3), 500521.
Ross, S. M. (1993): Introduction to Probability Models, 5th edition. Academic Press,
San Diego.
(1996): Stochastic processes, 2nd edition. Academic Press, San Diego.
Rouwenhorst, K. G. (1995): Asset Pricing Implications fo Equilibrium Business Cycle
Models. Princeton University press, in Frontiers of Business Cycles, ed. Thomas F.
Cooley, 294-330, New Jersey.
Santanu Chatterjee, P. G., and S. J. Turnovsky (2004): Capital Income Taxes and
Growth in a Stochastic Economy: A Numerical Analysis of the Role of Risk Aver-
sion and Intertemporal Substitution, Journal of Public Economic Theory, 6(2),
277310.
Schlegel, C. (2004): Analytical and Numerical Solution of a Poisson RBC model,
Dresden Discussion Paper Series in Economics, 5/04.
Sennewald, K. (2006): Controlled Stochastic Dierential Equations under Poisson
Uncertainty and with Unbounded Utility, Journal of Economic Dynamics and
Control, forthcoming.
243
Sennewald, K., and K. Wlde (2005): "Itos Lemma" and the Bellman equation for
Poisson processes: An applied view, Journal of Economics, forthcoming.
Shapiro, C., and J. E. Stiglitz (1984): Equilibrium Unemployment as a Worker
Discipline Device, American Economic Review, 74, 43344.
Shell, K. (1969): Applications of Pontryagins Maximum Principle to Economics.
Springer, Berlin.
Shimer, R. (2005): The Cyclical Behavior of Equilibrium Unemployment and Va-
cancies, American Economic Review, 95, 2549.
Smith, W. T. (1996): Feasibility and Transversality Conditions for Models of Port-
folio Choice with Non-Expected Utility in Continuous Time, Economics Letters,
53, 12331.
Steger, T. M. (2005): Stochastic growth under Wiener and Poisson uncertainty,
Economics Letters, 86(3), 311316.
Stein, J. L. (2006): Stochastic Optimal Control, International Finance, and Debt
Crises. Oxford University Press.
Stokey, N. L., R. E. Lucas Jr., and E. C. Prescott (1989): Recursive Methods in
Economics Dynamics. Harvard University Press, Cambridge, MA.
Thompson, P., and D. Waldo (1994): Growth and trustied capitalism, Journal of
Monetary Economics, 34, 445462.
Toche, P. (2005): A tractable model of precautionary saving in continuous time,
Economics Letters, 87, 267272.
Turnovsky, S. J. (1997): International macroeconomic dynamics. MIT Press.
(2000): International macroeconomic dynamics Methods of Macroeconomic
Dynamics. MIT Press.
Turnovsky, S. J., and W. T. Smith (2006): Equilibrium consumption and precaution-
ary savings in a stochastically growing economy, Journal of Economic Dynamics
and Control, 30(2), 243278.
Varian, H. (1992): Microeconomic Analysis. Norton, Third Edition.
Venegaz-Martinez, F. (2001): Temporary Stabilization: A Stochastic Analysis,
Journal of Economic Dynamics and Control, 25, 14291449.
Vissing-Jorgensen, A. (2002): Limited Asset Market Participation and the Elasticity
of Intertemporal Substittuion, Journal of Political Economy, 110, 825853.
Wlde, K. (1996): Proof of Global Stability, Transitional Dynamics, and Interna-
tional Capital Flows in a Two-Country Model of Innovation and Growth, Journal
of Economics, 64, 5384.
244
(1999a): A Model of Creative Destruction with Undiversiable Risk and
Optimising Households, Economic Journal, 109, C156C171.
(1999b): Optimal Saving under Poisson Uncertainty, Journal of Economic
Theory, 87, 194217.
(2002): The Economic Determinants of Technology Shocks in a Real Busi-
ness Cycle Model, Journal of Economic Dynamics & Control, 27, 128.
(2005): Endogenous Growth Cycles, International Economic Review, 46,
867894.
(2005b): Corrigendum to "Optimal Saving under Poisson Un-
certainty", Journal of Economic Theory, forthcoming, available at
www.waelde.com/publications.
Walsh, C. E. (2003): Monetary Theory and Policy. MIT Press.
Xie, D. (1991): Increasing Returns and Increasing Rates of Growth, Journal of
Political Economy, 99, 429 435.
(1994): Divergence in Economic Performance: Transitional Dynamics with
Multiple Equilibria, Journal of Economic Theory, 63, 97 112.
245
Index
adjustment cost, 164, 165
adjustment costs, 84, 163
Bellman equation, 35, 38, 40, 104, 148,
150, 152, 157, 210, 221, 222, 226
adjusting the basic form, 222
basic form, 210
Bellman equation, maximized, 41, 105, 149,
150, 158, 221
Denition, 36, 211
Brownian motion, 170, 171, 173, 178, 186,
195, 198, 221, 237
budget constraint, 25, 31, 37, 49, 69, 74,
76, 86, 148, 150, 157, 191, 192,
209, 221, 225
dynamic, 87
capital asset pricing (CAP), 139, 157, 224
central planner, 14, 33, 50, 100, 151, 167
CES utility, 146
closed-form solution, 8, 167, 213, 231
2 period model, 130, 131, 136
Cobb-Douglas utility, 7, 131
consumption level and interest rate, 88
control variable, 35, 78, 95, 105, 129, 160,
161, 221
CVF (change of variable formula), 177,
181, 185, 211, 212
depreciation rate, 17
Eulers theorem, 15, 16
expectations operator, 124, 129, 131, 150,
160, 199, 203, 219
expected value, 195, 202
rms, intertemporal optimization, 83, 162
rst-order condition with economic inter-
pretation, 135, 215
goods market equilibrium, 17
Hamiltonian, 76, 79, 81
Hamiltonian, current value, 84, 93, 106
Hamiltonian, present value, 95, 97
heterogeneous agents, 134
identically and independently distributed
(iid), 123, 126, 128
integral representation of dierential equa-
tions, 68, 176, 199, 202, 219
interest rate, denition, 18
investment, gross, 17
investment, net, 17
Itos Lemma, 177, 190, 223
Keynes-Ramsey rule, 106, 166
Leibniz rule, 65
level eect
dynamic, 41, 88
martingale, 198, 219
natural volatility, 140, 213, 228
numeraire good, 7, 25, 43, 152, 156, 228
numerical solution, 166
Poisson process, 170, 172, 180, 203, 205,
209, 237
representative agent assumption, 134
resource constraint, 17
risk aversion
absolute, 87
relative, 90
shadow price, 14, 15, 36, 79, 80, 94, 158
state variable, 35, 78, 79, 95, 107, 149,
160, 210, 222
sticky prices, 162
switch process, 193
Wiener process, 171, 237
zero-motion line, 57, 60, 91
246
12 Exams
sorry - all future exams deleted ... :-( ;-)
247

Applied Intertemporal Optimization

Încărcat de

Informații document

Descriere originală:

Titlu original

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

Applied Intertemporal Optimization

Încărcat de

Drepturi de autor:

Formate disponibile

These lecture notes are incomplete. Various sections still need polishing.

where the utility function

where the utility function was now replaced by a transformation

: how much does relative consumption c

)? This denition can be expressed alternatively (see

is constant and determined by

which we had computed in (2.31).

is positive as (2.37) shows. We will

of a function , (r) is dened by

is globally stable if, starting from an

in the left panel of g. 8 - it is

)) . The optimality condition is

subject to a resource constraint

as in (2.38)). Looking at the above graph of , (r) . we nd r

. , (r) < 0 and r (t) decreases.

is globally stable and have also obtained

. r(t) decreases, for r (0) < r

dt = n(c (t)) +jl (t) .

. Choose appropriate initial conditions.

) = 0 is necessary for an interior

+) could be positive for any ,= 0.) The

(t) is the path of (t) for an optimal choice of control variables.

which implies that there is zero

increases as the slope of the line becomes atter. Money growth in an

not only aects

depends on preferences of households.

stands for the point in time an instant

) +/ as the value of r to which

notation is the fact that the process x(t) is a so called

xpoint of a dierence or dierential equation (system)

S-ar putea să vă placă și