Bayesian Econometrics - Computer section

Leandro Magnusson

Department of Economics
Brown University
Leandro Magnusson@brown.edu
http://www.econ.brown.edu/students/Leandro Magnusson/
April 26, 2006
Preliminary Version
Abstract
This material is used as a reference for the computer exercises of the Bayesian econometrics course
taught by Professor Tony Lancaster. It covers a variety of models (linear regression models, limited
depended variable models (logit, probit and Tobit), etc). The Bayesian computation is done in BUGS
using R as its interface. Both softwares can be download freely from their respective web pages:
http://www.mrc-bsu.cam.ac.uk/bugs/ and
http://www.r-project.org/.

q
1
1 First Order Auto Regressive Model
Model: The AR(1) and ARMA(1,1) model are:
y
t
= y
t1
+
t
,
t
N(0, ) (1.1)
y
t
= y
t1
+
t
,
t
= u
t
u
t1
, {u
t
}
T
t=0
is i.i.d., u
t
N(0, ) (1.2)
where 1 < < 1 . The Durbin-Watson statistic of the AR(1) errors is:
dw =

T
t=2
(
t

t1
)

T
t=2
(
2
t1
)

t
= y
t
y
t1
(1.3)
Exercise 1. (Based on Example 2.15, page 95, Lancaster (2004))
In R, generate a sequence of AR(1) and a sequence of ARMA(1,1) observations with the same initial
value. Lets assume the both sample are AR(1), the prior distribution of is improper and is known.
(a) What is the likelihood for the AR(1) model (p(y|))?
(b) Generate from its posterior distribution ( N( ,

T
t=1
y
2
t1
)).
(c) Generate a sequence of y
rep
from its predictive distribution and compute
diff(, y
rep
, y
obs
) = dw(, y
rep
) dw(, y
obs
)
(d) Repeat items (b), (c) n times separately for AR(1) and ARMA(1,1).
(e) Plot the distributions of df(, y
rep
, y
obs
) and draw your conclusions.
2 Linear Regression Models
2.1 Independent, normal, homoscedastic errors
Database: See Mankiw et al. (1992).
Model: Solow Model
without human capital: Y
t
= K

t
(A
t
L
t
)
1
with human capital: Y
t
= K

t
H

t
(A
t
L
t
)
1
where L
t
= L
0
exp(nt) and A
t
= A
0
exp(gt). Consider that A
0
= a + , where a is constant and is a
country-specic shock. The basic empirical specications for the Solow model are:
ln
_
Y
L
_
= a +

1
ln(s
k
)

1
ln(n +g +) + (2.1)
ln
_
Y
L
_
= a +

1
ln(s
k
) +

1
ln(s
h
)
+
1
ln(n +g +) + (2.2)
where s
k
is the capital savings rate, s
h
is the human capital savings rate, n is the population growth rate, g
is technical growth rate, depreciation. Mankiw et al. (1992) assume that g + = 0.05.
2
Exercise 2.
(a) Consider the following unrestricted specication
ln
_
Y
L
_
= a +b
1
ln(s
k
) +b
2
ln(n +g +) +
What is the likelihood for the above model? How would you choose the priors?
(b) Write and run a BUGS program for the linear regression model. Do separately for nonoil, intermediate
and oecd countries. Compare your answers with the maximum likelihood estimates.
(c) Propose and run a test for the coecients b
1
and b
2
for each group of countries. How could you test the
hypothesis about the returns to scale?
(d) Include human capital and repeat parts (a), (b) and (c).
2.2 Heteroscedasticy
Database: See Yatchew & No (2001)
Model: Consider the log linear form of the demand of gasoline:
log(gas) =
0
+
1
log(price) +
2
log(income) +u (2.3)
In matrix notation, the OLS terms are: v = N k (the degrees of freedom),

= (X

X)
1
X

y and
s
2
=
(yX

)

(yX

)
v
. Assuming normality and homoscedasticity of the residuals, the likelihood function is:
1
p(y|, ) =
1
(2)
N
2
_

1
2
exp
_

2
(

)X

X(

)
___

v
2
exp
_

v
2s
2
__
If the residuals have the same variance we have:
E
_
u
2

2
= 2
4
where
2
is the variance (
2
=
1

). Therefore a simple statistic to verify if the residuals have equal variance

is:
T(,
2
) =

N
i=1
((y
i
x
i
)
2

2
)
N
2
4
Student linear model Take a look at Lancaster (2004), chapter 3, pages 159-162. Assume that:
u
i
|X, , ,
i
N(0,
i
)
where {
i
}
N
i=1
is independently gamma distributed with mean 1 and scalar hyperparameter d. Multiplying
the density of u
i
|X, , ,
i
by the density of {
i
}
N
i=1
and integrating out s one may show that u
i
|X, ,
follows a t- distribution with mean 0 and degrees of freedom equals to d.
1
For a formal derivation see Koop (2003)
3
Exercise 3.
(a) Assuming that the errors have the same variance, suggest a natural conjugate prior for this model
and derive the posterior distribution of and . Write a program in R for sampling of the posterior
distribution of and .
(b) Simulate and
2
from the joint distribution and evaluate T(,
2
). Repeat nrep times and the draw
the histogram of the test statistic. What are you conclusions?
(c) Write down a BUGS program for the student linear model. Explore how the posterior inferences about
change by using dierent values of d. Remember that low values of d approximates the t-distribution
to a Cauchy (have tails) and large values of d approximates to a normal distribution.
3 Limited Dependent Variable Models
3.1 Logit
Database: See Slonim & Roth (1998).
Denition: (Ultimatum Game)
The ultimatum game consists of two players bargaining over the amount of money which it will be
called pie. One player, the proposer, proposes a division of the pie, and the second player, the
responder, accepts or rejects it. If the responder accepts , each player earns the amount specied
in the proposal and if the responder rejects, each player earns zero. At perfect equilibrium the
proposer receives all or almost all of the pie.
Model: Rejection behavior
Prob(R = 1) = (a +b
0
Of +b
M
pM +b
H
pH) (3.1)
where is the logistic distribution function, R equals 1 if the oer is reject and 0 otherwise, Of is the
proportion of the pie oered (from 0 to 49,5%), pM = 1 if stakes are medium and pH = 1 is stakes are high.
Exercise 4.
(a) What is the likelihood for the above model? How would you choose the prior?
(b) Write and run a BUGS program for the logit model. Study the convergence of the sampler (try 3000 inter-
actions burning 1000 and 10000 interactions burning 8000). Compare your answers with the maximum
likelihood estimates.
(c) Evaluate the rejection probability at various oer values:
of =
_
0.1 0.2 0.3 0.4 0.5
_
;
do this separately for the low, medium, and high stakes conditions.
(d) Does the level of the stakes aect the rejection probability. Propose a test and compute it. What is your
conclusion?
4
3.2 Tobit
Database: See Mroz (1987)
Model: (Labor supply of Married Women) The reduced form of a womans labor supply is given by:
2
h
i
= a
0
+a
1
Y
i
+a

2
Z
i
+u
i
where h
i
is the ith womans hour of work during a given year, Y
i
is a measure of other income received by
the household, Z
i
is a set of control variables which includes her age, the number of children less than six
years old, the number of children between ages of ve and nineteen her years of schooling, her experience in
years and the square of experience; and u
i
is a stochastic term which is assumed to be normally distributed.
Dene the labor force participation as d
i
= 1 if h
1
> 0 and d
i
= 0 otherwise.
The Tobit likelihood is given by:
p(h|a

s, ) =
N

i=1
_

.5
(a
0
+a
1
Y
i
+a

2
Z
i
)
_
1d
i

.5

.5
(h
i
(a
0
+a
1
Y
i
+a

2
Z
i
))
_
d
i
_
(3.2)
where and are the normal distribution and density functions, respectively.
Sampling with data augmentation: Suppose we could observe h

the potential hours of work for

every married woman which may also be negative. Thence p(a

s, |h

) = p(a

s, |h

, h) is the posterior for

a

s and derived from the linear normal likelihood model. We do not observe h

but its conditional density

p(h

|h = 0, a

s, ) is normally left truncated. Therefore we have the following Gibbs Sampler Algorithm
with the parameter set augmented by h

for censored observations:

1) Choose a initial value for h

, as and ;
2) Sample a

s and from p(a

s, |h

);
3) Sample h

from the truncated normal p(h

|h = 0, a

s, );
4) Repeat 2 and 3 nrep times.
Exercise 5.
(a) Write a BUGS program for the Tobit model.
(b) Write a R program using data augmentation. (In the library msm provides the function tnorm(,
2
))
truncated normal
2
The womans wage rate, the endogenous explanatory variable, is assumed to be a linear function of Y
i
and Z
i
.
5
4 Instrumental Variable Model
4.1 Recursive Equation Model
Database: See Romer (1993)
Model: In a closed economy, the Lucas model for the relationship between output and ination is:
y = y

+(
e
) (4.1)
where y is actual output, y

the natural rate, ination, and

e
expected ination, and where > 0.
Unanticipated monetary shocks aect both prices and real output. The policy-makers objective (welfare)
function is:
W =
1
2

2
+y (4.2)
where > 0. The policy-maker chooses ination . Assuming rational expectations, i.e. private agents
known optimization problem of the policy maker, the equilibrium is =
e
= and y = y

: ination is
positive and output is at natural rate (suboptimal equilibrium).
Openness aects the output-ination trade-o (increased openness raises de amount of ination associated
with a given expansion of domestic output; that is, it reduces ) and the benet of higher output relative to
the cost of higher ination ( is decreasing in the degree of openness).
Romer (1993) considers the following linear relation between ination and output:
log = b
0
+b
1

I
Y
+b
2
log y +b3 dd +b4 rd (4.3)
where log is the log of the average ination 73-93,
I
Y
is the average share of imports in GDP or GNP, log y
is the log of real income per capita, dd are dummies for alternative measures of openness and ination and
rd are regional dummies. A priori we expect that b
1
< 0. According to the author,
I
Y
is endogenous since
the adoption of protectionism policies that benet some interest groups leads to larger budget decits and
therefore ination. As instrument he uses land area (in logarithms).
Exercise 6.
(a) How would you translate the instrumental variable model into a recursive system model? Dene the
likelihood and the priors for the recursive system.
(b) Write a Bugs program for the recursive model. Could you propose an exogeneity test for openness?
4.2 Multinomial Approach (Bayesian Bootstrap)
Database: See Romer (1993)
Model: Another approach for the previous model is to consider only moment conditions. Under mean
independence we have:
E[Z

(y X)] = 0
6
The likelihood is derived from the multinomial distribution and its natural conjugate prior is the dirichlet
distribution. For more details, take a look at Lancaster (2004) chapter 3, section 3.4, pages 141-147.
Exercise 7.
(a) Sample from its posterior and compare to the posterior distribution of the previous exercise. (Remember
that you can sample from
= [Z

GX]
1
Z

Gy
where G is a diagonal matrix whose components are i.i.d exponentially distributed)
(b) How would you propose and exogeneity test in this context?
7
References
Koop, G. (2003), Bayesian Econometrics, rst edn, Wiley.
Lancaster, T. (2004), An Introduction to Modern Bayesian Econometrics, rst edn, Blackwell Publishing.
Mankiw, N. G., Romer, D. & Weil, D. N. (1992), A contribution to the empirics of economic growth,
Quartely Journal of Economics 107(2), 407437.
Mroz, T. A. (1987), The sensitivity of an empirical model of married womens hours of work to economic
and statistical assumptions, Econometrica 55(4), 765799.
Romer, D. (1993), Openness and ination: Theory and evidence, Quartely Journal of Economics
108(4), 869903.
Slonim, R. & Roth, A. E. (1998), Learning in high stakes ultimum games: an experiment in the slovak
republic, Econometrica 66(3), 569596.
Yatchew, A. & No, J. (2001), Household gasoline demand in canada, Econometrica 69(6), 16971709.
8