CH 02

Principles of Econometrics, 4t
h
Edition Page 1 Chapter 2: The Simple Linear Regression Model

Chapter 2
The Simple Linear Regression Model:
Specification and Estimation

Walter R. Paczkowski
Rutgers University
h

2.1 An Economic Model
2.2 An Econometric Model
2.3 Estimating the Regression Parameters
2.4 Assessing the Least Squares Estimators
2.5 The Gauss-Markov Theorem
2.6 The Probability Distributions of the Least
Squares Estimators
2.7 Estimating the Variance of the Error Term
2.8 Estimating Nonlinear Relationships
2.9 Regression with Indicator Variables
Chapter Contents
h

2.1
An Economic Model

h
As economists we are usually more interested in
studying relationships between variables
Economic theory tells us that expenditure on
economic goods depends on income
Consequently we call y the dependent
variable and x the independent or
explanatory variable
In econometrics, we recognize that real-world
expenditures are random variables, and we
want to use data to learn about the relationship
2.1
An Economic
Model
h
The pdf is a conditional probability density
function since it is conditional upon an x
The conditional mean, or expected value, of y is
E(y|x)
The expected value of a random variable is
called its mean value, which is really a
contraction of population mean, the center of
the probability distribution of the random
variable
This is not the same as the sample mean,
which is the arithmetic average of numerical
values
2.1
An Economic
Model
h
Figure 2.1a Probability distribution of food expenditure y given
income x = $1000
2.1
An Economic
Model
h

The conditional variance of y is
2
which
measures the dispersion of y about its mean
y|x

The parameters
y|x
and
2
, if they were known,
would give us some valuable information about
the population we are considering
2.1
An Economic
Model
h
Figure 2.1b Probability distributions of food expenditures y
given incomes x = $1000 and x = $2000
2.1
An Economic
Model
h

In order to investigate the relationship between
expenditure and income we must build an
economic model and then a corresponding
econometric model that forms the basis for a
quantitative or empirical economic analysis
This econometric model is also called a
regression model
2.1
An Economic
Model
h
Eq. 2.1
x x y E
y 2 1
) | ( | | + = =

The simple regression function is written as

where
1
is the intercept and
2
is the slope

Eq. 2.1
2 2
( | )
y
E y x x = = +
2.1
An Economic
Model
h
x x y E
y 2 1
) | ( | | + = =

It is called simple regression not because it is easy,
but because there is only one explanatory variable
on the right-hand side of the equation
2.1
An Economic
Model
h
2.1
An Economic
Model
Figure 2.2 The economic model: a linear relationship between
average per person food expenditure and income
h
Eq. 2.2
denotes change in and dE(y|x)/dx denotes the derivative of
the expected value of y given an x value
dx
x y dE
x
x y E ) | ( ) | (
2
=
A
A
= |

The slope of the regression line can be written as:

where denotes change in and dE(y|x)/dx
denotes the derivative of the expected value of y
given an x value

Eq. 2.2
2
( | ) ( | )
E y x dE y x
x dx
A
= =
A
2.1
An Economic
Model
h

2.2
An Econometric Model

h
2.2
An Econometric
Model
Figure 2.3 The probability density function for y at two levels of income
h

There are several key assumptions underlying the
simple linear regression
More will be added later
2.2
An Econometric
Model
h
ASSUMPTIONS OF THE SIMPLE LINEAR REGRESSION MODEL - I
Assumption 1:
The mean value of y, for each value of x, is
given by the linear regression
1 2
( | ) E y x x = +
2.2
An Econometric
Model
h
Assumption 2:
For each value of x, the values of y are
distributed about their mean value, following
probability distributions that all have the same
variance
2
var( | ) y x =
2.2
An Econometric
Model
h
Assumption 3:
The sample values of y are all uncorrelated,
and have zero covariance, implying that there
is no linear association among them
0 ) , cov( =
j i
y y
This assumption can be made stronger by
assuming that the values of y are all
statistically independent
2.2
An Econometric
Model
h
Assumption 4:
The variable x is not random, and must take at
least two different values
2.2
An Econometric
Model
h
Assumption 5:
(optional) The values of y are normally
distributed about their mean for each value of x
2
1 2
~ ( , ) y N x +
2.2
An Econometric
Model
h
2.2.1
Introducing the
Error Term
The random error term is defined as

Rearranging gives

where y is the dependent variable and x is the
independent variable

( )
1 2
| e y E y x y x = = Eq. 2.3
1 2
y x e = + + Eq. 2.4
2.2
An Econometric
Model
h

The expected value of the error term, given x, is

The mean value of the error term, given x, is zero

1 2
( | ) ( | ) 0 E e x E y x x = =
2.2
An Econometric
Model
2.2.1
Introducing the
Error Term
h
Figure 2.4 Probability density functions for e and y
2.2
An Econometric
Model
2.2.1
Introducing the
Error Term
h
ASSUMPTIONS OF THE SIMPLE LINEAR REGRESSION MODEL - II
Assumption SR1:
The value of y, for each value of x, is:
1 2
y x e = + +
2.2
An Econometric
Model
2.2.1
Introducing the
Error Term
h
Assumption SR2:
The expected value of the random error e is:
This is equivalent to assuming that
0 ) ( = e E
1 2
( ) E y x = +
2.2
An Econometric
Model
2.2.1
Introducing the
Error Term
h
Assumption SR3:
The variance of the random error e is:
2
var( ) var( ) e y = =
The random variables y and e have the
same variance because they differ only by
a constant.
2.2
An Econometric
Model
2.2.1
Introducing the
Error Term
h
Assumption SR4:
The covariance between any pair of random
errors, e
i
and e
j
is:
The stronger version of this assumption is that
the random errors e are statistically independent,
in which case the values of the dependent
variable y are also statistically independent
0 ) , cov( ) , cov( = =
j i j i
y y e e
2.2
An Econometric
Model
2.2.1
Introducing the
Error Term
h
Assumption SR5:
The variable x is not random, and must take at
least two different values
2.2
An Econometric
Model
2.2.1
Introducing the
Error Term
h
Assumption SR6:
(optional) The values of e are normally
distributed about their mean if the values of y
are normally distributed, and vice versa
2
~ (0, ) e N
2.2
An Econometric
Model
2.2.1
Introducing the
Error Term
h
Figure 2.5 The relationship among y, e and the true regression line
2.2
An Econometric
Model
2.2.1
Introducing the
Error Term
h

2.3
Estimating the Regression Parameters

h
2.3
Estimating the
Regression
Parameters
Table 2.1 Food Expenditure and Income Data
h
Figure 2.6 Data for food expenditure example
2.3
Estimating the
Regression
Parameters
h

The fitted regression line is:

The least squares residual is:

2.3.1
The Least Squares
Principle
1 2
i i
y b b x = +
i i i i i
x b b y y y e
2 1

= =
Eq. 2.5
Eq. 2.6
2.3
Estimating the
Regression
Parameters
h
Figure 2.7 The relationship among y, and the fitted regression line
2.3
Estimating the
Regression
Parameters
2.3.1
The Least Squares
Principle
h

Suppose we have another fitted line:

The least squares line has the smaller sum of
squared residuals:

* * *
1 2
i i
y b b x = +
*
1
2 * *
1
2
then
and
if SSE SSE e SSE e SSE

N
i
i
N
i
i
< = =

= =
2.3
Estimating the
Regression
Parameters
2.3.1
The Least Squares
Principle
h

Least squares estimates for the unknown
parameters
1
and
2
are obtained my minimizing
the sum of squares function:
2
1 2 1 2
1
( , ) ( )
N
i i
i
S y x
=
=
2.3
Estimating the
Regression
Parameters
2.3.1
The Least Squares
Principle
h

=
2
2
) (
) )( (
x x
y y x x
b
i
i i
x b y b
2 1
=
Eq. 2.7
Eq. 2.8
2.3
Estimating the
Regression
Parameters
2.3.1
The Least Squares
Principle
THE LEAST SQUARES ESTIMATORS
h
2.3.2
Estimates for the
Food Expenditure
Function
2096 . 10
7876 . 1828
2684 . 18671
) (
) )( (
2
2
= =
x x
y y x x
b
i
i i
4160 . 83 ) 6048 . 19 )( 2096 . 10 ( 5735 . 283
2 1
= = = x b y b
A convenient way to report the values for b
1

and b
2
is to write out the estimated or fitted
regression line:
i i
x y 21 . 10 42 . 83 + =
2.3
Estimating the
Regression
Parameters
h
Figure 2.8 The fitted regression line
2.3
Estimating the
Regression
Parameters
2.3.2
Estimates for the
Food Expenditure
Function
h
The value b
2
= 10.21 is an estimate of |
2
, the
amount by which weekly expenditure on food per
household increases when household weekly
income increases by $100. Thus, we estimate that
if income goes up by $100, expected weekly
expenditure on food will increase by
approximately $10.21
Strictly speaking, the intercept estimate b
1
=
83.42 is an estimate of the weekly food
expenditure on food for a household with zero
income

2.3.3
Interpreting the
Estimates
2.3
Estimating the
Regression
Parameters
h
Income elasticity is a useful way to characterize
the responsiveness of consumer expenditure to
changes in income. The elasticity of a variable y
with respect to another variable x is:

In the linear economic model given by Eq. 2.1 we
have shown that

2.3.3a
Elasticities
y
x
x
y
x
y
A
A
= =
in change percentage
in change percentage
c
2
( )
E y
x
A
=
A
2.3
Estimating the
Regression
Parameters
h
The elasticity of mean expenditure with respect to
income is:

A frequently used alternative is to calculate the
elasticity at the point of the means because it is
a representative point on the regression line.

2
( ) ( ) ( )
( ) ( )
E y E y E y x x
x x x E y E y
c
A A
= = =
A A
Eq. 2.9
71 . 0
57 . 283
60 . 19
21 . 10
2
= = =
y
x
b c
2.3
Estimating the
Regression
Parameters
2.3.3a
Elasticities
h

Suppose that we wanted to predict weekly food
expenditure for a household with a weekly income
of $2000. This prediction is carried out by
substituting x = 20 into our estimated equation to
obtain:

We predict that a household with a weekly income
of $2000 will spend $287.61 per week on food

2.3.3b
Prediction
61 . 287 ) 20 ( 21 . 10 42 . 83 21 . 10 42 . 83
= + = + =
i
x y
2.3
Estimating the
Regression
Parameters
h
2.3.3c
Computer Output
Figure 2.9 EViews Regression Output
2.3
Estimating the
Regression
Parameters
h
2.3.4
Other Economic
Models

The simple regression model can be applied to
estimate the parameters of many relationships in
economics, business, and the social sciences
The applications of regression analysis are
fascinating and useful
2.3
Estimating the
Regression
Parameters
h

2.4
Assessing the Least Squares Fit

h
We call b
1
and b
2
the least squares estimators.
We can investigate the properties of the estimators
b
1
and b
2
, which are called their sampling
properties, and deal with the following important
questions:
1. If the least squares estimators are random
variables, then what are their expected values,
variances, covariances, and probability
distributions?
2. How do the least squares estimators compare
with other procedures that might be used, and
how can we compare alternative estimators?

2.4
Assessing the
Least Squares Fit

h
The estimator b
2
can be rewritten as:

where

It could also be write as:

2.4.1
The Estimator b
2
=
=
N
i
i i
y w b
1
2
=
2
) ( x x
x x
w
i
i
i
2 2
i i
b we = +
Eq. 2.10
Eq. 2.11
Eq. 2.12
2.4
Assessing the
Least Squares Fit

h
We will show that if our model assumptions hold,
then E(b
2
) =
2
, which means that the estimator is
unbiased. We can find the expected value of b
2

using the fact that the expected value of a sum is
the sum of the expected values:

using and

2.4.2
The Expected
Values of b
1
and b
2
2 2 2 1 1 2 2
2 1 1 2 2
2
2
2
( ) ( ) ( ... )
( ) ( ) ( ) ... ( )
( ) ( )
( )
i i N N
N N
i i
i i
E b E b we E we w e w e
E E we E w e E w e
E E we
wE e
= + = + + + +
= + + + +
= +
= +
=
) ( ) (
i i i i
e E w e w E =
0 ) ( =
i
e E
Eq. 2.13
2.4
Assessing the
Least Squares Fit

h
The property of unbiasedness is about the average
values of b
1
and b
2
if many samples of the same size
are drawn from the same population
If we took the averages of estimates from many
samples, these averages would approach the true
parameter values b
1
and b
2

Unbiasedness does not say that an estimate from
any one sample is close to the true parameter value,
and thus we cannot say that an estimate is unbiased
We can say that the least squares estimation
procedure (or the least squares estimator) is
unbiased

2.4
Assessing the
Least Squares Fit

2.4.2
The Expected
Values of b
1
and b
2
h
2.4.3
Repeated
Sampling

Table 2.2 Estimates from 10 Samples
2.4
Assessing the
Least Squares Fit

h
Figure 2.10 Two possible probability density functions for b
2

The variance of b
2
is defined as
2
2 2 2
)] ( [ ) var( b E b E b =
2.4
Assessing the
Least Squares Fit

2.4.3
Repeated
Sampling

h
If the regression model assumptions SR1-SR5 are
correct (assumption SR6 is not required), then the
variances and covariance of b
1
and b
2
are:

2.4.4
The Variances and
Covariances of b
1

and b
2
( )
2
2
1
2
var( )
i
i
x
b
N x x
(
( =
(

( )
2
2
2
var( )
i
b
x x
=
( )
2
1 2
2
cov( , )
i
x
b b
x x
(
( =
(

Eq. 2.14
Eq. 2.15
Eq. 2.16
2.4
Assessing the
Least Squares Fit

h
1. The larger the variance term
2
, the greater the uncertainty
there is in the statistical model, and the larger the variances
and covariance of the least squares estimators.

2. The larger the sum of squares, , the smaller the
variances of the least squares estimators and the more
precisely we can estimate the unknown parameters.

3. The larger the sample size N, the smaller the variances and
covariance of the least squares estimators.

4. The larger the term , the larger the variance of the least
squares estimator b
1
.

5. The absolute magnitude of the covariance increases the
larger in magnitude is the sample mean , and the
covariance has a sign opposite to that of .
( )

2
x x
i
2
i
x
x
x
MAJOR POINTS ABOUT THE VARIANCES AND COVARIANCES
OF b
1
AND b
2
2.4
Assessing the
Least Squares Fit

2.4.4
The Variances and
Covariances of b
1

and b
2
h
Figure 2.11 The influence of variation in the explanatory variable x on
precision of estimation (a) Low x variation, low precision (b) High x
variation, high precision
The variance of b
2
is defined as
| |
2
2 2 2
) ( ) var( b E b E b =
2.4
Assessing the
Least Squares Fit

2.4.4
The Variances and
Covariances of b
1

and b
2
h

2.5
The Gauss-Markov Theorem

h
2.5
The Gauss-Markov
Theorem

Under the assumptions SR1-SR5 of the linear
regression model, the estimators b
1
and b
2
have the
smallest variance of all linear and unbiased
estimators of b
1
and b
2
. They are the Best Linear
Unbiased Estimators (BLUE) of b
1
and b
2

GAUSS-MARKOV THEOREM
h
MAJOR POINTS ABOUT THE GAUSS-MARKOV THEOREM

1. The estimators b
1
and b
2
are best when compared to similar
estimators, those which are linear and unbiased. The Theorem does
not say that b
1
and b
2
are the best of all possible estimators.

2. The estimators b
1
and b
2
are best within their class because they
have the minimum variance. When comparing two linear and
unbiased estimators, we always want to use the one with the
smaller variance, since that estimation rule gives us the higher
probability of obtaining an estimate that is close to the true
parameter value.

3. In order for the Gauss-Markov Theorem to hold, assumptions SR1-
SR5 must be true. If any of these assumptions are not true, then b
1

and b
2
are not the best linear unbiased estimators of
1
and
2
.
2.5
The Gauss-Markov
Theorem

h
4. The Gauss-Markov Theorem does not depend on the assumption
of normality (assumption SR6).

5. In the simple linear regression model, if we want to use a linear
and unbiased estimator, then we have to do no more searching.
The estimators b
1
and b
2
are the ones to use. This explains why
we are studying these estimators and why they are so widely used
in research, not only in economics but in all social and physical
sciences as well.

6. The Gauss-Markov theorem applies to the least squares
estimators. It does not apply to the least squares estimates from a
single sample.
2.5
The Gauss-Markov
Theorem

MAJOR POINTS ABOUT THE GAUSS-MARKOV THEOREM

h

2.6
The Probability Distributions of the
Least Squares Estimators

h
If we make the normality assumption (assumption
SR6 about the error term) then the least squares
estimators are normally distributed:

2.6
The Probability
Distributions of the
Least Squares
Estimators

( )
2 2
1 1
2
~ ,
i
i
x
b N
N x x
| |
|
|
\ .
( )
2
2 2
2
~ ,
i
b N
x x
| |
|
|
\ .
Eq. 2.17
Eq. 2.18
h
If assumptions SR1-SR5 hold, and if the sample
size N is sufficiently large, then the least squares
estimators have a distribution that approximates the
normal distributions shown in Eq. 2.17 and Eq. 2.18
2.6
The Probability
Distributions of the
Least Squares
Estimators

A CENTRAL LIMIT THEOREM
h

2.7
Estimating the Variance of the Error
Term

h
The variance of the random error e
i
is:

if the assumption E(e
i
) = 0 is correct.

Since the expectation is an average value we might
consider estimating
2
as the average of the
squared errors:

where the error terms are

2.7
Estimating the
Variance of the
Error Term

2 2 2
var( ) [ ( )] ( )
i i i i
e E e E e E e = = =
2
2
i
e
N
=

1 2

i i i
e y x =
h
The least squares residuals are obtained by replacing
the unknown parameters by their least squares
estimates:

There is a simple modification that produces an
unbiased estimator, and that is:

so that:

1 2
2
2

i i i i i
i
e y y y b b x
e
N
= =
=

2
2
2
=

N
e
i
o
( )
2 2
E =
Eq. 2.19
2.7
Estimating the
Variance of the
Error Term

h
Replace the unknown error variance
2
in Eq. 2.14
Eq. 2.16 by to obtain:

2.7.1
Estimating the
Variance and
Covariance of the
Least Squares
Estimators
2
o
( )
2
2
1
2
var( )
i
i
x
b
N x x
(
( =
(

( )
2
2
2
var( )
i
b
x x
=
( )
2
1 2
2
cov( , )
i
x
b b
x x
(
( =
(

Eq. 2.20
Eq. 2.21
Eq. 2.22
2.7
Estimating the
Variance of the
Error Term

h

The square roots of the estimated variances are the
standard errors of b
1
and b
2
:

1 1
se( ) var( ) b b =
Eq. 2.23
Eq. 2.24
2 2
se( ) var( ) b b =
2.7
Estimating the
Variance of the
Error Term

2.7.1
Estimating the
Variance and
Covariance of the
Least Squares
Estimators
h
2.7.2
Calculations for the
Food Expenditure
Data
2
2
304505.2
8013.29
2 38
i
e
N
= = =
Table 2.3 Least Squares Residuals

2.7
Estimating the
Variance of the
Error Term

h

The estimated variances and covariances for a
regression are arrayed in a rectangular array, or
matrix, with variances on the diagonal and
covariances in the off-diagonal positions.

1 1 2
1 2 2
var( ) cov( , )
cov( , ) var( )
b b b
b b b
(
(
(

2.7
Estimating the
Variance of the
Error Term

2.7.2
Food Expenditure
Data
h

For the food expenditure data the estimated
covariance matrix is:

2.7.2
Food Expenditure
Data
C Income
C 1884.442 -85.90316
Income -85.90316 4.381752
2.7
Estimating the
Variance of the
Error Term

h
The standard errors of b
1
and b
2
are measures of
the sampling variability of the least squares
estimates b
1
and b
2
in repeated samples.
The estimators are random variables. As such,
they have probability distributions, means, and
variances.
In particular, if assumption SR6 holds, and the
random error terms e
i
are normally distributed,
then:

2.7.3
Interpreting the
Standard Errors
( )
( )
2
2
2 2 2
~ , var( )
i
b N b x x =
2.7
Estimating the
Variance of the
Error Term

h
The estimator variance, var(b
2
), or its square
root, which we might call the true
standard deviation of b
2
, measures the sampling
variation of the estimates b
2
The bigger is the more variation in the least
squares estimates b
2
we see from sample to
sample. If is large then the estimates might
change a great deal from sample to sample
If is small relative to the parameter b
2
,we
know that the least squares estimate will fall
near b
2
with high probability

( )
2
2
var
b
b =
2
b
o
2
b
o
2
b
o
2.7
Estimating the
Variance of the
Error Term

2.7.3
Interpreting the
Standard Errors
h

The question we address with the standard error is
How much variation about their means do the
estimates exhibit from sample to sample?

2.7
Estimating the
Variance of the
Error Term

2.7.3
Interpreting the
Standard Errors
h
We estimate
2
, and then estimate using:

The standard error of b
2
is thus an estimate of
what the standard deviation of many estimates
b
2
would be in a very large number of samples,
and is an indicator of the width of the pdf of b
2
shown in Figure 2.12

2
b
o
( )
2 2
2
2
se( ) var( )
i
b b
x x
=
=
2.7
Estimating the
Variance of the
Error Term

2.7.3
Interpreting the
Standard Errors
h
Figure 2.12 The probability density function of the least squares estimator b
2
.
2.7
Estimating the
Variance of the
Error Term

2.7.3
Interpreting the
Standard Errors
h

2.8
Estimating Nonlinear Relationships

h
Economic variables are not always related by straight-line
relationships; in fact, many economic relationships are
represented by curved lines, and are said to display curvilinear
forms.

Fortunately, the simple linear regression model y =
1
+
2
+ e
is much more flexible than it looks at first glance, because the
variables y and x can be transformations, involving logarithms,
squares, cubes or reciprocals, of the basic economic variables,
or they can be indicator variables that take only the values
zero and one.

Including these possibilities means the simple linear
regression model can be used to account for nonlinear
relationships between variables
2.8
Estimating
Nonlinear
Relationships
THE WORLD IS NOT LINEAR
h

Consider the linear model of house prices:

where SQFT is the square footage.
It may be reasonable to assume that larger and
more expensive homes have a higher value for
an additional square foot of living area than
smaller, less expensive, homes

1 2
PRICE SQFT e = + + Eq. 2.25
2.8
Estimating
Nonlinear
Relationships
h

We can build this into our model in two ways:
1. a quadratic equation in which the explanatory
variable is SQFT
2
2. a loglinear equation in which the dependent
variable is ln(PRICE)
In each case we will find that the slope of the
relationship between PRICE and SQFT is not
constant, but changes from point to point.

2.8
Estimating
Nonlinear
Relationships
h

The quadratic function y =
1
+
2
x
2
is a parabola
The elasticity, or the percentage change in y
given a 1% change in x, is:

2.8.1
Quadratic
Functions
y bx
y x slope
2
2 =
= c
2.8
Estimating
Nonlinear
Relationships
h
Figure 2.13 A quadratic function
2.8
Estimating
Nonlinear
Relationships
2.8.1
Quadratic
Functions
h
A quadratic model for house prices includes the
squared value of SQFT, giving:

The slope is:

If , then larger houses will have larger
slope, and a larger estimated price per
additional square foot

2.8.2
Using a Quadratic
Model
2
1 2
PRICE SQFT e = + +
( )
2
2
d PRICE
SQFT
dSQFT
=
Eq. 2.26
Eq. 2.27
2
0 >
2.8
Estimating
Nonlinear
Relationships
h
Figure 2.14 A fitted quadratic relationship
2.8
Estimating
Nonlinear
Relationships
2.8.2
Using a Quadratic
Model
h
For 1080 houses sold in Baton Rouge, LA during
mid-2005, the estimated quadratic equation is:

The estimated slope is:

The elasticity is:

2
55776 56 0 0154 PRICE . . SQFT = +
( )
2
2
SQFT
slope
PRICE
SQFT
SQFT
PRICE
c =
=
( )
2 0 0154 slope . SQFT =
2.8
Estimating
Nonlinear
Relationships
2.8.2
Using a Quadratic
Model
h

To compute an estimate we must select values for
SQFT and PRICE
A common approach is to choose a point on the
fitted relationship
That is, we choose a value for SQFT and
choose for price the corresponding fitted
value
2.8
Estimating
Nonlinear
Relationships
2.8.2
Using a Quadratic
Model
h

For houses of 2000, 4000 and 6000 square feet,
the estimated elasticities are:

respectively

For a 2000-square-foot house, we estimate that a
1% increase in house size will increase price by
1.05%

1.05 using
1.63 using
1.82 using
7 $117,461.7 CE I
PR =
9 $302,517.3 CE I
PR =
2 $610,943.4 CE I
PR =
2.8
Estimating
Nonlinear
Relationships
2.8.2
Using a Quadratic
Model
h
The log-linear equation ln(y) = a + bx has a
logarithmic term on the left-hand side of the
equation and an untransformed (linear) variable on
the right-hand side
Both its slope and elasticity change at each
point and are the same sign as b
The slope is:

The elasticity, the percentage change in y given
a 1% increase in x, at a point on this curve is:

2.8.3
A Log-Linear
Function
by dx dy =
bx y x slope = = c
2.8
Estimating
Nonlinear
Relationships
h

Using the slope expression, we can solve for a
semi-elasticity, which tells us the percentage
change in y given a 1-unit increase in x:

( )
100
100
dy dx
b
dx
= =
Eq. 2.28
2.8
Estimating
Nonlinear
Relationships
2.8.3
A Log-Linear
Function
h

Consider again the model for the price of a house
as a function of the square footage, but now
written in semi-log form:

This logarithmic transformation can regularize
data that is skewed with a long tail to the right

2.8.4
Using a Log-Linear
Model
( )
1 2
ln PRICE SQFT e = + + Eq. 2.29
2.8
Estimating
Nonlinear
Relationships
h
Figure 2.16 (a) Histogram of PRICE (b) Histogram of ln(PRICE)
2.8
Estimating
Nonlinear
Relationships
2.8.4
Using a Log-Linear
Model
h

Using the Baton Rouge data, the fitted log-linear
model is:

To obtain predicted price take the anti-
logarithm, which is the exponential function:

( )
ln 10.8386 0.0004113 PRICE SQFT = +
( )
( )
exp ln exp 10.8386 0.0004113 PRICE PRICE SQFT
(
= = +

2.8
Estimating
Nonlinear
Relationships
2.8.4
Using a Log-Linear
Model
h
Figure 2.17 The fitted log-linear model
2.8
Estimating
Nonlinear
Relationships
2.8.4
Using a Log-Linear
Model
h

The slope of the log-linear model is:

For a house with a predicted PRICE of $100,000,
the estimated increase in PRICE for an additional
square foot of house area is $41.13, and for a
house with a predicted PRICE of $500,000, the
estimated increase in PRICE for an additional
square foot of house area is $205.63

( )
2
0.0004113
d PRICE
PRICE PRICE
dSQFT
= =
2.8
Estimating
Nonlinear
Relationships
2.8.4
Using a Log-Linear
Model
h
The estimated elasticity is:

For a house with 2000-square-feet, the
estimated elasticity is 0.823:
A 1% increase in house size is estimated to
increase selling price by 0.823%
For a house with 4000 square feet, the
estimated elasticity is 1.645:
A 1% increase in house size is estimated to
increase selling price by 1.645%

2

0.0004113 SQFT SQFT c = =
2.8
Estimating
Nonlinear
Relationships
2.8.4
Using a Log-Linear
Model
h

Using the semi-elasticity defined in Eq. 2.28 we
can say that, for a one-square-foot increase in size,
we estimate a price increase of 0.04%
Or, perhaps more usefully, we estimate that a
100-square-foot increase will increase price by
approximately 4%.

2.8
Estimating
Nonlinear
Relationships
2.8.4
Using a Log-Linear
Model
h

We should do our best to choose a functional form
that is:
consistent with economic theory
that fits the data well
that is such that the assumptions of the
regression model are satisfied

2.8.5
Choosing a
Functional Form
2.8
Estimating
Nonlinear
Relationships
h

In real-world problems it is sometimes difficult to
achieve all these goals
Furthermore, we will never truly know the
correct functional relationship, no matter how
many years we study econometrics
The truth is out there, but we will never know it
In applications of econometrics we must simply
do the best we can to choose a satisfactory
functional form

2.8
Estimating
Nonlinear
Relationships
2.8.5
Choosing a
Functional Form
h

2.9
Regression with Indicator Variables

h

An indicator variable is a binary variable that takes the
values zero or one; it is used to represent a
nonquantitative characteristic, such as gender, race, or
location

How do we model this?

2.9
Regression with
Indicator Variables
=
Oaks Golden in is house 0
Town ity in Univers is house 1
UTOWN
1 2
PRICE UTOWN e = + +
h
Figure 2.18 Distributions of house prices
2.9
Regression with
Indicator Variables
h
( )
1 2
1
if 1
if 0
UTOWN
E PRICE
UTOWN
+ =
=

=
1 2
215.7325 61.5091
277.2416 if 1
215.7325 if 0
PRICE b b UTOWN
UTOWN
UTOWN
UTOWN
= +
= +
=
=

=
2.9
Regression with
Indicator Variables
When an indicator variable is used in a regression,
it is important to write out the regression function
for the different values of the indicator variable

The estimated regression is:

( )
1 2
1
if 1
if 0
UTOWN
E PRICE
UTOWN
+ =
=

=
1 2
215.7325 61.5091
277.2416 if 1
215.7325 if 0
PRICE b b UTOWN
UTOWN
UTOWN
UTOWN
= +
= +
=
=

=
h

The least squares estimators b
1
and b
2
in this
indicator variable regression can be shown to be:

In the simple regression model, an indicator
variable on the right-hand side gives us a way
to estimate the differences between population
means

Oaks Golden Town University
2
Oaks Golden
1
PRICE PRICE b
PRICE b
=
=
2.9
Regression with
Indicator Variables
h

Key Words

h
assumptions
asymptotic
B.L.U.E.
biased estimator
degrees of freedom
dependent variable
deviation from the
mean form
econometric model
economic model
elasticity
Gauss-Markov
Theorem
heteroskedastic
homoskedastic
independent
variable
least squares
estimates
least squares
estimators
least squares
principle
least squares
residuals
linear estimator
prediction
random error term
regression model
regression
parameters
repeated sampling
sampling precision
sampling properties
scatter diagram
simple linear
regression function
specification error
unbiased estimator

Keywords
h

Appendices

h
2A Derivation of the Least Squares Estimates
2B Deviation from the Mean Form of b
2
2C b
2
is a Linear Estimator
2D Derivation of Theoretical Expression for b
2
2E Deriving the Variance of b
2
2F Proof of the Gauss-Markov Theorem
h
2A
Derivation of the
Least Squares
Estimates
Eq. 2A.1
Eq. 2A.2
( )
( ) ( )
2
1 2 1 2
1
1 2
1
2
2 1
2
( , ) ( )
2 2 2
2 2 2
N
i i
i
i i
i i i i
S y x
S
N y x
S
x x y x
=
=
c
= +
c
c
= +
c

h
Figure 2A.1 The sum of squares function and the minimizing values b
1
and b
2
2A
Derivation of the
Least Squares
Estimates
h
Eq. 2A.3
Eq. 2A.4
( ) | |
( ) ( ) | | 0 2
0 2
2
2
1
2 1
=
=

b x b x y x
b x Nb y
i i i i
i i
Set the derivatives equal to zero to get two
equations:
( )
( ) ( )
1 2
2
1 2
i i
i i i i
Nb x b y
x b x b x y
+ =
+ =

Simplify these to:
2A
Derivation of the
Least Squares
Estimates
h
Eq. 2A.5
Solving the two equations simultaneously, we
get for b
2
:
( )

=
2
2
2
i i
i i i i
x x N
y x y x N
b
2A
Derivation of the
Least Squares
Estimates
h
2B
Deviation from the
Mean Form of b
2
Eq. 2B.1
We can rewrite the equation 2A.5 by noting
the following:

( )
2 2
2 2 2
2 2
2 2
2
1
2
2 2
x N x
x N x N x
x N x
N
N x x
x N x x x x x
i
i
i i
i i i
=
+ =
+
|
.
|
\
|
=
+ =

Also note that:
( )
( )
N
x
x
x x x
x N x x x
i
i
i i
i i
2
2
2
2 2
2

=
=
=
Eq. 2B.2
h
Eq. 2B.3
Finally, we have:

( )( )
N
y x
y x
y x N y x y y x x
i i
i i
i i i i

=
=

2B
Deviation from the
Mean Form of b
2
h
We can rewrite b
2
in deviation from the mean
form as:
( )( )
( )

=
2
2
x x
y y x x
b
i
i i
2B
Deviation from the
Mean Form of b
2
h
First note that we will always have:
( ) 0 =
x x
i
2B
Deviation from the
Mean Form of b
2
h
Now rewrite our formula for b
2
and use this fact:
( )( )
( )
( ) ( )
( )
( )
( )
( )
( )
=
(
(

=
i i
i
i
i
i
i i
i
i i i
i
i i
y w
y
x x
x x
x x
y x x
x x
x x y y x x
x x
y y x x
b
2
2
2
2
2
2C
b
2
is a Linear
Estimator
h
2D
Derivation of
Theoretical
Expression for b
2
To obtain Eq. 2.12, replace y
i
in Eq. 2.11 by
y
i
=
1
+
2
x
i
+e
i
and simplify:
( )
2
1 2
1 2
2

i i
i i i
i i i i i
i i
b w y
w x e
w wx we
we
=
= + +
= + +
= +
h
For this, we used the facts that:
( )
( )
( )
( )
( )
2
2
2 2
1

0
1

0
i
i
i
i
i
i i
i i
i
x x
w
x x
x x
x x
w x
w x
x x
(
( =
(

=
=
=
=
=
2D
Derivation of
Theoretical
Expression for b
2
h
We can show that by using:
( ) ( )( )
( ) ( )
( )
i i
i i i
i i i
x x x
x x x x x x
x x x x x x

=
=
=
2
so that:
=1
i i
x w
( )
( )
( )
( )
1

2
=

i i
i i
i
i i
i i
x x x
x x x
x x
x x x
x w
2D
Derivation of
Theoretical
Expression for b
2
h
2E
Deriving the
Variance of b
2
First note that:
2 2
i i
b we = +
and that:
| |
2
2 2 2
) ( ) var( b E b E b =
h
( ) ( )
( )
2
2 2 2
2
2 2
2 2
i
2 2
2
2
var( )
2
E e 2
i i
i i
i i i j i j
i j
i i j i j
i j
i
i
b E we
E we
E w e ww e e
w ww E e e
w
x x
=
=
( = +

( =

(
= +
(

= +
=
=
2E
Deriving the
Variance of b
2
h
| | | |
( )
( )
( ) ( ) ( )
( )
( )
( )
( )
{ }
( )
( )
{ }
( )
2 2
2 2
2 2
2
2 2 2
2 2
2 2
var( ) ( ) 0
cov , 0
1
var( ) var( ) var( ) 2cov( , )
i i i i i
i j i i j j i j
i i
i
i
i i
e E e E e E e E e
e e E e E e e E e E e e
x x x x
w
x x
x x x x
aX by a X b Y X Y
= = = =
(
= = =

(

(
= = =
(

(

+ = + +

2E
Deriving the
Variance of b
2
h
( )
( )
( )
2 2
2
2
2 2
2
2
var( ) var
var cov( , )
var( )
i i
i i i j i j
i j
i i
i
i
b we
w e ww e e
w e
w
x x
|
=
= +
= +
=
=
=
[since is a constant]
2
[generalizing the variance rule]

[using cov(e
i
,e
j
) = 0]
[using var(e
i
) = ]
2
o
2E
Deriving the
Variance of b
2
h
2F
Proof of the
Gauss-Markov
Theorem

To begin, let be any other linear
estimator of
1
. Also, suppose k
i
= w
i
+ c
i
.
Then:
=
i i
y k b
*
2
( ) ( )( )
( ) ( ) ( )
( )
( )
*
2 1 2
1 2
1 1 2 2
1 2 2

i i i i i i i i i
i i i i i i i i
i i i i i i i i i
i i i i i i
b k y w c y w c x e
w c w c x w c e
w c w x c x w c e
c c x w c e
= = + = + + +
= + + + + +
= + + + + +
= + + + +

Eq. 2F.1
h
Now:
( )
( ) ( )
( )
*
2 1 2 2
1 2 2
*
2 2

0 and 0
i i i i i i
i i i
i i i
i i i i i
E b c c x w c E e
c c x
c c x
b k y w c e
= + + + +
= + +
= =
= = + +

Eq. 2F.2
Eq. 2F.3
Eq. 2F.4
2F
Proof of the
Gauss-Markov
Theorem

h
( )
( ) ( ) ( )
2 2 2
1
0
i i
i i i i i
i i i
c x x
x
c w c x c
x x x x x x
(
= = = (

(

For now, observe that :
Now we can write:
( )
( ) ( )
( )
( )
( )
*
2 2
2
2
2
2 2 2 2
2 2
2
2
var( ) var
var

var
var
i i i
i i i
i i
i i
i
b w c e
w c e
w c
w c
b c
b
( = + +

= +
= +
= +
= +
>
2F
Proof of the
Gauss-Markov
Theorem

h
Monte Carlo simulation experiments use random
number generators to replicate the random way that
data are obtained
In Monte Carlo simulations we specify a data
generation process and create samples of artificial
data
Then we try out estimation methods on the
data we have created
We create many samples of size N and examine the
repeated sampling properties of the estimators
In this way, we can study how statistical
procedures behave under ideal, as well as not so
ideal, conditions

2G
Monte Carlo
Simulation

h
The data generation process for the simple linear
regression model is given by:

Each value of the dependent variable y
i
is
obtained, or generated, by adding a random
error e
i
to the regression function E(y
i
|x
i
)
To simulate values of y
i
we create values for the
systematic portion of the regression relationship
E(y
i
|x
i
) and add to it the random error e
i

( )
N i
e x b b
e x y E y
i i
i i i i
,..., 1
|
2 1
=
+ + =
+ =
2G
Monte Carlo
Simulation

h

To create the variables for the regression function,
we must:
1. Select sample size N
2. Choose x
i
values
3. Choose
1
and
2

2G.1
The Regression
Function
2G
Monte Carlo
Simulation

h
To be consistent with assumptions SR2SR4 the
random errors should have mean zero, a constant
variance and be uncorrelated with one another
We can generate random numbers
Of course the computer-generated numbers
cannot be truly random, because they are
generated by a computer code
The random numbers created by computer
software are pseudorandom, in that they
behave like random numbers

2G.2
The Random Error
2G
Monte Carlo
Simulation

h
2G.3
Theoretically True
Values
Figure 2G.1 The true probability density functions of the data
2G
Monte Carlo
Simulation

h
Figure 2G.2 The true probability density functions of the estimator b
2

2G
Monte Carlo
Simulation

2G.3
Theoretically True
Values
h
2G.4
Creating a Sample
of Data
( )( ) ( )
75.7679 11.9683
se 25.7928 1.6313
51.5857
i i
y x = +
=
b
1
b
2
b
1
665.2699 -39.9162
b
2
-39.9162 2.6611
Variance-Covariance Matrix
2G
Monte Carlo
Simulation

h
What do we hope to achieve with a Monte Carlo
experiment?
We would like to verify that under SR1SR5
the least squares estimators are unbiased
We would like to verify that under SR1SR5
the least squares estimators have sampling
variances given by Eq. 2.14 and Eq. 2.16
We would like to verify that the estimator of the
error variance Eq. 2.19 is unbiased
Because we have assumed the random errors
are normal, SR6, we expect the least squares
estimates to have a normal distribution.

2G.5
Monte Carlo
Objectives
2G
Monte Carlo
Simulation

h
2G.6
Monte Carlo
Results
Table 2G.1 Summary of 1,000 Monte Carlo Samples
2G
Monte Carlo
Simulation

h
Figure 2G.3 The sampling distribution of b
2
in 1000 Monte Carlo samples
2G
Monte Carlo
Simulation

2G.6
Monte Carlo
Results

CH 02

Încărcat de

Informații document

Descriere originală:

Titlu original

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

CH 02

Încărcat de

Drepturi de autor:

Formate disponibile

Principles of Econometrics, 4t

if SSE SSE e SSE e SSE

Table 2.3 Least Squares Residuals

[generalizing the variance rule]

S-ar putea să vă placă și