Ojs20120500012 14277069

Open Journal of Statistics, 2012, 2, 534-546
http://dx.doi.org/10.4236/ojs.2012.25069 Published Online December 2012 (http://www.SciRP.org/journal/ojs)
Estimators of Linear Regression Model and Prediction

under Some Assumptions Violation
Kayode Ayinde1, Emmanuel O. Apata2, Oluwayemisi O. Alaba2
1
Department of Statistics, Ladoke Akintola University of Technology, Ogbomoso, Nigeria
2
Department of Statistics, University of Ibadan, Ibadan, Nigeria
Email: bayoayinde@yahoo.com
Received August 21, 2012; revised October 23, 2012; accepted November 5, 2012
ABSTRACT
The development of many estimators of parameters of linear regression model is traceable to non-validity of the as-
sumptions under which the model is formulated, especially when applied to real life situation. This notwithstanding,
regression analysis may aim at prediction. Consequently, this paper examines the performances of the Ordinary Least
Square (OLS) estimator, Cochrane-Orcutt (COR) estimator, Maximum Likelihood (ML) estimator and the estimators
based on Principal Component (PC) analysis in prediction of linear regression model under the joint violations of the
assumption of non-stochastic regressors, independent regressors and error terms. With correlated stochastic normal vari-
ables as regressors and autocorrelated error terms, Monte-Carlo experiments were conducted and the study further iden-
tifies the best estimator that can be used for prediction purpose by adopting the goodness of fit statistics of the estima-
tors. From the results, it is observed that the performances of COR at each level of correlation (multicollinearity) and
that of ML, especially when the sample size is large, over the levels of autocorrelation have a convex-like pattern while
that of OLS and PC are concave-like. Also, as the levels of multicollinearity increase, the estimators, except the PC es-
timators when multicollinearity is negative, rapidly perform better over the levels autocorrelation. The COR and ML
estimators are generally best for prediction in the presence of multicollinearity and autocorrelated error terms. However,
at low levels of autocorrelation, the OLS estimator is either best or competes consistently with the best estimator, while
the PC estimator is either best or competes with the best when multicollinearity level is high    0.8 or   0.49  .
Keywords: Prediction; Estimators; Linear Regression Model; Autocorrelated Error Terms;

Correlated Stochastic Normal Regressors
1. Introduction is known to possess some ideal or optimum properties of

an estimator which include linearity, unbiasedness and
Linear regression model is probably the most widely
efficiency [1]. These had been summed together as Best
used statistical technique for solving functional relation-
Linear Unbiased Estimator (BLUE). However, these
ship problems among variables. It helps to explain ob-
assumptions are not satisfied in some real life situation.
servations of a dependent variable, y, with observed
values of one or more independent variables, X1, X2,  , Consequently, various methods of estimation of the mo-
Xp. In an attempt to explain the dependent variable, pre- del parameters have been developed.
diction of its values often becomes very essential and The assumption of non-stochastic regressors is not al-
necessary. Moreover, the linear regression model is for- ways satisfied, especially in business, economic and so-
mulated under some basic assumptions. Among these as- cial sciences because their regressors are often gene-
sumptions are regressors being assumed to be non-sto- rated by stochastic process beyond their control. Many
chastic (fixed in repeated sampling) and independent. authors, including Neter and Wasserman [2], Fomby et al.
The error terms also assumed to be independent, have [3], Maddala [4] have given situations and instances
constant variance and are also independent of the re- where this assumption may be violated and have also dis-
gressors. When all these assumptions of the classical cussed its consequences on the OLS estimator when used
linear regression model are satisfied, the Ordinary Least to estimate the model parameters. They emphasized that
Square (OLS) estimator given as: if regressors are stochastic and independent of the error
terms, the OLS estimator is still unbiased and has mini-
 
1
ˆ  X 1 X X 1Y (1) mum variance even though it is not BLUE. They also
Copyright © 2012 SciRes. OJS

K. AYINDE ET AL. 535
pointed out that the traditional hypothesis testing remains scending eigen values, 1  2     p and that the
valid if the error terms are further assumed to be normal. last of these eigen values are approximately equal to zero.
However, modification is required in the area of confi- In principal components regression, the principal com-
dence interval calculated for each sample and the power ponents corresponding to near zero eigen values are re-
of the test. moved from the analysis and the least squares applied to
The violation of the assumption of independent re- the remaining component.
gressors leads to multicollinearity. With strongly interre- When all the assumptions of the Classical Linear
lated regressors, interpretation given to the regression co- Regression Model hold except that the error terms are not
efficients may no longer be valid because the assump-
tion under which the regression model is built has been
 
homoscedastic i.e. E U1U    2 I n but are heterosce-
violated. Although the estimates of the regression coeffi- dastic  i.e. E U U      , the resulting model is the
1
2
cients provided by the OLS estimator is still unbiased as Generalized Least Squares (GLS) Model. Aitken [15] has
long as multicollinearity is not perfect, the regression shown that the GLS estimator β of β given as
coefficients may have large sampling errors which affect
   X 11 X  X 11Y is efficient among the class of
1
both the inference and forecasting resulting from the
model [5]. Various methods have been developed to esti- linear unbiased estimators of β with variance-cova-
mate the model parameters when multicollinearity is
 
1
present in a data set. These estimators include Ridge Re- riance matrix of β given as V      2 X 11 X ,
gression estimator developed by Hoerl [6] and Hoerland where Ω is assumed to be known. The GLS estimator
Kennard [7], Estimator based on Principal Component described requires Ω, and in particular ρ to be known
Regression suggested by Massy [8], Marquardt [9] and before the parameters can be estimated. Thus, in linear
Bock, Yancey and Judge [10], Naes and Marten [11], and model with autocorrelated error terms having AR(1):
method of Partial Least Squares developed by Hermon
ˆ GLS    X 11 X  X 1 1Y
1
Wold in the 1960s [12-14]. (3)
The methodology of the biased estimator of regression
coefficients due to principal component regression in- and
volves two stages. This two-stage procedure first reduces
   
1
the predictor variables using principal component analy- V ˆ GLS    2 X 1 1 X (4)
sis and then uses the reduced variables in an OLS regres-
sion fit. While it often works well in practice, there is no where
general theoretical reason that the most informative li-
near function of the predictor variables should lie among E UU     2 
the dominant principal components of the multivariate
distribution of the predictor variables.  1  2   n2  n 1 
 
Consider the linear regression model,   1    n 3  n2 
Y  X  (2)  2  1   n4  n 3 
2  
       
Let X X  T  T , where   diag  1 , 2 , ,  p  is
1 1
  n2
a pxp diagonal matrix of the eignvalues of X 1 X and T  n 3  n4  1  
 n 1 
is a p × p orthogonal matrix whose columns are the ei-    n2  n 3   1 
genvectors associated with 1 , 2 , ,  p . Then the above
model can be written as:  2
and  2   u2  , and the inverse of Ω is
Y  XTT 1     Z    (3)
1   2 
 1  0  0 
0
where Z  XT , Z 1 Z  T 1 X 1 XT  T 1T  T 1T  .   1   2 
   0 0

The columns of Z, which define a new set of orthogo-
1  0  1   2  0 
0
nal regressors, such as Z  Z1 , Z 2 , , Z p are referred to  1   
as principle components. The principle components re- 1   
2
       
gression approach combats multicollinearity by using  0 0 0  1  2  
less than the full set of principle components in the mo-  
 0 0 0   1 
del. Using all will give back into the result of the OLS
estimator. To obtain the principle component estimator, Now with a suitable  n  1 xn matrix transformation
assume that the regressors are arranged in order of de- P defined by

536 K. AYINDE ET AL.
  1 0  0 0 Iyaniwura and Nwabueze [27], Nwabueze [28-30], Ay-

 0  1  0 0  inde and Ipinyomi [31] and many other authors have not
 only examined these estimators but have also noted that
P   0 0   0 0 (5) their performances and efficiency depend on the structure
 
       of the regressor used. Rao and Griliches [32] did one of
 0 0 0   1   n 1  n the earliest Monte-Carlo investigations on the small
sample properties of several two-stage regression me-
Multiplying then shows that P P gives an n × n thods in the context of autocorrelated error terms. Other
matrix which, apart from a proportional constant, is iden- recent works done on these estimators and the violations
tical with  1 except for the first elements in the lead- of the assumptions of classical linear regression model
ing diagonal, which is  2 rather than unity. With an- include that of Ayinde and Iyaniwura [33], Ayinde and
other n × n transformation matrix P obtained from P Oyejola [34], Ayinde [35], Ayinde and Olaomi [36],
by adding a new row with 1   2 in the first position Ayinde and Olaomi [37] and Ayinde [38].
In spite of these several works on these estimators,
and zero elsewhere, that is
none has actually been done on prediction especially as it
 1  2 0 
 
1
2
0 0  0 relates multicollinearity problem. Therefore, this paper
 does not only examine the predictive ability of some of
  1 0  0 0 these estimators but also does it under some violations of
P   0  1  0 0
 (6) assumption of regression model making the model much
  closer to reality.
      
 0 0 0   1  nxn
2. Materials and Methods

Multiplying shows that P P  1    . The dif- 2
 1
Consider the linear regression model of the form:
ference between P and P lies only in the treatment of
the first sample observation. However, when n is large, Yt   0  1 X 1t   2 X 2t   3 X 3t  ut (7)
the difference is negligible, but in small sample, the dif-
ference can be major. If Ω or more precisely ρ is known,  
where ut   ut 1   t ,  t ~ N 0,  2 , t  1, 2,3, , n and
the GLS estimation could be achieved by applying the X i ~ N  0,1 , i  1, 2,3 are stochastic and correlated.
OLS via the transformation matrix P and P above. For Monte-Carlo simulation study, the parameters of
However, this is not often the case; we resort to estimat- equation (1) were specified and fixed as β0 = 4, β1 = 2.5,
ing Ω to have a Feasible Generalized Least Squares Es- β2 = 1.8 and β3 = 0.6. The levels of intercorrelation
timator. This estimator becomes feasible when ρ is re- (multicollinearity) among the independent variables were
placed by a consistent estimator ̂ [3]. There are sev- sixteen (16) and specified as:
eral ways of consistently estimating ρ, however, some of   x12     x13     x23   
them either use the P or P transformation matrix.
Several authors have worked on this violation espe-  0.49, 0.4, 0.3, , 0.8, 0.9, 0.99.
cially in terms of the parameters’ estimation of the linear The levels of autocorrelation is twenty-one (21) and
regression model with autoregressive of orders one. The are specified as   0.99, 0.9, 0.8, , 0.8, 0.9, 0.99.
OLS estimator is inefficient even though unbiased. Its Furthermore, the experiment was replicated in 1000
predicted values are also inefficient and the sampling times  R  1000  under six (6) levels of sample sizes
variances of the autocorrelated error terms are known to
 n  10,15, 20,30,50,100  . The correlated stochastic nor-
be underestimated causing the t and the F tests to be in- mal regressors were generated by using the equations
valid [3-5] and [16]. To compensate for the loss of effi- provided by Ayinde [39] and Ayinde and Adegboye [40]
ciency, several feasible GLS estimators have been devel- to generate normally distributed random variables with
oped. These include the estimator provided by Cochrane specified intercorrelation. With P  3 , the equations
and Orcutt [17], Paris and Winstern [18], Hildreth and Lu give:
[19], Durbin [20], Theil [21], the Maximum Likelihood
and the Maximum Likelihood Grid [22], and Thornton X 1  1   1 Z1
[23]. Among others, the Maximum Likelihood and Maxi- X 2   2  12 2  m22 Z 2 (8)
mum Likelihood Grid impose stationary by constraining
the serial correlation coefficient to be between –1 and 1 m23
X 3  3  13 3 Z1  Z 2  n33 Z 3
and keep the first observation for estimation while that of m22
Cochrane and Orcutt and Hildreth and Lu drops the first
observation. Chipman [24], Kramer [25], Kleiber [26], where 
m22   22 1  122  , m23   2 3   23  12 13 

2
m23 were used. The Adjusted Coefficient of Determination of
and n33  m33  ; and Z i ~ N  0,1 , i  1, 2, 3. the model was averaged over the numbers of replications.
m22
i.e.
By these equations, the inter-correlation matrix has to
be positive definite and hence, the correlations among the 1 R 2
independent variables were taken as prescribed earlier
R  Ri
R i 1
(11)
 12  13   23    . In the study, we assumed An estimator is the best if its Adjusted Coefficient of
X i ~ N  0,1 , i  1, 2,3. Determination is the closest to unity.
The error terms were generated using one of the dis-
tributional properties of the autocorrelated error terms 3. Results and Discussion
   2  The full summary of the simulated results of each esti-
 ut ~ N  0,  2   and the AR(1) equation as follows:
  1    mator at different level of sample size, muticollinearity,
and autocorrelation is contained in the work of Apata
1 [42]. The graphical representations of the results when n
1  (9)
1  2 = 10, 15, 20, 30, 50 and 100 are respectively presented in
Figures 1, 2, 3, 4, 5 and 6.
ut   ut 1   t , t  2,3, 4, , n (10) From these figures, it is observed that the perfor-
Since some of these estimators have now been incor- mances of COR at each level of multicollinearity and
porated into the Time Series Processor (TSP 5.0) [41] those of ML, especially when the sample size is large,
software, a computer program was written using the over the levels of autocorrelation have a convex-like pat-
software to examine the goodness of fit statistics of the tern, while those of OLS, PC1 and PC2 are generally
estimators by calculating their Adjusted Coefficient of concave-like. Also, as the level of multicollinearity in-
Determination of the model R 2 . The estimators are   creases the estimators, except PC estimators when mul-
Ordinary Least Square (OLS), Cochrane Orcutt (COR), ticolinearity is negative, rapidly perform better as their
Maximum Likelihood (ML) and the estimator based on averaged adjusted coefficient of determination increases
Principal Component (PC) Analysis.The two possible over the levels of autocorrelation. The PC estimators
PCs (PC1 and PC2) of the Principal Component Analysis perform better as multicollinearity level increases in its
1
Averaged Adjusted R2
0.8
0.6
0.4 OLS
0.2 COR
0 ML
‐0.99
‐0.99
‐0.99
‐0.99
0.2
‐0.7
0.5
‐0.4
0.8
‐0.1
0.2
‐0.7
0.5
‐0.4
0.8
‐0.1
0.2
‐0.7
0.5
‐0.4
0.8
‐0.1
0.2
‐0.7
0.5
‐0.4
0.8
‐0.1
PC1
PC2
‐0.49 ‐0.1 0.3 0.7
Multicollinearity Levels
Figure 1. Predictive ability of the estimators at each level of multicollinearity and all levels of autocorrelation when n = 10.
1
0.8
0.6
0.4 OLS
0.2 COR
ML
0
PC1
‐0.99
‐0.99
‐0.99
‐0.99
0.2
0.5
0.8
0.2
0.5
0.8
0.2
0.5
0.8
0.2
0.5
0.8
‐0.7
‐0.4
‐0.1
‐0.7
‐0.4
‐0.1
‐0.7
‐0.4
‐0.1
‐0.7
‐0.4
‐0.1
PC2
‐0.49 ‐0.1 0.3 0.7


0.8
0.6
0.4 OLS
0.2 COR
0
ML
0.2
0.5
0.8
0.2
0.5
0.8
0.2
0.5
0.8
0.2
0.5
0.8
‐0.7
‐0.4
‐0.1
‐0.7
‐0.4
‐0.1
‐0.7
‐0.4
‐0.1
‐0.7
‐0.4
‐0.1
‐0.99
‐0.99
‐0.99
‐0.99
PC1
PC2
‐0.49 ‐0.1 0.3 0.7
1
0.8
0.6
0.4 OLS
COR
0.2
ML
0 PC1
‐0.99
‐0.99
‐0.99
‐0.99
0.2
‐0.7
0.5
‐0.4
0.8
‐0.1
0.2
‐0.7
0.5
‐0.4
0.8
‐0.1
0.2
‐0.7
0.5
‐0.4
0.8
‐0.1
0.2
‐0.7
0.5
‐0.4
0.8
‐0.1
PC2
‐0.49 ‐0.1 0.3 0.7

1
0.8
0.6
0.4 OLS
0.2 COR
0 ML
‐0.99
‐0.99
‐0.99
‐0.99
0.2
‐0.7
0.5
‐0.4
0.8
‐0.1
0.2
‐0.7
0.5
‐0.4
0.8
‐0.1
0.2
‐0.7
0.5
‐0.4
0.8
‐0.1
0.2
‐0.7
0.5
‐0.4
0.8
‐0.1
PC1
PC2
‐0.49 ‐0.1 0.3 0.7
1
0.8
0.6
0.4 OLS
COR
0.2
ML
0
PC1
‐0.1
‐0.4
0.8
‐0.1
0.2
‐0.7
0.5
‐0.4
0.8
0.2
‐0.7
0.5
‐0.1
0.2
‐0.7
0.5
‐0.4
0.8
‐0.1
0.2
‐0.7
0.5
‐0.4
0.8
‐0.99
‐0.99
‐0.99
‐0.99
PC2
‐0.49 ‐0.1 0.3 0.7


absolute value. The COR and ML estimators are gene- average adjusted co-efficient of determination of the ML
rally good for prediction in the presence of multicollin- and COR estimators is often greater than 0.8. The OLS
earity and autocorrelated error term. However, at low estimator consistently performs well and competes with
levels of autocorrelation, the OLS estimator is either best the ML and COR estimators at low and occasionally at
or competes consistently with the best estimator, while moderate levels of autocorrelation in all the levels of mul-
the PC2 estimator is also either best or competes with the ticollinearity. Also, the PR1 and PR2 do perform well
best when multicollinearity is high and compete with ML and COR except at high and very
   0.8 or   0.49  . high level of autocorrelation when   0.7 or   0.49 .
Specifically, according to Figure 1 when n = 10, the The best estimator for prediction is summarized in Table 1.
Table 1. The best estimator for prediction at different level of multicollinearity and autocorrelation when n = 10.
λ
ρ
–0.49 –0.4 –0.3 –0.2 –0.1 0 0.1 0.2
–0.99 COR COR COR COR COR COR COR COR
–0.3 PR2 ML ML ML ML COR COR COR
–0.2 PR2 ML ML ML ML ML ML ML
–0.1 PR2 ML ML ML ML ML ML ML
0 PR2 ML ML ML ML ML ML ML
0.1 PR2 ML OLS OLS OLS OLS OLS OLS
0.2 PR2 ML OLS OLS OLS OLS OLS OLS
0.3 PR2 ML ML ML ML ML ML ML
0.6 COR COR COR COR COR COR COR COR
λ
ρ
0.3 0.4 0.5 0.6 0.7 0.8 0.9 0.99
–0.3 COR COR COR COR COR COR COR PR2
–0.2 ML ML ML ML COR COR PR2 PR2
–0.1 ML ML ML ML ML ML PR2 PR2
0 ML ML OLS OLS OLS OLS PR2 PR2
0.1 OLS OLS OLS OLS OLS OLS PR2 PR2
0.2 OLS OLS OLS OLS OLS OLS PR2 PR2
0.3 ML ML ML ML ML ML PR2 PR2
0.4 ML ML ML ML ML ML PR2 PR2
0.5 ML ML ML ML ML ML ML PR2

From Table 1, when n = 10, the COR estimator is best When n = 15, Figure 2 reveals that the pattern of the
except when 0.3    0.5 . At these instances, the PC2 results is not different from when n = 10 except that PR
estimator is often best when   0.9 and   0.49. estimators now compete very well with the ML and COR
Moreover, when 0    0.2 and 0.3    0.8 , the when   0.8 . The best estimator for prediction is pre-
OLS estimator is generally best. At other instances, the sented in Table 2.
best estimator is frequently ML and very sparsely COR. According to Table 2, the COR estimator is generally
λ
ρ
–0.49 –0.4 –0.3 –0.2 –0.1 0 0.1 0.2
0 COR COR COR COR COR COR COR COR
λ
ρ
0.3 0.4 0.5 0.6 0.7 0.8 0.9 0.99
–0.1 COR COR COR COR COR ML ML PR2
0 COR COR COR COR ML ML ML PR2
0.1 COR COR COR COR ML ML ML PR2
0.2 COR COR COR COR COR ML ML PR2
0.3 COR COR COR COR COR COR COR PR2

best except when 0.1    0.3 and   0.7 . At these from Table 3 when n = 20, the COR estimator is gener-
instances, the PC2 estimator is best when   1 and at ally best except when   0.3 and   0.3 . At these
other instances, the best estimator is frequently ML or instances, the PC2 estimator is best when
COR. 0.1    0.2 and   1 . At other instances, the ML
When n = 20, 30, 50 and 100, the results according to or COR is best.
Figures 3, 4, 5 and 6 are not too different. However, When n = 30 from Table 4, COR estimator is gener-
λ
ρ
–0.49 –0.4 –0.3 –0.2 –0.1 0 0.1 0.2
0 COR COR COR COR COR COR COR ML
λ
ρ
0.3 0.4 0.5 0.6 0.7 0.8 0.9 0.99
–0.3 COR COR COR COR COR COR ML ML
–0.2 COR COR ML ML ML ML ML ML
–0.1 ML ML ML ML ML ML ML PR2
0 ML ML ML ML ML ML ML PR2
0.2 COR COR ML ML ML ML ML PR2
0.3 COR COR COR COR COR ML ML ML

λ
ρ
–0.49 –0.4 –0.3 –0.2 –0.1 0 0.1 0.2
–0.2 COR COR COR COR ML ML ML ML
–0.1 ML ML ML ML ML ML ML ML
0 ML ML ML ML ML ML ML ML
0.1 ML ML ML ML ML ML ML ML
0.2 COR COR COR COR ML ML ML ML
λ
ρ
0.3 0.4 0.5 0.6 0.7 0.8 0.9 0.99
–0.3 COR ML ML ML ML ML ML ML
–0.2 ML ML ML ML ML ML ML ML
–0.1 ML ML ML ML ML ML ML PR2
0 ML ML ML ML ML ML ML PR2
0.2 ML ML ML ML ML ML ML ML
0.3 COR ML ML ML ML ML ML ML
0.4 COR COR COR COR COR COR ML ML
ally best except when   0.3 . At these instances, the sparsely COR.
PC2 estimator is best when   0.1 and   1 . At From Table 5 when n = 50, COR estimator is gener-
other instances, the best estimator is frequently ML and ally best except when 0    0.1 and   1 . At these

λ
ρ
–0.49 –0.4 –0.3 –0.2 –0.1 0 0.1 0.2
λ
ρ
0.3 0.4 0.5 0.6 0.7 0.8 0.9 0.99
0 COR COR COR COR COR COR COR PR2
0.1 COR COR COR COR COR COR COR PR2

instances, the PC2 estimator is best. When n = 100 from tion of the assumptions of fixed regressors, independent
Table 6, COR estimator is generally best. regressors and error terms. The paper has not only gener-
ally revealed how the performances of these estimators
4. Conclusion are affected by multicollinearity, autocorrelation and
The performances COR, ML, OLS and PCs estimators in sample sizes but has also specifically identified the best
prediction have been critically examined under the viola- estimator for prediction purpose. The COR and ML are
Table 6. The best estimator for prediction at different level of multicolli nearity and autocorrelation when n = 100.
λ
ρ
–0.49 –0.4 –0.3 –0.2 –0.1 0 0.1 0.2
λ
ρ
0.3 0.4 0.5 0.6 0.7 0.8 0.9 0.99

generally best for prediction. At low levels of autocorre- [17] D. Cochrane and G. H. Orcutt, “Application of Least
lation, the OLS estimator is either best or competes con- Square to Relationship Containing Autocor-Related Error
Terms,” Journal of American Statistical Association, Vol.
sistently with the best estimator while the PC2 estimator
44, No. 245, 1949, pp. 32-61.
is either best or competes also with the best when multi-
collinearity level is high. [18] S. J. Paris and C. B. Winstein “Trend Estimators and
Serial Correlation,” Unpublished Cowles Commission,
Discussion Paper, Chicago, 1954.
REFERENCES [19] C. Hildreth and J. Y. Lu, “Demand Relationships with
Autocorrelated Disturbances,” Michigan State University,
[1] D. N. Gujarati, “Basic Econometric,” 4th Edition, Tata
East Lansing, 1960.
McGraw-Hill Publishing Company Limited, New Delhi
and New York, 2005. [20] J. Durbin, “Estimation of Parameters in Time Series Re-
gression Models,” Journal of Royal Statistical Society B,
[2] J. Neter and W. Wasserman, “Applied Linear Model,”
Vol. 22, No. 1, 1960, pp. 139-153.
Richard D. Irwin Inc., 1974.
[21] H. Theil, “Principle of Econometrics,” John Willey and
[3] T. B. Formby, R. C. Hill and S. R. Johnson, “Advance
Sons, New York, 1971.
Econometric Methods,” Springer-Verlag, New York, Ber-
lin, Heidelberg, London, Paris and Tokyo, 1984. [22] C. M. Beach and J. S. Mackinnon, “A Maximum Likeli-
hood Procedure Regression with Autocorrelated Errors,”
[4] G. S. Maddala, “Introduction to Econometrics,” 3rd Edi-
Econometrica, Vol. 46, No. 1, 1978, pp. 51-57.
tion, John Willey and Sons Limited, Hoboken, 2002.
[23] D. L. Thornton, “The Appropriate Autocorrelation Trans-
[5] S. Chartterjee, A. S. Hadi and B. Price, “Regression by
formation When Autocorrelation Process Has a Finite
Example,” 3rd Edition, John Wiley and Sons, Hoboken,
Past,” Federal Reserve Bank St. Louis, 1982, pp. 82-102.
2000.
[24] J. S. Chipman, “Efficiency of Least Squares Estimation of
[6] A. E. Hoerl, “Application of Ridge Analysis to Regres-
Linear Trend When Residuals Are Autocorrelated,” Eco-
sion Problems,” Chemical Engineering Progress, Vol. 58,
nometrica, Vol. 47, No. 1, 1979, pp. 115-127.
No. 3, 1962, pp. 54-59.
[25] W. Kramer, “Finite Sample Efficiency of OLS in Linear
[7] A. E. Hoerl and R. W. Kennard “Ridge Regression Bias-
Regression Model with Autocorrelated Errors,” Journal
ed Estimation for Non-Orthogonal Problems,” Techno-
of American Statistical Association, Vol. 75, No. 372,
metrics, Vol. 8, No. 1, 1970, pp. 27-51.
1980, pp. 1005-1054.
[8] W. F. Massy, “Principal Component Regression in Ex-
[26] C. Kleiber, “Finite Sample Efficiency of OLS in Linear
ploratory Statistical Research,” Journal of the American
Regression Model with Long Memory Disturbances,”
Statistical Association, Vol. 60, No. 309, 1965, pp. 234-
Economic Letters, Vol. 72, No. 2, 2001, pp.131-136.
246.
[27] J. O. Iyaniwura and J. C. Nwabueze, “Estimators of Li-
[9] D. W. Marquardt, “Generalized Inverse, Ridge Regres-
near Model with Autocorrelated Error Terms and Trended
sion, Biased Linear Estimation and Non-Linear Estima-
Independent Variable,” Journal of Nigeria Statistical As-
tion,” Technometrics, Vol. 12, No. 3, 1970, pp. 591-612.
sociation, Vol. 17, 2004, pp. 20-28.
[10] M. E. Bock, T. A. Yancey and G. G. Judge, “The Statis-
[28] J. C. Nwabueze, “Performances of Estimators of Linear
tical Consequences of Preliminary Test Estimators in Re-
Model with Auto-Correlated Error Terms When Indepen-
gression,” Journal of the American Statistical Associa-
dent Variable Is Normal,” Journal of Nigerian Associa-
tion, Vol. 68, No. 341, 1973, pp. 109-116.
tion of Mathematical Physics, 2005, Vol. 9, pp. 379-384.
[11] T. Naes and H. Marten, “Principal Component Regres-
sion in NIR Analysis: View Points, Background Details
Model with Auto-Correlated Error Terms with Expo-
Selection of Components,” Journal of Chemometrics, Vol.
nential Independent Variable,” Journal of Nigerian Asso-
2, No. 2, 1988, pp. 155-167.
ciation of Mathematical Physics, Vol. 9, 2005, pp. 385-
[12] I. S. Helland “On the Structure of Partial Least Squares 388.
Regression,” Communication Is Statistics, Simulations
and Computations, Vol. 17, No. 2, 1988, pp. 581-607.
Model with Auto-Correlated Error Terms When the Inde-
[13] I. S. Helland “Partial Least Squares Regression and Sta- pendent Variable Is Autoregressive,” Global Journal of
tistical Methods,” Scandinavian Journal of Statistics, Vol. Pure and Applied Sciences, Vol. 11, 2005, pp. 131-135.
17, No. 2, 1990, pp. 97-114.
[31] K. Ayinde and R. A. Ipinyomi, “A Comparative Study of
[14] A. Phatak and S. D. Jony, “The Geometry of Partial Least the OLS and Some GLS Estimators When Normally Dis-
Squares,” Journal of Chemometrics, Vol. 11, No. 4, 1997, tributed Regressors Are Stochastic,” Trend in Applied
pp. 311-338. Sciences Research, Vol. 2, No. 4, 2007, pp. 354-359.
[15] A. C. Aitken, “On Least Square and Linear Combinations doi:10.3923/tasr.2007.354.359
of Observations,” Proceedings of Royal Statistical Society, [32] P. Rao and Z. Griliches, “Small Sample Properties of
Edinburgh, Vol. 55, 1935, pp. 42-48. Several Two-Stage Regression Methods in the Context of
[16] J. Johnston, “Econometric Methods,” 3rd Edition, Mc- Autocorrelation Error,” Journal of American Statistical
Graw Hill, New York, 1984. Association, Vol. 64, 1969, pp. 251-272.

[33] K. Ayinde and J. O. Iyaniwura, “A Comparative Study Error Terms When Stochastic Regressors Are Normally
of the Performances of Some Estimators of Linear Model Distributed,” Journal of Modern Applied Statistical
with Fixed and Stochastic Regressors,” Global Journal of Methods, Vol. 7 No. 1, 2008, pp. 246-252.
Pure and Applied Sciences, Vo.14, No. 3, 2008, pp. 363- [38] K. Ayinde, “Performances of Some Estimators of Linear
369. doi:10.4314/gjpas.v14i3.16821
Model When Stochastic Regressors are Correlated with
[34] K. Ayinde and B. A. Oyejola, “A Comparative Study of Autocorrelated Error Terms,” European Journal of Scien-
Performances of OLS and Some GLS Estimators When tific Research, Vol. 20 No. 3, 2008, pp. 558-571.
Stochastic Regressors Are Correlated with Error Terms,”
[39] K. Ayinde, “Equations to Generate Normal Variates with
Research Journal of Applied Sciences, Vol. 2, No. 3,
Desired Intercorrelation Matrix,” International Journal of
2007, pp. 215-220.
Statistics and System, Vol. 2, No. 2, 2007, pp. 99-111.
[35] K. Ayinde, “A Comparative Study of the Perfomances of
the OLS and Some GLS Estimators When Stochastic Re- [40] K. Ayinde and O. S. Adegboye, “Equations for Generat-
gressors Are both Collinear and Correlated with Error ing Normally Distributed Random Variables with Speci-
Terms,” Journal of Mathematics and Statistics, Vol. 3, fied Intercorrelation,” Journal of Mathematical Sciences,
No. 4, 2007, pp. 196-200. Vol. 21, No. 2, 2010, pp. 83-203.
[36] K. Ayinde and J. O. Olaomi, “Performances of Some [41] TSP, “Users Guide and Reference Manual,” Time Series
Estimators of Linear Model with Autocorrelated Error Processor, New York, 2005.
Terms when Regressors are Normally Distributed,” In- [42] E. O. Apata, “Estimators of Linear Regression Model
ternational Journal of Natural and Applied Sciences, Vol. with Autocorrelated Error term and Correlated Stochastic
3, No. 1, 2007, pp. 22-28. Normal Regressors,” Unpublished Master of Science
[37] K. Ayinde and J. O. Olaomi, “A Study of Robustness of Thesis, University of Ibadan, Ibadan, 2011.
Some Estimators of Linear Model with Autocorrelated

Ojs20120500012 14277069

Încărcat de

Informații document

Titlu original

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

Ojs20120500012 14277069

Încărcat de

Drepturi de autor:

Formate disponibile

Open Journal of Statistics, 2012, 2, 534-546

http://dx.doi.org/10.4236/ojs.2012.25069 Published Online December 2012 (http://www.SciRP.org/journal/ojs)

Estimators of Linear Regression Model and Prediction

Keywords: Prediction; Estimators; Linear Regression Model; Autocorrelated Error Terms;

1. Introduction is known to possess some ideal or optimum properties of

Copyright © 2012 SciRes. OJS

Copyright © 2012 SciRes. OJS

  1 0  0 0 Iyaniwura and Nwabueze [27], Nwabueze [28-30], Ay-

Copyright © 2012 SciRes. OJS

‐0.49 ‐0.1 0.3 0.7

Copyright © 2012 SciRes. OJS

‐0.49 ‐0.1 0.3 0.7

‐0.49 ‐0.1 0.3 0.7

Copyright © 2012 SciRes. OJS

Copyright © 2012 SciRes. OJS

Copyright © 2012 SciRes. OJS

Copyright © 2012 SciRes. OJS

Copyright © 2012 SciRes. OJS

Copyright © 2012 SciRes. OJS

Copyright © 2012 SciRes. OJS

Copyright © 2012 SciRes. OJS

Copyright © 2012 SciRes. OJS

S-ar putea să vă placă și