Psy 633 Multiple regression: Hierarchical versus Step-wise Notes adapted from http://faculty chass ncsu edu/garson/P!

"6#/regress htm Stepwise multiple regression$ also called statistical regression$ is a way of computing regression in stages %n stage one$ the independent &est correlated with the dependent is included in the e'uation %n the second stage$ the ne(t remaining independent with the highest partial correlation with the dependent$ controlling for the first independent$ is entered )his process is repeated$ at each stage partialling for previously-entered independents$ until the addition of a remaining independent does not increase *+ &y a significant amount ,or until all varia&les are entered$ of course- !lternatively$ the process can wor. &ac.ward$ starting with all varia&les and eliminating independents one at a time until the elimination of one ma.es a significant difference in *-s'uared %n SPSS$ select !naly/e$ *egression$ 0inear1 set the Method: &o( to Stepwise Stepwise regression and theory Stepwise regression is used in the exploratory phase of research or for purposes of pure prediction, not theory testing %n the theory testing stage the researcher should &ase selection of the varia&les and their order on theory$ not on a computer algorithm Menard ,233#: #4- writes$ 5there appears to &e general agreement that the use of computer-controlled stepwise procedures to select varia&les is inappropriate for theory testing &ecause it capitali/es on random variations in the data and produces results that tend to &e idiosyncratic and difficult to replicate in any sample other than the sample in which they were originally o&tained 5 0i.ewise$ the nominal 6# significance level used at each step in stepwise regression is su&7ect to inflation$ such that the real significance level &y the last step may &e much worse$ even &elow #6$ dramatically increasing the chances of )ype % errors See 8raper$ N * $ 9uttman$ % : 0apc/a.$ 0 ,23"3- ;or this reason$ ;o( ,2332: 2<- strongly recommends any stepwise model &e su&7ected to cross-validation Other problems of stepwise regression =y &rute force fitting of regression models to the current data$ stepwise methods can overfit the data$ ma.ing generali/ation across data sets unrelia&le ! corollary is that stepwise methods can yield *+ estimates which are su&stantially too high$ significance tests which are too lenient ,allow )ype 2 error-$ and confidence intervals that are too narrow Hierarchical multiple regression ,not to &e confused with hierarchical linear models- is similar to stepwise regression, but the researcher, not the computer, determines the order of entry of the variables ;-tests are used to compute the significance of each added varia&le ,or set of varia&les- to the e(planation reflected in *-s'uare )his hierarchical procedure is an alternative to comparing &etas for purposes of assessing the importance of the independents %n more comple( forms of hierarchical regression$ the model may involve a series of intermediate varia&les which are dependents with respect to some other independents$ &ut are themselves independents with respect to the ultimate dependent Hierarchical multiple regression may then involve a series of regressions for each intermediate as well as for the ultimate dependent

SPSS data analysis ;or hierarchical multiple regression$ in SPSS first specify the dependent varia&le1 then enter the first independent varia&le or set of varia&les in the independent varia&les &o(1 clic. on 5Ne(t5 to clear the %> &o( and enter a second varia&le or set of varia&les1 etc ?ne also clic.s on the Statistics &utton and selects 5*-s'uared change 5 Note that the error term will change for each &loc. or step in the hierarchical analysis %f this is not desired$ it can &e avoided &y selecting Statistics$ 9eneral 0inear Model$ 90M-9eneral ;actorial$ then specifying )ype % sums of s'uares )his will yield 90M results analogous to hierarchical regression &ut with the same error term across &loc.s Assumptions Multiple regression shares all the assumptions of correlation: linearity of relationships$ the same level of relationship throughout the range of the independent varia&le ,5homoscedasticity5-$ interval or near-interval data$ a&sence of outliers$ and data whose range is not truncated %n addition$ it is important that the model &eing tested is correctly specified )he e(clusion of important causal varia&les or the inclusion of e(traneous varia&les can change mar.edly the &eta weights and hence the interpretation of the importance of the independent varia&les