REGRESSION ANALYSIS SUMMARY

Business Analytics
XPGDM (2019-20)
Term-III (Nov’19-Jan’20)

Kakali Kanjilal
Professor, OM/QT
IMI, Delhi
 Steps in Regression Analysis
Step1: Data Handling and variable Transformation
- Missing data and outlier identification and adjustments
- Divide the data into TEST and CONTROL or DEVELOPMENT and
VALIDATION (assuming that the sample is large) for a Cross Section (CS) data
OR
- Divide the data into IN-SAMPLE and OUT-OF-SAMPLE (assuming that the
sample is large) for a Time Series (TS) data
- Work on the DEVELOPMENT (in case of CS data) or IN-SAMPLE data (for TS
data)
- Plot the GRAPH of dependent (Y) and independent variable(s) (X) (say)
- Identify if any TRANSFORMATIONS in variables needed to make the
relationship between Y and X linear.
- If yes, TRANSFORM the variable(s).

Step 2: Estimate the Model with OLS

- ESTIMATE a bivariate/multiple linear regression model for the
DEVELOPMENT (in case of CS data) or IN-SAMPLE data (for TS data)
 Steps in Regression Analysis
Step 3: Analyze the Regression Results
Look at the following four points in regression results in the following order
- Statistical Significance of F-statistic
- R2 or adj R2 (in case of multiple regression) is high
- Signs of estimated coefficients/parameters
- Statistical Significance of each coefficient or significance of t-statistic

Step 4: Identify Problems if any in Regression Results

Scenario 1 :
- If F-stat is statistically insignificant, no need to look at regression result
 Steps in Regression Analysis
Step 4: Identify and Fix Problems if any in Regression
Results for ‘Cross Section’(CS) Data
Scenario 2 :
- If F-stat is statistically significant and
- R2 or adj R2 (in case of multiple regression) is high and/or
- Estimated Coefficient signs are not in line with expectations and
- Majority of independent variables are statistically insignificant

Then there could be problems of ‘Multicollinearity’ and/or ‘Heteroscedasticity’

In that case, proceed as shown below
- Confirm ‘Multicollinearity’ problem with Variance Inflation Factor (VIF) or
tolerance limit (in case of multiple regression)
- Confirm ‘Heteroscedasticity’ with ‘WHITE’s test
- Drop/Combine the variable(s) causing the ‘Multicollinearity’ problem and re-
check VIF
- Address ‘Heteroscedasticity’ problem by Newey and West adjusted standard
errors. Re-estimate regression results.
- Re-check ‘Heteroscedasticity’ with WHITE’s test
- Go to Step 5
 Steps in Regression Analysis
Step 4: Identify and Confirm the Problems with Statistical
Tests for ‘Time Series (TS) Data’
Scenario 3 :
- If F-stat is statistically significant and
- R2 or adj R2 (in case of multiple regression) is high, and/or
- Estimated Coefficient signs are not in line with expectations and
- Majority of independent variables are statistically insignificant
Then there could be problems of ‘Multicollinearity’ and/or ‘Autocorrelation’
In that case, proceed as follows
- Confirm ‘Multicollinearity’ problem with Variance Inflation Factor (VIF) or
tolerance limit (in case of multiple regression)
- Confirm ‘Autocorrelation’ with ‘Serial Correlation Lagrange Multiplier (LM)
test
- Drop/Combine the variable(s) causing the ‘Multicollinearity’ problem and re-
check VIF
- Address ‘Autocorrelation’ problem by introducing either ‘one-period lagged’
residual or dependent variable (Y), an AR(1) term in residual or dependent variable
and Re-estimate regression results.
- Re-check ‘Autocorrelation’ test
- Go to Step 5.
 Steps in Regression Analysis
Step 5.1: Interpret the Regression Results for Development
(for CS)/In-sample data (for TS)
- Interpret the estimated coefficients in line with the objectives of the study
- Evaluate Goodness of fit
- Forecast/Project the future behaviour of dependent variable ,wherever
applicable
- Evaluate forecast accuracy
Step 5.2: Check the Regression Results for Validation Data (for CS)/
Out-of-sample data (for TS)
- Score the model for Validation (for CS)/ Out-of-Sample (for TS)
- Evaluate Goodness of fit /accuracy
- Forecast/Project the future behaviour of dependent variable wherever
applicable
- Evaluate forecast accuracy
Step 6: Recommendations
- Recommend Policy suggestions and action items
- Identify Limitations if any.