One-Variable Investigation
Mean = 0 Look for residiuals to be centered
If points lie within 95% confidence on
probability plot then normality holds.
H0 : The Transformed Data Are Normal
HA: They are not
Standard Error
estimates
3.
1. Point estimation: Estimating the unknown
parameter by a single number calculated from
the sample.
2. Interval estimation: Estimating the unknown spread.
parameter with a set of plausible values, and
attaching a level measuring your confidence
that the set (i.e., the interval) covers the
parameter.
3. Significance Testing about hypotheses:
Using the data to assess evidence in favor of or
against some claim about the population
parameter.
95% Confidence level determined by
Point-estimate +/- 2*standard error
If p-value >/= significance level, the data dont b1/Seslope
sufficiently contradict the null.
If p-value < significance level, reject the null
hypothesis
For a 95% Confidence level, significance level
= 0.05
Simple Linear Regression
Errors are assumed to be: 1. Independent
2. Mean = 0, 3. Constant standard deviation, for Option 2 Higher Order Models
all X, and 4. Normally Distributed.
1. linear inspect the scatter-plot of the data
2. Are independent look for the residuals to
be patternless scattered on a residual plot.
around zero. Multiple Linear Regression
4. Have constant standard deviation look for the
residuals to show approximately constant
5 Normally distributed -- inspect a probability
plot of the the residuals; look for the points to
be close to the line; supplement with 95%
confidence limits for Normality. Identify p-
value.
Can find b1 = R(SY/Sx) where R = sample
correlation, Sy,x = Sample st.dev of Y or X.
Can get b0 = Y - b1 X where they are the means
of Y and X.
t = (estimate null value)/standard error =
Non-Linear Relationships
Conduct the F-Test
Option 1 Transformation
H0 : B1 = B2 =B3 = = Bp = 0
HA:: At least one of these is not zero.
If the F-test is non-significant then the model is
useless.
If the F-test yields a significant result, then at
least one variable is important so then proceed
to conduct tests of individual effects.
Conduct T-Tests for individual effects
For each explanatory variable X, decide if there
is sufficient evidence that it is significant, in the
presence of all the other explanatory variables.
Ho : B1 = 0 H0 : B2 = 0 . ETC
Ha: B1 ! = 0 H1: B2 != 0 .
Partial R2
Y = B0 + B1 X + B2 X2 + err (Quadratic) (the R2 between Y and both predictors) (the R2
between Y and just X1) / (100% - (the R2
Y = B0 + B1 X + B2 X2 + B3 X3+ err (Cubic) between Y and just X1)
Transformation should be used wherever it is
possible, instead of higher order models.
 The Danger of Multicollinearity
In a multiple linear regression multicolinearity
is when the explanatory variables are perfectly Adjusted R2 used for comparing models to
or strongly linearly associated with each other. indicate which is better.
Ideal we want to have weakly correlated
predictors.
Weakly correlated if similar coefficients
between single variable analysis and multi
variable analysis and standard errors are the
same or smaller.
Total multicollinearity means that one
explanatory variable depends entirely on
another. This will give us an error, because
wed divide by zero. model.
Strong multicollinearity, but not total, will give
misleading results. Determined by wildly
different coefficients and standard errors. Also
look at the high correlations. More precise
method is VIF.
VIF
VIF is the reciprocal of a measure called
tolerance associated with each explanatory
variable.
Tolerance obtained by:
1. Produce a multiple linear regression with Xj
as the response, versus all the other explanatory
variables.
Xj = Bo + B1X1 + B2X2 +.
2. Obtain the R2 from this regression.
3. The tolerance = 1 R2j
4. VIF = 1/tolerance
If VIF > 2.5, then worry about dangerous
multicollinearity.
If we suspect multicollinearity, consider
dropping some of the highly correlated
variables from the model.
Adjusted R2 for comparing best subsets. It 1. Its the difference in slopes, it measures how
adjusts for number of variables. the quantitative effect depends on the group
value in the following sense: F statistic
For every one unit increase in the F = (SSG/DFG)/(SSE/DFE) = MSG/MSE
quantitative variable X1 , the change in Y is B3
greater/less in the category coded as 1 relative R2= sum of squares for groups/sum of
Non-adjusted measures the predictive strength to the category coded as 0 squares total = SSG/SSTotal = SSG/
of models. (SSG+SSE)
2. B3 captures how the vertical difference
Check for multicollinearity problems in between the lines changes, which measures R2 is the proportion of variability in Y
2
addition to adj R how the group effect depends on the attributable to (or predicted from) the
quantitative value, in the following sense. explanatory variable(s) based on the model in
Including Categorical Variables in the For a particular value x 1 of the quantitative question.
Regression Model variable, the difference in Y for the group coded
as 1 compared to the group coded as 0 is
So far used quantitative variables. Often want more/less than it would be without the
to include categorical variables in the prediction interaction term, by an amount equal to x1 times
B3 (on average.)
Can add a dummy variable depending on
categorical number, 0 or 1.
Beta coefficient shows how much higher or
lower on average the response is for the
category coded as 1 compared to the response
for the category coded as 0.
Can add multiple levels. Must add two dummy
variables, each of which is binary. The group
represented by (0,0) is the baseline group.
Modeling Interaction
One Way Anova
Regression models model Q Q, Anova Tukey Pairwise Comparisons
models C Q .
If an interval for the difference doesnt contain
H 0 : 1 = 2 = = n zero, then we conclude that pair is different
from each-other.
HA : The s are not the same.
Two-Way ANOVA
If the interaction term is significant, then both
explanatory variables must be kept in the model
(regardless of the p-values of the individual
effect tests for the separate variables.
B3 (the coefficient of the interaction term) has
two equivalent interpretations:

Repeated Measure Anova
For one-way Anova we measure k independent
groups separate groups of subjects, each subject P = 1/ (1 + exp [- (B0 + B1 x)]
is measured once.
For repeated measures Anova, we measure one
group of subjects k times, non-independent
groups.
Two-Factor Repeated-Measures (within
subjects) and Two Factor Mixed (with-
between subjects)
Recall that repeated measures is where the
same subjects get measured k times (I.e where
the same subjects are exposed to all
treatments).
Check to see on what variable there is a
repeated test.
Ancova
When repeated design is not feasible or it
wouldnt be desirable there is an alternative
analysis. This is ANCOVA since the variables
are covariates. We include other quantitative
variables that are correlated with, but not part
of the main experimental manipulation.
ANCOVAs usefulness depends on good
correlation between the covariate and the
response, otherwise nothing is gained by it.
For studies with human subjects, typical
covariates are age, socioeconomic status,
aptitude, or pre-study attitude.
Equality of the regression slope across the
conditions.
The different treatments have no effect on the
covariate.
Logistic Regression
Model appropriate fro a categorical response Y.
Linear Regressions Q Q , ANOVA C Q
Logistic Q C
Odds = P/(1-P) Nominal Logistic Regression (Y= more than
2 categories, unordered)
P = Odds/(1+Odds)
log(p/1-p) = B0 + B1 X1 + B2X2 + + BkXk
We have Y = cat 1 , with prob P1, cat 2 , etc
ln(p/1-p) = B0 + B1 X
sum of all p = 1
B1 governs steepness
Reference category Pc
Goodness-ofFit Tests
We define P1/Pc = odds favoring category 1
H0 : The model fits the data over category c
HA: The model is not a good fit to the data.
exp(B1) is the odds ratio, (odds favoring Y=1 Ordinal logistic regression
when X = x+1 / odds favoring Y=1 when X=x)
Is used in cases when the response variable Y is
H0 : B1 = 0 a categorical variable with more than two levels
which have a natural ordering.
HA:: B1 != 0
For Binary Logistic there is no equivalent
measure of associoation which has a nice
mathematical properties of R2
Percent concordant pairs.
Concordant
if X1 > X2 , Y1 > Y2
Binary Multiple Logistic Regression
log(p/1-p) = B0 + B1 X1 + B2X2 + + BkXk
P = 1/(1+ exp[-(B0 + B1 X1 + B2X2 + + BkXk)]