Marynisa R. Flores Dr.

Antonio Dacanay
MAMATHED-SST QUANTITATIVE RESEARCH

Balagtas, Marilyn U. (2005). Power of ANCOVA vs Change-Score Model in a Quasi-

Experimental Design in Education

Abstract
This study investigated the relative statistical power of Analysis of Covariance
(ANCOVA) and Change-Score Model (CSM) in determining the effects of the treatment variable
on the dependent variable in experimental studies using the nonequivalent pretest-posttest
controlgroup design. In particular, the study compared the effect sizes (ES) computed using
ANCOVA and that using CSM when different values are assigned to alpha, beta and pretest
reliability coefficients, while other variables are held constant. It also examined the relationship
between ES calculated using (1) ANCOVA (2) CSM and research methodological characteristics
such as group sample size, pretest variance, posttest variance, pretest-posttest correlation
coefficient, selection bias index, and pretest difference between groups. Furthermore, it
examined if there are differences between the ES values and their corresponding interpretations
when computed using ANCOVA or CSM. Finally, it explored which among the research
methodological characteristics account for the equal and the different ES values computed using
ANCOVA and CSM.
The study employed a descriptive-comparative research method in which 30 quasi-
experimental studies using nonequivalent pretest-posttest control group design (28 theses, 2
dissertations) conducted from 1992 to 2003 were described in terms of their substantive and
research methodological characteristics. The primary data of these studies were reanalyzed using
ANCOVA and CSM to investigate the degree to which the experimental and the control groups
differed in their cognitive learning after an intervention has been given. Meta-analysis was
performed in which the ES of the 30 studies were aggregated to determine the statistical
procedure that is more powerful in detecting the effect of the treatment variable.
The research was conducted in three phases: data collection, data preparation and data
reanalysis. The study revealed the following findings. First, the mean ES values for both
ANCOVA and CSM are higher when alpha and beta are set at a lower level but the ES valued
for CSM are greater than those for ANCOVA regardless of the alpha and beta level. Second, the
mean ES value for the ANCOVA whose pretest reliability has been assumed perfect is lower
than those for CSM even if the estimate of pretest reliability for CSM has been set as low as 0.7.
Third, the pretest-posttest correlation coefficient is the only predictor of the ES values for
ANCOVA and for CSM. Fourth, the mean ES value for CSM is significantly higher than that for
ANCOVA but the ES values for ANCOVA are more homogenous than those for CSM.
However, the proportion of significant ES values computed using ANCOVA is significantly
higher than that for CSM because ANCOVA assumes that the pretest reliability coefficient of the
studies in the sample is perfect (r = 1.0), which CSM does not assume. Finally, the research
methodological characteristics that could explain the equal ES values for ANCOVA and CSM
are selection bias index and the difference between the pretest scores of the control and the
experimental groups while those that could discriminate significant from non-significant ES
values for ANCOVA and for CSM are the estimate of the pretest reliability coefficient, pretest-
posttest correlation coefficient, and group sample size.