Research Analysis & Critique

Sarah Fitzpatrick

ETEC 500

University of British Columbia

PURPOSE

In the paper “The effectiveness of digital storytelling in the classrooms: a comprehensive

study” researchers Najat Smeda, Eva Dakich and Nalin Sharda investigate the effect that digital

storytelling (DS) has on student engagement, achievement, and motivation. The purpose of their

research project was to investigate digital storytelling through a pedagogical lens by looking at

how it could impact learning outcomes and engagement by asking the question “How can digital

storytelling enhance the student engagement and provide better educational outcomes for

learners?” (pg. 4). They also had a secondary purpose which was to research what teachers and

students’ perspectives were on (DS). As identified by the authors, our education system is

struggling with student engagement. Through this research project they sought to confirm what

much of the literature indicates about DS; that it is a tool that teachers can use to increase

motivation in learners at the primary and secondary levels. Smeda et al. (2014) discuss previous

research that has found that technology integration can enhance students’ engagement in the

classroom but while the technology needed for DS is available it is not being utilized, and DS

has not yet been “recognized as a valuable tool for developing students’ learning skills and

achieving 21st century learning outcomes” (pg. 4).

SIGNIFICANT RESEARCH

Smeda et al. (2014) look at various research to build their argument for DS as a

pedagogical tool, the most significant being that of Pitler (2006), Armstrong (2003) and Neal

(2001). They summarize the research from Pitler (2006) who found that technology integration

in the classroom can build “motivation to learn, encourages collaborative learning and develops

critical thinking and problem-solving strategies.” (pg. 3). To build on these findings Smeda et al.

(2014) use Armstrong (2003) and Neal’s (2001) research. Armstrong (2003) confirms that DS

2
can help students build and develop their creativity as well as problem-solving skills, while Neal

(2001) asserts that DS is a valuable pedagogical tool. Combining these two findings, gave Smeda

et al. (2014) a foundation for their research.

RESEARCH DESIGN

To collect their data Smeda et al. (2014) used a mixed-methods approach by collecting

qualitative and quantitative data to evaluate “if and how digital storytelling enhances teaching

and learning outcomes” (pg. 4). The research in this study is problem-based. As noted, Smeda et

al. (2014) state that “the education system still face[s] many challenges” (pg. 1) one being

student engagement. Through this research, they wished to prove that DS has the power to act as

both an educational tool to motivate learners and increase their technological skills while helping

teachers meet curriculum outcomes. The primary research design was non-intervention as they

did not change or alter any of the participants’ conditions. The participants were not compared to

a control group, they were solely looking for a relationship between student engagement and

motivation (two significant constructs) when implementing digital storytelling as a pedagogical

tool.

VARIABLES

The independent variable in this study was the assignment of creating a digital story in

the classroom. The dependent variable was the students’ success with the educational outcomes.

In addition, there were the attribute variables of student engagement and motivation with levels

of low, moderate and high. Three extraneous variables, age, skill set, and teaching styles were

mentioned but not measured. Also, other extraneous variables such as class size, the time allotted

for completion, and gender may have been correlated with the results but no mention was given.

3
SAMPLE

The researchers used five cases in this research project. These cases included five

different classrooms and teachers from different age groups and subjects. They observed and

collected data from Years 3 and 4 together, 7, 9 and 11, as well as from an English as a Second

Language classroom from an Australian school. The subjects taught in these various classes

varied as did the sample size.

Year Level Subject Sample Size

3/4 Library 8

7 Arts 92

9 Sciences 29

11 Improve personal skills 17

ESL English 4

RELIABILITY & VALIDITY

Smeda et al. (2014) state that “In order to enhance the reliability and validity of the

research, both qualitative and quantitative methods of data collection and analysis were used”

(pg. 7), however, the researchers in this study did not address or present any information on

validity. If the researchers were going to do so they would report on two kinds; content validity

when measuring student success with educational outcomes, and construct validity when

measuring student engagement and motivation. After reviewing students scored on their digital

story, each of the five cases had a mean average above 65 percent. Although the results are

favorable, the reliability of these results is low because no elements of control were

4
implemented, no follow up study was conducted, and the results were not compared to another

like study in any way.

DATA COLLECTION

For this study Smeda et al. (2014) used a mixed-method approach. They used a case

study design which consisted of using observations, interviews and field notes to collect their

qualitative data and a timed observation form and scoring rubric to collect quantitative data. The

scoring rubric for students’ digital story was scored by the classroom teacher to assess the quality

of the story by measuring their achievement on the educational outcomes. In addition, students'

level of engagement was also measured using a scoring rubric. Following this step, researchers

interviewed teachers to confirm whether or not curriculum outcomes were achieved. A case

report was prepared for each of the five grade levels with a cross-matrix for each of the research

questions.

MAJOR FINDINGS

According to Smeda et al. (2014) all their major findings are in accordance with the

current literature. When answering their question of “How can digital storytelling enhance the

student engagement and provide better educational outcomes for learners?” (pg. 4) they were

able to find many positive relationships. They concluded through this research that although it

fluctuated throughout the process, overall, the “levels of student engagement fluctuate between

moderate and high.” (pg. 12). They also concluded that DS increased collaboration between

students and helped to increase students’ technological skills as well as enhance students

learning skills other academic areas such as reading, writing, and spelling.

CRITIQUE

Smeda et al. (2014) collected a range of data to help confirm that DS has many positive

5
effects when used in the classroom. To help strengthen their research they used a mix-methods

approach to confirm much of what the literature has to say about using DS in the classroom as a

pedagogical tool to help motivate and keep students engaged, as well as deliver a creative way to

meet educational outcomes. This being said, there are gaps and weaknesses in their research.

Firstly, no control features were implemented as the participants were not compared to

any other groups or data. Although final scores and final interviews from the five teachers

confirmed that student engagement and achievement of educational outcomes was higher, it

would increase reliability if they had been compared to students who did not partake in DS.

Because no comparison was given it is impossible to say whether or the same results would have

been yielded using a traditional way of teaching. Smeda et al. (2014) also make no mention of

how these classrooms were selected. If not random, it is possible that they were chosen because

teachers in these classrooms already integrate technology on a regular basis. If this were the case

then they would not have needed to teach them new skills which could have increased the final

grades.

Another weakness of this study was the sample. Although this study used various grade

levels to prove that DS can have a positive effect no matter what age, they used a wide range of

sample sizes for their different cases that are not ideal for research. The Art case, for instance,

had 92 students while the Grade 11 group only had 17. As Suter (2014) mentions, if a sample

size is too small can have an effect on finding a true relationship that exists in that particular

population. Therefore, although the findings within this population were positive the small

sample sizes decreases the reliability of the results.

Another gap in this paper is that Smeda et al. (2014) make no note of research that

suggests an alternative; using DS has no effect, or a negative effect on transforming learning.

6
They also do not offer what Sutter (2014) describes as alternative explanations. For instance,

there is a possibility these students performed well because they were excited to use a new form

of technology and/or to work together with their peers. Therefore, if they were to repeat a similar

assignment in the following grade level they may not be as engaged because they would have

had previous experience with DS.

Finally, another weakness is that the researchers apply generalisation by promoting DS

without considering the extraneous variables such as teaching approach, class size and time

given to students to complete their assignment. As Suter (2014) explains, one program or idea

may work in one classroom but have a completely different effect in another because of such

variables.

CONCLUSION

While Smeda et al. (2014) found many positive relationship between DS and student

engagement there are weaknesses in their research. Follow up research studies would need to be

done in order to yield a more reliable conclusion of the correlated relationship between the two.

Implementing a longitudinal study to re-measure student engagement with DS would help

increase reliability and validity, as well as a new experimental research study where the same

scoring rubric is used.

7
 Work Cited

Armstrong, S., The power of storytelling in education, in Snapshots, ed. by S Armstrong.

Snapshots! Educational insights from the Thornburg Centre, 2003, pp. 11–20


Neal, L., Storytelling at a distance. eLearn Mag 5, 4 (2001)


Pitler, H., Viewing technology through three lenses. Principal-Arlington 85(5), 38–42 (2006)


Smeda, N., Dakich, E., & Sharda, N. (2014). The effectiveness of digital storytelling in the
classrooms: A comprehensive study. Smart Learning Environments, 1(1), 1-21.
doi:10.1186/s40561-014-0006-3

Suter, W. N., & SAGE Research Methods Core. (2012). Introduction to educational research: A
critical thinking approach (2nd ed.). Thousand Oaks, Calif: SAGE.