See

discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/263739381

Improving biotech education through gamified

laboratory simulations

Article in Nature Biotechnology · July 2014

DOI: 10.1038/nbt.2955 · Source: PubMed

CITATIONS READS

14 285

7 authors, including:

Guido Makransky Jakob Wandall

University of Southern Denmark Aarhus University
30 PUBLICATIONS 134 CITATIONS 5 PUBLICATIONS 20 CITATIONS

SEE PROFILE SEE PROFILE

Mette Voldby Larsen Hanne Jarmer

113 PUBLICATIONS 1,457 CITATIONS Technical University of Denmark
67 PUBLICATIONS 2,117 CITATIONS
SEE PROFILE

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

EvoTAR (Evolution and Transfer of Antibiotic Resistance; http://www.evotar.eu/) View project

Detailing stromal heterogeneity in the human breast View project

All content following this page was uploaded by Mette Voldby Larsen on 04 August 2014.

The user has requested enhancement of the downloaded file. All in-text references underlined in blue are added to the original document
and are linked to publications on ResearchGate, letting you access and read them immediately.
 CAREERS AND RECRUITMENT
Improving biotech education through gamified
laboratory simulations
Mads T Bonde, Guido Makransky, Jakob Wandall, Mette V Larsen, Mikkel Morsing, Hanne Jarmer &
Morten O A Sommer
Gamified laboratory simulations motivate students and improve learning outcomes compared with traditional
teaching methods.
© 201 Nature America, Inc. All rights reserved.
A large proportion of high school and college
students indicate that they have little inter-
est in science, and many students graduate with
marginal science competencies1,2. It has been
suggested that this results from an exaggerated
focus on memorizing facts, listening passively
to lectures and performing ‘cookbook’ labora-
tory exercises in science education, rather than
stimulating students’ natural curiosity, and
highlighting the intricate connection between
science and “real world problems”3. Although
several studies have challenged the effective-
ness of traditional teaching methods4–8, these
methods continue to dominate science educa-
tion. This is not only problematic for students
but is a major challenge for the biotech indus-
try, which depends on highly educated gradu-
ates with up-to-date knowledge and skills.
npg
A recent report published by the US
National Research Council regarding the use
of computer games and simulations in edu-
cation analyzed all available studies and con-
Mads T. Bonde & Morten O.A. Sommer
are at the Novo Nordisk Foundation Center
for Biosustainability, Technical University of
Denmark, and the Department of Systems
Biology, Technical University of Denmark;
Mette V. Larsen & Hanne Jarmer are at the
Center for Biological Sequence Analysis,
Department of Systems Biology, Technical
University of Denmark; Guido Makransky is
at the Department of Psychology, University
of Southern Denmark; Jakob Wandall is at
NordicMetrics, Denmark; and Mikkel Morsing
is at the Department of Biology, University
of Copenhagen, Denmark. Hanne Jarmer
and Morten O.A. Sommer have shared
senior authorship.
e-mail: mab@bio.dtu.dk, hanne@cbs.dtu.dk, or
msom@bio.dtu.dk
694
cluded that “simulations and games have great
potential to improve science learning in ele-
mentary, secondary and undergraduate science
classrooms”2. Moreover, the US Department of
Education’s National Education Technology
Plan states, “The challenge for our education
system is to leverage the learning sciences and
modern technology to create engaging, rel-
evant and personalized learning experiences
for all learners that mirror students’ daily lives
and the reality of their futures”9.
Because laboratory experiments can be
expensive, time consuming and occasionally
constrained by safety concerns, laboratory
courses as an adjunct to classroom lectures
are often the first classes to be removed from
a curriculum. This is unfortunate because sev-
eral theoretical science courses benefit from
an experimental counterpart. Particularly
within biotech, new techniques and methods
are constantly enhancing and replacing exist-
ing research practices, and these developments
soon become essential knowledge for biotech
professionals. Nevertheless, the latest equip-
ment and consumables are often prohibitively
expensive, making it almost impossible for
universities and schools to provide students
with access to updated equipment such as next-
generation DNA sequencing machines.
In response to this need, several simulations
have been developed for science education,
most of which focus on symbolic represen-
tations of experiments wherein students
can alter parameters and simulate different
outcomes. De Jong et al.10 recently reviewed
studies comparing physical and simulated
laboratory education and concluded that both
physical and virtual laboratories “can achieve
similar objectives such as exploring the nature
of science, developing teamwork abilities,
VOLUME 32 NUMBER 7 JULY 2014 NATURE BIOTECHNOLOGY
cultivating interest in science, promoting con-
ceptual understanding and developing inquiry
skills.” Although physical laboratories are
required for students to develop practical labo-
ratory skills, virtual laboratories offer several
other advantages, including allowing students
to explore unobservable phenomena, enabling
learners to conduct a number of experiments
in a short period of time and providing adap-
tive guidance11. However, most simulations
are primarily focused on accurately imitating
physical phenomena and not on optimizing
student learning12.
A recent literature review identified only a
few studies that compared traditional class-
room teaching with the use of simulations
in biotech teaching between 2001 and 2010
(refs. 13–15). One study reported an increase
in students’ usage of accurate explanations after
using a bioinformatics simulation15, and oth-
ers reported a significant increase in test scores
using a simulation based on cell theory14.
Similarly, a learning effect was demonstrated
using the simulation MyDNA, a program that
involves a two-dimensional representation
of gel electrophoresis wherein students can
alter voltage and gel concentrations and then
observe the differential speed of DNA frag-
ments16.
Educational games are increasingly being
used for learning biotech. Sadler et al.17
reported the implementation of a three-
dimensional (3D) biotech educational game
(Mission Biotech), wherein gaming features
were highlighted. A high learning outcome,
particularly with lower-level students, was
observed. Research regarding the effective-
ness of games for science education is only
beginning to emerge2, and to our knowledge
no prior research studies performed to assess

the effectiveness of gamified simulations for
enhancing biotech education have included a
scientific design with control groups.
We hypothesized that combining gamifica-
tion elements with simulations may provide an
opportunity for even greater gains in learning
effectiveness and motivation of biotech stu-
dents. We developed and tested an advanced
laboratory simulation platform based on math-
ematical algorithms supporting open-ended
investigations and combined this with gamifi-
cation elements such as an immersive 3D uni-
verse, storytelling, conversations with fictional
characters and a scoring system. We then set
out to assess the effect on learning effectiveness
and motivation to investigate whether gami-
fied laboratory simulations may be an afford-
able opportunity for providing state-of-the-art
training in biotech.
Development of a gamified biotech
© 201 Nature America, Inc. All rights reserved.
laboratory simulation
Ten gamified laboratory simulations have
been developed, two of which were tested: a
crime-scene lab and a genetic engineering lab
(http://www.labster.com/biolabs/) (Fig. 1). In
the crime-scene case, students start by explor-
ing a crime scene that reveals an engaging story
and then proceed to analyze blood samples
from the scene using PCR and gel electro-
phoresis. Through scientific inquiry, students
ultimately provide conclusive evidence to con-
vict the perpetrator. Interactive 3D animations
of microscopic events pause while students
answer questions and identify elements and
processes in animations. If a student convicts
an innocent suspect, he or she experiences con-
sequences of not adhering to scientific meth-
ods through a critical virtual newspaper article
npg
received during the gamified simulation. In the
genetic engineering case, students embark on
a mission to produce medicine and test it on a
virtual mouse through molecular cloning, fer-
mentation and animal experiments.
In developing these simulations, one of our
key priorities was providing a realistic and
immersive laboratory environment and 3D
animations. We chose these because previous
Figure 1 Screenshots from the gamified laboratory simulation tested in this study (Labster, http://www.labster.com/biolabs/) (left, center). Students
used the simulations from regular computers at home (in the case of testing at Stanford University Online High School) or in class (right).
NATURE BIOTECHNOLOGY VOLUME 32 NUMBER 7 JULY 2014
CAREERS AND RECRUITMENT
studies have demonstrated increased learning
effectiveness among students who use highly
realistic animations and because their use is
supported by cognitive theories of multime-
dia learning and picture comprehension18–20.
Simulations were designed for an inquiry-
based approach in which students must deduce
and apply necessary actions by acquiring
knowledge from integrated text and figures.
Inquiry-based methods have been found dif-
ficult to successfully implement in classical
teaching with a single teacher and numerous
students21–23. However, computer-based labo-
ratory simulations may offer a framework for
an effective and feasible solution.
A combination of learning and assessment
methods, as described in recent research24,
was integrated into the simulation through
built-in, multiple-choice test items triggered
by specific actions. In the simulation, students
were asked to select a response until the cor-
rect answer was identified, and they were sub-
sequently presented with an explanation of why
the answer was correct. Immediate feedback
inspires students to reflect on their choices
of specific laboratory procedures, thereby
improving their understanding of the under-
lying theory. Furthermore, to increase students’
understanding of molecular details invisible to
the human eye, 3D animations were integrated
to demonstrate what happens at the molecu-
lar level as a direct consequence of laboratory
actions performed by students.
Motivation and self-efficacy
Recent studies indicate noncognitive skills
such as motivation and conscientiousness as
crucial factors in the efficient development of
cognitive skills in science learning25–27. Many
students have a general perception of science
as being boring and disconnected from the real
world. For instance, approximately 50% of the
students surveyed in physics classes found the
subject “boring” or “very boring”28. We wanted
to investigate whether gamified laboratory
simulations could stimulate a higher degree of
motivation for studying biotech. We set up a
program to assess the genetic engineering lab
in an online AP Biology class at the Stanford
University Online High School (Stanford, CA,
USA). We immediately followed the comple-
tion of the labs by an online questionnaire on
motivation and interest. Forty out of 41 stu-
dents found the laboratory simulation “inter-
esting and relevant subject matter” and 23 out
of 41 found it “more motivating than classroom
or home wet labs.”
We conducted a larger study of the more
advanced crime-scene lab with 149 students
from two biology classes at Archbishop
Williams High School (Braintree, MA, USA)
and in an introductory college-level life science
course at the Technical University of Denmark
(DTU). A high level of motivation was mea-
sured, as 97% of 149 students found it interest-
ing to use the simulation; 86% indicated that
the laboratory simulation was more interest-
ing compared with ordinary exercises; and 97%
felt that the course content was more interest-
ing when working with gamified simulations
(Fig. 2). Furthermore, 89% of users indicated
that they learned something by using the gami-
fied simulation, which indicates a high level of
perceived learning and self-efficacy. Further
studies in which students experience repeated
use of simulations are needed to investigate
how motivation and interest are affected when
students become accustomed to using simula-
tions in learning.
We also conducted a study on 57 students
from four Danish high schools. Forty-four
percent of the students agreed or completely
agreed with the statement, “I consider
pursuing an education within biotechnol-
ogy or other biological subject to a greater
extent, after having used the gamified labora-
tory simulator.” This indicates that gamified
laboratory simulations may provide a pow-
erful case for initiatives to increase students’
engagement and motivation for further studies
and careers within biotech. Large-scale longi-
tudinal studies measuring the actual study and
career choices of students exposed to gamified
laboratory simulations compared with a con-
trol group would be valuable to guide future
decisions.
695
 CAREERS AND RECRUITMENT

It is a good idea to use Labster before skills. By comparing test results of the pre-test,
using a real laboratory mid-test, post-test and retention tests for each
student on the 38 items, learning outcomes of
I would like Labster to be used more in each session could be assessed.
teaching Students’ scores improved by 1.48 standard
deviation (s.d.) units from a mean Z score of
Labster can be a good supplement to <1.37 to 0.11 after the laboratory simulation
regular teaching but only by 0.84 s.d. units from <1.20 to <0.36
after traditional teaching at the mid-test sam-
I learned something by using Labster pling point (Fig. 3). The results demonstrate
that using the laboratory simulation led to
significantly improved learning outcomes
Labster is more motivating than ordinary
exercises (76% higher score) compared with traditional
teaching (t (89) = <4.37, P < 0.0005). Effects
It makes course content more interesting of combining the simulation with traditional
to work with practical examples teaching were assessed with the post-test,
and the measured learning outcomes were
The experience with Labster inspired me to greater than any one of the methods alone
work with laboratory analyses (t (90) = <7.49, P < 0.0005; see Fig. 3b).
Students who used the simulation first had
It was interesting to use Labster a 4.2% greater gain in measured learning out-
© 201 Nature America, Inc. All rights reserved.

comes compared with those who received tra-

Completely agree Mostly agree
Mostly disagree Completely disagree
Figure 2 Survey results from 149 students from Stanford University Online High School, Archbishop
Williams High School and the Technical University of Denmark after using the gamified laboratory
simulation crime-scene lab.
Improved learning outcomes
To assess the learning effectiveness of the
gamified simulation compared with tradi-
tional teaching, we conducted a study test-
ing the crime-scene case in an introductory,
college-level, life science course with 91 stu-
dents at DTU. All participants answered a
pre-test consisting of 40 multiple-choice ques-
tions intended to fit an item response theory
educational measurement model29,30. Students
npg
0 50 100
Students (%)
comprising the same questions. In the second
lesson, students switched conditions: Group
A performed the laboratory simulation, and
Group B received the lecture. After the sec-
ond lesson, all students were administered
the test again as a post-test. Students took the
test for the fourth time 40 days later as a reten-
tion test. Test results were analyzed by fitting
the data to the two parameter logistic (2PL)
item response theory model that describes, in
ditional teaching first; however, this difference
was not statistically significant (comparison
of post-tests, t (89) = <0.383, P = 0.702). This
finding was consistent with those of previous
studies16, indicating flexibility in the order of
methods used and in integration possibili-
ties for simulations. For instance, simulations
could be used as a homework assignment as
a pre-laboratory or post-laboratory activity in
combination with traditional teaching. In fact,
the traditional teaching increased learning
outcomes by only 14% after students had com-
pleted the gamified simulation, which suggests
that traditional teaching was almost redundant
after the gamified simulation. However, the
combination may have the strongest impact,
particularly on long-term retention, because

were then divided into two groups (alphabeti-
cally by first name). In the first lesson, Group
A received a traditional lecture including a
group excercise, and Group B performed the
crime-scene simulation. Following the first les-
son, all students were administered a mid-test
probabilistic terms, the relationship between
an individual’s response to a test item and his or
her standing on the construct being measured
by the test31,32. Of 40 items used on the test, 38
items fitted the 2PL model, indicating that the
test was suitable for measurement of student
practicing new skills only once can lead to
rapid deterioration of acquired competencies11.
To test the durability, we administered a follow-
up test 40 days after the sessions, and learning
was found to be well retained; no significant
difference was observed between scores on the

a b
0.50 **
1.8
0 *
1.5
Z score

–0.50 1.2
Δ Z score

–1.00 0.9

–1.50 0.6
Pre-test Mid-test Post-test Retention
0.3
Gr. A Labster Lecture 40 days
Gr. B Lecture Labster 40 days 0.0
Lecture Labster Lecture & Labster
Figure 3 Measurement of learning outcomes from 91 students. (a) Test outcome of groups A and B receiving the laboratory simulation (Labster) and
lecture including group exercise (Lecture) in the opposite order. (b) Increase in learning outcomes observed after students attended a session with a
lecture including group exercise (Lecture), laboratory simulation (Labster) and both methods combined. *Students t-test, t (89) = <4.37, P < 0.0005;
**t (90) = <7.49, P < 0.0005.

696 VOLUME 32 NUMBER 7 JULY 2014 NATURE BIOTECHNOLOGY


post-test and the follow-up test (t (87) = 0.641,
P = 0.523).
Gamified laboratory simulations as an
integral part of future biotech education?
This study indicates that a gamified labora-
tory simulation can significantly increase
both learning outcomes and motivation levels
when compared with, and particularly when
combined with, traditional teaching. Further
research is urgently needed to investigate
whether our results can be extrapolated to a
general tendency of the effectiveness of gami-
fied simulations. If our results represent a gen-
eral tendency, increased focus in this area could
provide an important opportunity to address
some of the current challenges that science
education is facing and ultimately to enhance
science education.
Currently, simulations and games are used
only sporadically in biotech education, per-
© 201 Nature America, Inc. All rights reserved.
haps because educational institutions still are
more focused on delivering instructions than
producing learning outcomes33. Other sectors
have successfully integrated simulations as a
well-established part of training. For instance,
flight simulators have been successfully used for
decades in training sessions of aspiring pilots,
and combining simulations with real airplane-
flying experience has proven to be more effec-
tive with regard to both time and resources34.
Particularly within biotech education, gamified
simulations can benefit students by provid-
ing simulated access to exercises that would
otherwise require expensive equipment and
hazardous techniques that most educational
institutions are unable to offer. To fully explore
and deploy the potential of gamified simula-
tions in biotech education, policymakers, end
npg
users such as school districts and universities,
researchers and companies must work together
to develop, research and assess new gamified
simulations to reap benefits of modern technol-
ogy for the improvement of science education.
NATURE BIOTECHNOLOGY VOLUME 32 NUMBER 7 JULY 2014
View publication stats
CAREERS AND RECRUITMENT
ACKNOWLEDGMENTS
We would like to acknowledge H.J. Genee,
P. Rugbjerg, J. Serritslev, A. Laustsen, D. Møller,
L. Holst, S. Heilmann, O. Filtenborg, E.B. Hansen,
S. Molin, O.D. Madsen, B. Nauntofte, R. Frandsen,
O. Thastrup, H.H.Wang, P. Falholt, J. Vind,
J.T. Andersen, T. Damhus, B.T. Simonsen,
C.B. Jorgensen, C.V.S. Bruun, K. Spohr, P. Gibson,
J. Keasling, G.M. Church, J. Nielsen and B. Palsson
for fruitful discussions, support and advice. We
would also like to thank the Labster team for co-
development of the gamified laboratory simulations,
H. Bonde, G. Nixon and E. Earls for critical reading
of the manuscript, as well as K. Failor and M. Huang
for collaboration regarding the tests at Stanford
University OHS and Archbishop Williams High
School. We would like to acknowledge the student
organization Biotech Academy at DTU for its role in
disseminating science education and awareness to the
young. This research was funded by the Novo Nordisk
Foundation and the Danish Market Development
Fund, and the development of the laboratory
simulations has been supported by The Danish
Ministry of Science, the Danish Market Development
Fund, The Lundbeck Foundation, The Danish Film
Institute, Novo Nordisk A/S and Novozymes A/S.
COMPETING FINANCIAL INTERESTS
The authors declare competing financial interests:
details are available in the online version of the paper
(doi:10.1038/nbt.2955).
1. Business Higher Education Forum. The STEM Interest
and Proficiency Challenge: Creating the Workforce of
the Future (Washington, DC, 2011).
2. Honey, M.A., et al. Committee on Science Learning:
Computer Games, Simulations, and Education.
(National Research Council, 2011).
3. Gago, J.M. et al. Increasing Human Resources
for Science and Technology in Europe. European
Commission High Level Group report (2004).
4. Lanahan, L., McGrath, D.J., McLaughlin, M.,
Burian-Fitzgeral, M. & Salganik, L. 1–32, Fundamental
Problems in the Measurement of Instructional
Processes: Estimating Reasonable Effect Sizes
and Conceptualizing What is Important to Measure
(American Institutes for Research, Washington, DC,
2005).
5. Alexander, K., Natriello, G. & Pallas, A.M. Am. Sociol.
Rev. 50, 409–420 (1985).
6. Natriello, G., Pallas, A. & Alexander, K. Sociol. Educ.
62, 109–118 (1989).
7. Ralph, J., Keller, D. & Crouse, J. Phi Delta Kappan 76,
144–150 (1994).
8. Keller, D. An Assessment of National Academic
Achievement Growth (University of Delaware, 1995).
9. The US Department of Education’s National Education
Technology Plan. Transforming American Education:
Learning Powered by Technology (2010).
10. De Jong, T., Linn, M.C. & Zacharia, Z.C. Science 340,
305–308 (2013).
11. Zacharia, Z.C., Olympiou, G. & Papaevripidou, M.
J. Res. Sci. Teach. 45, 1021–1035 (2008).
12. Mislevy, R.J. Evidence-centered design for simulation-
based assessment. CRESST Report 800 (National
Center For Research on Evaluation, Standards, and
Student Testing, Los Angeles, 2011).
13. Rutten, N., van Joolingen, W.R. & van der Veen, J.T.
Comput. Educ. 58, 136–153 (2012).
14. Kiboss, J.K., Ndirangu, M. & Wekesa, E.W. J. Sci. Educ.
Technol. 13, 207–213 (2004).
15. Gelbart, H. & Yarden, A. J. Biol. Educ. 40, 107–112
(2006).
16. Toth, E.E., Morrow, B.L. & Ludvico, L.R. Innovative
High. Educ. 33, 333–344 (2009).
17. Eastwood, J.L. & Sadler, T.D. Comput. Educ. 66,
11–24 (2013).
18. Höffler, T.N. & Leutner, D. Learn. Instr. 17, 722–738
(2007).
19. Mayer, R.E. Multimedia Learning (Cambridge University
Press, 2001).
20. Mayer, R.E. in The Cambridge Handbook of
Multimedia Learning (ed. R.E. Mayer) (Cambridge
University Press, 2005).
21. Gormally, C., Brickman, P., Hallar, B. & Armstrong, N.
J. Coll. Sci. Teach. 40, 45–52 (2011).
22. Anderson, R.D. J. Sci. Teach. Educ. 13, 1–12
(2002).
23. Donnelly, D., O’Reilly, J. & McGarr, O. Res. Sci. Educ.
43, 1571–1592 (2012).
24. The Gordon Commission on the Future of Assessment
in Education. To Assess, to Teach, to Learn: a Vision
for the Future (Princeton, NJ, 2013).
25. Levin, H.M. Prospects 42, 269–284 (2012).
26. Heckman, J.J. Econ. Inq. 46, 289–324 (2008).
27. Singer, S.R. et al. America’s Lab Report: Investigations
in High School Science. Committee on High School
Science Laboratories: Role and Vision (The National
Academies Press, Washington, DC, 2005).
28. National Foundation for Educational Research.
Exploring Young People’s Views on Science Education.
Report to the Wellcome Trust (http://www.nfer.ac.uk/
publications/999901; 2011).
29. Masters, G.N. Psykometrika 47, 149–174 (1982).
30. Embretson, S.E. & Reise, S.P. Item Response Theory for
Psychologists (Lawrence Erlbaum Associates, Mahwah,
NJ, 2000).
31. Wainer, H., Bradlow, E.T. & Wang, X. Testlet Response
Theory and its Applications (Cambridge University
Press, 2007).
32. Birnbaum, A. in Statistical Theories of Mental Test
Scores (eds. F.M. Lord & M.R. Novick) 395–479
(Addison-Wesley, Reading, MA, 1968).
33. Barr, R.B. & Tagg, J. Change 27, 12–26 (1995).
34. Hays, R.T., Jacobs, J.W., Prince, C. & Eduardo, S. Mil.
Psychol. 4, 63–74 (1992).
697