Studies in Educational Evaluation 60 (2019) 43–58

Contents lists available at ScienceDirect

Studies in Educational Evaluation

journal homepage: www.elsevier.com/locate/stueduc

Effect of problem-based learning in pharmacology education: A meta- T

analysis
⁎
Lu Liua,1, Xiaohuang Dub,1, Zuo Zhangc, Jiyin Zhouc,
a
Department of Gastroenterology, the Second Affiliated Hospital, Army Medical University, Chongqing, 400037, China
b
Department of Scientific Research, Army Medical University, Chongqing, 400037, China
c
National Drug Clinical Trial Institution, the Second Affiliated Hospital, Army Medical University, Chongqing, 400037, China

A R T I C LE I N FO A B S T R A C T

Keywords: A consensus is emerging for meta-analyses regarding a range of selected topics in pharmacology education. This
Pharmacology review provides a critical overview of problem-based learning (PBL) in pharmacology education. Based on
Meta-analysis several inclusion criteria, databases were searched, and 37 controlled trial studies were identified and extracted.
Problem-based learning The meta-analysis found that PBL had a positive effect on gaining higher theoretical scores (SMD = 6.77, 95% CI
[5.23, 8.31], p < 0.00,001) assessed through examinations. The results of questionnaires for students’ feedback
showed that PBL was superior to conventional teaching methods in improving students’ outcomes of self-study,
learning interest, team spirit, problem solving, analyzing, scope of knowledge, communication, and expression.
These results suggest that PBL in pharmacology education is considered superior to traditional lecture-based
learning.

1. Introduction
Problem-based learning (PBL) is an “active learning” teaching
method in which complex real-world problems are used as the vehicle
to promote student learning of concepts and principles as opposed to
the passive, spoon-feeding rote learning based on teacher-designed di-
dactic lectures and instructions typical of the traditional curriculum
(Kwan, 2002). PBL has five core characteristics (Barrows, 1996; Hmelo-
Silver, 2004; Schmidt, Molen, Winkel, & Wijnen, 2009), (1) the use of
problems as the start of the learning process, (2) collaborative learning
in small groups, (3) student-centered learning, (4) the guiding role of
teachers, and (5) ample time for self-study. PBL emphasizes active
participation, problem solving, and critical-thinking skills (Smits et al.,
2003). PBL encourages students to identify their own knowledge and
skills and apply them to novel situations or to use them to achieve
specific goals by combining previous knowledge or principles
(Yaqinuddin, 2013), which makes PBL distinct from other student-
centered methods, such as project-based, inquiry-based, and case-based
learning.
Originally, PBL was devised in an attempt to develop a teaching
method for use with medical students (Goodnough, 2006). PBL has
since been used in a variety of disciplines (nursing, medical, dental,
pharmacy, pediatric, physical diagnostics, and even pharmacology)
(Gao et al., 2016; Huang, Zheng, Li, Li, & Yu, 2013; Imran et al., 2015;
Kong, Qin, Zhou, Mou, & Gao, 2014; Wang et al., 2016; Zhou et al.,
2016) and at differing education levels for undergraduate and graduate
education (Miller, 2003; Nybo & May, 2015; Zhang, Zhou et al., 2015).
PBL is adopted to teach pharmacological knowledge to medical stu-
dents. The placement of information regarding drug action in a clinical
context is associated with the hope that medical students will be more
motivated and find it easier to understand and memorize pharma-
cology. A frequently raised concern regarding PBL education is whether
the students are able to acquire sufficient knowledge in pharmacology
via the student-centered tutorials that are facilitated by non-pharma-
cological teachers. Similarly, it is conceivable that non-pharmacological
teachers may feel anxious being placed in a student-centered environ-
ment in which the teachers do not have the total control of what and
how the relevant pharmacological issues are to be learned (Kwan,
2002). When considering the application of a PBL curriculum in phar-
macology education, a pharmacology teacher is confronted with the
problems that previous studies show mixed results for PBL in pharma-
cology.
The major driving force of pharmacology education in PBL is
characterized by student-initiated integration of learning and rea-
soning. Because pharmacology is an ever more rapidly changing sci-
ence, with rapid changes in drug number, knowledge of action

⁎
Corresponding author at: National Drug Clinical Trial Institution, the Second Affiliated Hospital, Army Medical University, Chongqing, 400037, China.
E-mail address: zhoujiyin@gmail.com (J. Zhou).
1
Equal contributors.

https://doi.org/10.1016/j.stueduc.2018.11.004
Received 17 May 2018; Received in revised form 23 November 2018; Accepted 26 November 2018
Available online 30 November 2018
0191-491X/ © 2018 Elsevier Ltd. All rights reserved.
L. Liu et al.
mechanisms, and subcellular genomic information, a change towards
the PBL curriculum appears to be beneficial in better preparing medical
students as life-long learners capable of coping with changes in
knowledge and skills associated with the progressive and dynamic so-
cial/economic transformation (Kwan, 2004). PBL allows students
greater control over their pharmacology learning than the traditional
approach; students determine what they need to learn, and discussions
are typically carried out in small groups (Barrows, 1986, 1996).
Pharmacology is a key curriculum among a number of preclinical
courses in the schools of medicine, nursing, pharmacy, dentistry, and
preventive medicine. Experimental pharmacology is vital to our un-
derstanding of drug action in the treatment of disease, as well as to the
pharmaceutical industry for drug discovery and development. Clinical
pharmacology is essential for prescribing practice in medicine and
nursing and underpins pharmacy practice and therapeutics (Lloyd et al.,
2013). The teaching and learning of pharmacology conventionally oc-
curs after the other core courses of medical sciences, such as anatomy
and physiology. After a half-year of study, students can acquire basic
knowledge of the fundamental principles of pharmacology and its
clinical applications, basic skills in performing pharmacology experi-
ments, and professional knowledge of the laws and regulations of drug
administration (Li, Zhang, Zhang, & Zhang, 2004).
Pharmacology needs the integrated effort of multiple techniques
(molecular, biochemical, cellular, and system-based) to reach a total
understanding of the action of drugs. The increasing number of drugs,
changes in knowledge on general principles of mechanisms of drug
action and more detailed information (particularly on the subcellular
and genomic levels) regarding properties of drugs require consequently
innovative approaches to undergraduate pharmacology education. The
pharmacology education of medical students differs from that of science
students because medical students not only have to understand me-
chanisms of drug action but also must memorize numerous detailed
facts regarding drug classes, individual compounds (hundreds of med-
icines), and their practical uses many times before they learn suffi-
ciently about related diseases. Pharmacology is one of the core subjects
for further graduation in both preclinical and clinical areas.
Traditional pharmacology teaching-learning has been criticized for
neither preparing students for medical practice nor teaching the safe
and rational use of medicines (Shah, Desai, Jorwekar, Badyal, & Singh,
2016). Medical schools have a major challenge in teaching students to
choose medicines based on objective, scientific principles and to use
them in a safe and effective manner. Problem-based undergraduate
medical education in pharmacology, which is based on national es-
sential medicines lists and standard treatment guidelines, has been
strongly recommended as a key intervention to improve prescribing
(Shankar, 2011).
Various countries and schools organize medical school teaching in
diverse ways, and the optimal approaches to pharmacology education
will rely on the whole curriculum structure. There are also differences
in compulsory background information level, extent of knowledge and
the number of lectures and exercises in pharmacology classes (Tisonova
et al., 2005). In China and Germany for example, students enter medical
school directly from high school, while in the USA, medical school
admission requires graduation from a college. Moreover, pharmacology
education is performed in different semesters of the medical school
curriculum. The combination of the above-mentioned factors and other
factors will decide how much background knowledge medical students
will have before beginning their pharmacology education. According to
the different pharmacology education contents, different universities/
colleges may use different variations of PBL Michel, Bischoff, & Jakobs,
2002; Michel, Bischoff, Zu Heringdorf et al., 2002), which may impair
the outcome of PBL education on pharmacology. In most studies for
example, student feedback on the PBL education in pharmacology is
obtained using a questionnaire that consists of 5 to 8 statements. The
students in most studies are asked to select their agreement with the
following 3 items: no improvement, average improvement, and
44
Studies in Educational Evaluation 60 (2019) 43–58
significant improvement (Shankar et al., 2004).
A meta-analysis is a formal statistical analysis of data from various
subgroups of pre-existing studies. This analysis is conducted to obtain
an overall estimate of the effectiveness of the selected subject/inter-
vention. Importantly, a meta-analysis refers to statistical methods of
combining evidence, leaving aside other aspects of “research synthesis”
or “evidence synthesis”, such as combining information from qualita-
tive studies. PBL is increasingly used in medical programs around the
world, but more evidence is needed to support that PBL is more effec-
tive than traditional methods, specifically in pharmacology. Medical
teachers are naturally interested in whether PBL would contribute to
the desired competencies of future physicians (Albanese, 2000). How-
ever, a portion of teachers remain skeptical about whether PBL will be
equally effective in teaching pharmacological facts and, if it is as ef-
fective, whether this is possible without excessive use of existing re-
sources (Michel, Bischoff, & Jakobs, 2002). Although several meta-
analyses of the evaluative research evidence for PBL have used assess-
ment to suggest its effectiveness as a conventional approach in medical
education, such as increased physician competency (Albanese &
Mitchell, 1993; Dochy, Segers, Van den Bossche, & Gijbels, 2003; Gao
et al., 2016; Gijbels, Dochy, Den Bossche, & Segers, 2005; Huang et al.,
2013; Koh, Khoo, Wong, & Koh, 2008; Tayyeb, 2013; Vernon & Blake,
1993; Wang et al., 2016), meta-analysis with specific to pharmacology
education is still lacking. The aim of this meta-analysis was to examine
the theoretical scores assessed through examinations and questionnaire
for students’ feedback of PBL in relation to pharmacology education.
2. Method
2.1. Inclusion criteria
The inclusion criteria included the following elements. First, the
studies must have quantitative outcomes that focused on student
learning processes. Although we also selected non-empirical literature
and literature reviews as relevant sources of research, they were not
included in this analysis. Second, an experimental and comparative
design must have been used that examined the effectiveness of PBL
(treatment group) relative to lecture-based learning (control group).
The dependent variable in these studies had to consist of the theoretical
scores (test scores) as the main goal of PBL. Third, the subject of the
study had to be pharmacology education. In total, we identified 1266
articles; 34 articles met the inclusion criteria and were included in the
analysis (Fig. 1).
2.2. Literature search
A thorough search was conducted on the electronic databases
EBSCO, PubMed, Web of Science, PQDT, ERIC, EMBASE, Cochrane
Database, Chinese National Knowledge Infrastructure (CNKI), VIP
Information (Chinese database), Wanfang Data (Chinese database), and
Chinese Biomedical documental database (CBM). We searched each
electronic database specifically for empirical articles that fit our in-
clusion criteria; 2 combined keywords for the first search are “problem-
based education” and “pharmacology education”, for the second search
are “problem-based learning” and “pharmacology education”.
Literature was selected on the basis of the abstracts provided from 1965
to May 2017 on the effectiveness of PBL. In addition, we gathered re-
view articles and theoretical overviews in order to verify the references.
2.3. Data extraction
Data were extracted by two independent reviewers. Disagreements
about eligibility were resolved by consensus. However, there were no
disagreements about eligibility in this meta-analysis. For each study,
the following information was extracted in the current analysis: the first
author, publication year, country, university/college, the involved
L. Liu et al.
Fig. 1. Flow of study analysis through different phases of the meta-analysis.
disciplines, sample size (intervention group and control group), char-
acteristics of participants, intervention method, teaching method in the
control group, outcomes (the mean difference of theoretical scores for
the intervention group and control group), time of measuring outcome,
outcomes measured (all were assessed by examination) and length of
intervention.
2.4. Quality assessment
Methodological quality assessments of the included studies were
assessed as adequate, uncertain, or inadequate by two members, and
this assessment was based on the six general sources of bias described in
the Cochrane Handbook for Systematic Reviews of Interventions
(Higgins, 2008). The methodological quality factors checked were the
following: adequacy of the generation of the allocation sequence, con-
cealment of allocation, blinding procedures, incomplete outcome data,
selective outcome reporting and other sources of bias. Information
addressed by these methodological quality checks was obtained from
the published reports, and authors were contacted if additional in-
formation was required (Galvao, Silva, Neiva, Ribeiro, & Pereira, 2014).
The Cochrane Collaboration's tool for assessing risk of bias is available
online at http://handbook.cochrane.org/.
45
Studies in Educational Evaluation 60 (2019) 43–58
2.5. Statistical analysis
The literature data, including authors, publication year, and cases of
participants, were input into Review Manager 5.3.5 software
(Collaboration, 2014) for tests of heterogeneity and meta-analysis. We
analyzed the theoretical scores of the PBL group and control group.
Because continuous data from different scales were extracted, the
standardized mean difference (SMD) was calculated for effect size based
on sample size and 95% confidence intervals (CIs) for each study and
for the pooled studies using variance analysis. Weighted mean differ-
ence and 95% confidence intervals (CIs) were calculated for continuous
data from the same scale. A two-sided P value less than 0.05 was re-
garded as significant for all analyses. There were two models of meta-
analysis. The fixed effect model was used to pool data if there was no
heterogeneity; otherwise, we used the random effects model. Hetero-
geneity was considered significant for p value of Cochran's Q sta-
tistic < 0.10 and I2 > 50% (Higgins & Thompson, 2002; Higgins,
Thompson, Deeks, & Altman, 2003). I2 was the percentage of variation
attributed to heterogeneity and was easily interpreted. I2 statistic of
25%–50% was considered low, 50%–75% was considered moderate,
and ≥ 75% was considered high.
If there was heterogeneity, we conducted sensitivity analysis to
assess if this significantly altered the results of the meta-analysis. A
sensitivity analysis was performed by excluding each one of these
 Table 1
The study characteristics of 34 included articles.
Study ID
Antepohl and Herzig (1999)
Bao et al. (2008)
Cao et al. (2011)
Chen et al. (2012)
46
Cui et al. (2011)
Fu et al. (2016)
Gao (2015)
Guo et al. (2013)
Herzig et al. (2003)
Country/Name of University or
College
Germany/University of Cologne
China/Changsha Medical
College
China/Capital Medical
University
China/China Medical University
China/Tianjin University of
Traditional Chinese Medicine
China/Liaoning University of
Traditional Chinese Medicine
China/Zunyi Medical College
China/Dalian University
Germany/University of Cologne
L. Liu et al.
Disciplines Sample size Participants characteristics Interventions Comparator Duration of
(PBL/LBL) intervention
Medicine 55/57 Fifth semester medical students Using PBL as the educational approach. Using lecturing as the teaching 1 semester
from one class There are 8 PBL tutorial groups and each group approach.
Mean age 24.0 for PBL and 24.7 for consisted of 6-9 participants. The students were divided into 3
LBL, 60 females and 64 males The PBL approach used in the tutorials was a seminary groups, each
modified version of the Maastricht “Seven Jump”. comprising 20 students. The
The scheduled course time (3 h) was divided into 2 h scheduled lecture time was 3 h.
in a PBL tutorial plus 1 h of classroom teaching for
the whole study group.
Medicine 25/25 Year two medical students from 1 Using PBL as the educational approach. Using lecturing as the teaching 1 semester
classes. Each group consisted of either 5-6 students. Total approach.
Mean age 20.8 (SD 1.7) for LBL, have 20 class hour. Total have 68 class hour.
21.1 (SD 0.8) for PBL. The teaching process includes laying out problem,
self-study, discussion, summary, and solving
problem.
Medicine 61/65 Year three medical students from 4 Using PBL as the educational approach. Using lecturing as the teaching 1 semester
classes. The scheduled lecture time was 3 h. approach.
The teaching process includes laying out problem, The scheduled lecture time was
search information, discussion, comment, and 3 h.
conclusion.
Medicine 30/29 Year two medical students from 5 Using PBL as the educational approach. Using lecturing as the teaching 1 semester
classes Each group consisted of 8-10 students. Total have 68 approach.
class hour. Total have 68 class hour.
The teaching process includes laying out problem,
search information, communication, discussion,
summary, and solving problem.
Pharmacy 51/48 Year two pharmacy students from Using PBL as the educational approach. Using lecturing as the teaching 1 semester
2 classes There are 4 groups and each group consisted of approach.
either 10-12 students.
The teaching process includes laying out problem,
search information, communication, discussion,
summary, and effect evaluation.
Medicine 54/134 Year two medicine students from 2 Using PBL as the educational approach. Using lecturing as the teaching 1 semester
classes The teaching process includes laying out problem, approach.
search information, summarize problem, group
discussion, summary, and answer questions.
Pharmacy 60/58 Year two pharmacy students from Using PBL as the educational approach. Using lecturing as the teaching 1 semester
2 classes There are 12 groups and each group consisted of 4-6 approach.
students.
The teaching process includes laying out problem,
search information, group discussion, evaluation,
and summary.
Nurse 27/29 Year two nurse students from 2 Using PBL as the educational approach. Using lecturing as the teaching 1 semester
classes There are 4 groups and each group consisted of 6-7 approach.
students. Total have 6 class hour. Total have 6 class hour.
The teaching process includes laying out problem,
search information, communication, discussion, and
summary.
Medicine 55/57 for Year three to five medical students PBL was introduced into a 3rd-year course of basic Using lecturing as the teaching 1 semester
year three from 2 classes medical pharmacology consisting of ten 2-hour approach.
45/45 for sessions with 112 students, 90 took the 4th-year
year four course and final exam on clinical pharmacology, 32
17/15 for participated an additional, formative exam of the
year five
(continued on next page)
Studies in Educational Evaluation 60 (2019) 43–58
 Table 1 (continued)
Study ID Country/Name of University or
College
Huang (2017) China/ Harbin Medical
University
Huo et al. (2016) China/Guangxi University of
Chinese Medicine
Imran et al. (2015) India/MSDS Medical College
Jia (2013) China/Qufu Normal University
47
Li et al. (2011) China/Henan Agricultural
University
Li et al. (2013) China/Liaoning Medical
University
Li and Xi (2014) China/Shandong Wanjie
Medical College
Liu (2014) China/Yongzhou Vocational
Technical College
Luo (2012) China/Guangdong Xinxing
traditional Chinese Medicine
College
Miao et al. (2013) China/Anhui Science and
Technology University
Disciplines Sample size Participants characteristics Interventions Comparator Duration of
L. Liu et al.
(PBL/LBL) intervention
same type after a mandatory 5th-year course (non-
pharmacology).
Medicine 40/40 Third semester medicine students Using PBL as the educational approach. Using lecturing as the teaching 1 semester
from 1 classes The teaching process includes provide typical cases, approach.
laying out problem, search information, discussion,
solve problem, and summary.
Medicine 56/57 Year two medicine students from 1 Using PBL as the educational approach. Using lecturing as the teaching 1 semester
classes The teaching process includes laying out problem, approach.
provide typical cases, discussion, solve problem, and
summary.
Medicine 25/25 MBBS students posted in the Comprehensive discussion on the topic over 2 weeks. Using lecturing as the teaching 3 months
Department of Pharmacology for Students were given case documents, including approach.
practical classes as batch A from patient complaints, presenting illness, and sample
January to March 2015. questions about the etiology, diagnosis, relevant
investigations like drug resistance and treatment
before each session. The group presented the drugs
prescribed/treatment given with rationale from
pharmacology point of view. Students discussed and
debated various aspects of the case to arrive at an
approach to the treatment and rationale of
prescribing drugs to the individual patients.
Pharmacy 45/46 Year two pharmacy students from Using PBL as the educational approach. Using lecturing as the teaching 1 semester
2 classes. There are 9 groups and each group consisted of 5 approach.
Mean age 19.3 (SD 0.5), male 29, students.
female 16 for PBL; mean age 19.5 The teaching process includes laying out problem,
(SD 0.4), male 30, female 16, for search information, group discussion, and summary.
LBL.
Pharmacy 54/54 Fifth semester pharmacy students Using PBL as the educational approach. Using lecturing as the teaching 1 semester
from 4 classes There are 8 groups and each group consisted of 6-7 approach.
students. Total have 6 class hour.
The teaching process includes laying out problem,
search information, designing experiment, doing
experiment, evaluation, and summary.
Medicine 57/63 Year three medical students from 2 Using PBL as the educational approach. Using lecturing as the teaching 1 semester
classes The teaching process includes laying out problem, approach.
search information, discussion, and summary.
Medicine 40/40 Year two medical students from 1 Using PBL as the educational approach. Using lecturing as the teaching 1 semester
class Each group consisted of 5 students. approach.
The teaching process includes laying out problem,
group discussion, class discussion, and summary.
Nurse 108/108 Year two nurse students from 1 Using PBL as the educational approach. Using lecturing as the teaching 1 semester
classes Total have 68 class hour. approach.
The teaching process includes laying out problem, Total have 68 class hour.
self-study, discussion, and summary.
Pharmacy 52/52 Year two pharmacy students from Using PBL as the educational approach. Using lecturing as the teaching 1 semester
4 classes Each group consisted of 5-6 students. approach.
The teaching process includes laying out problem,
slef-study, communication, group discussion, class
discussion, and summary.
Pharmacy 30/30 Year two pharmacy students from Using PBL as the educational approach. Using lecturing as the teaching 1 semester
2 classes There are 6 groups and each group consisted of 5 approach.
students.
The teaching process includes laying out problem,
(continued on next page)
Studies in Educational Evaluation 60 (2019) 43–58
 Table 1 (continued)
Study ID
Michel, Bischoff, & Jakobs
(2002), Michel, Bischoff,
Zu Heringdorf et al.
(2002)
Miller (2003)
Pan (2014)
Song (2011)
Song et al. (2015)
48
Wang, Wu et al. (2008),
Wang, Wie et al. (2008)
Wang, Wu et al. (2008),
Wang, Wie et al. (2008)
Wei and Zheng (2013)
Xiang and Yang (2015)
Yang and Jin (2008)
Country/Name of University or
College
Germany/Universitätsklinikum
Essen
USA/University of Nevada at
Las Vegas
China/Changsha Social Work
College
China/Shandong University of
Traditional Chinese Medicine
China/Hunan University of
Chinese Medicine
China/China Medical University
China/Henan University of
Science and Technology
China/Capital Medical
University
China/Honghe Health
Vocational College
China/Dalian University
Disciplines Sample size Participants characteristics Interventions Comparator Duration of
L. Liu et al.
(PBL/LBL) intervention
search information, designing experiment, doing
experiment, and evaluation.
Medicine 60/129 Year four medical students from The PBL groups (9-12 students each) received ten 2-h LBL students were assigned to 1 semester
one class sessions in which a clinical case was discussed and groups of 14-15 students and
ten 2-h sessions in which areas of pharmacology not received 20 2-h sessions.
covered by the case discussions were presented in an
LBL format (one group with all 31 students).
Nurse 10/12 Year two nurse master from two Using PBL as the educational approach. The control group class was 1 semester
class After the first class meeting for introduction, conducted as any lecture class.
students met weekly to discuss the new concepts
learned as a result of investigating the learning issues
that emerged from the previous week’s problem,
then the teacher presented the class with a new
clinical problem.
Nurse 99/102 Year two nurse students from four Using PBL as the educational approach. The control group class was 1 semester
classes Each group consisted of either 3-5 students. conducted as any lecture class.
The teaching process includes laying out problem,
searching and gain information, group discussion,
and summary.
Medicine 63/63 Year two medical students from 2 Using PBL as the educational approach. Using lecturing as the teaching 1 semester
classes Total have 10 class hour. approach. Total have 10 class
The teaching process includes laying out problem, hour.
search information, designing experiment, doing
experiment, and solving problem.
Pharmacy 90/93 Year two pharmacy students from Using PBL as the educational approach. Using lecturing as the teaching 1 semester
8 classes Total have 96 class hour (64 theory and 32 approach. Total have 96 class
experimental class hour). hour (64 theory and 32
There are 10 groups and each group consisted of experimental class hour).
either 8-10 students.
The teaching process includes laying out problem,
search information, self-study, solving problem,
discussion, supplemental answer, and summary.
Medicine 65/172 Medical students from 4 classes Using PBL as the educational approach. Using lecturing as the teaching 1 semester
There are 12 groups and each group consisted of approach.
either 5-6 students.
The teaching process includes laying out problem,
preparing lessons, discussion, analysis, rediscussion,
and summary.
Medicine 189/120 Medical students from 7 classes Using PBL as the educational approach. Using lecturing as the teaching 1 semester
The teaching process includes laying out problem, approach.
search information, solving problem, and effect
evaluation.
Medicine 87/91 Year three medical students from 4 Using PBL as the educational approach. Using lecturing as the teaching 1 semester
classes There are 9 groups and each group consisted of approach. Total have 8 class
either 9-10 students. Total have 8 class hour. hour.
The teaching process includes laying out problem,
discussion, evaluation, and supplementation.
Nurse 114/58 Nurse students from 3 classes Using PBL as the educational approach. Using lecturing as the teaching 1 semester
Each group consisted of either 6-7 students. approach.
The teaching process includes designing problem,
assignment learning task, group discussion, class
discussion,evaluation, and summary.
Nurse 26/30 Nurse students from 2 classes Using PBL as the educational approach. Using lecturing as the teaching 1 semester
There are 3 groups and each group consisted of approach.
(continued on next page)
Studies in Educational Evaluation 60 (2019) 43–58
 Table 1 (continued)
Study ID
Zhang et al. (2005)
Zhang, Zhang et al. (2015);
Zhang, Zhou et al.
(2015)
Zhao et al. (2014)
Zhao (2015)
Zhou et al. (2012)
Note: LBL, lecture based learning.
49
Country/Name of University or
College
China/Shaoxing Arts and
Sciences College
China/Xinxiang Medical
College
China/Hubei Medical College
China/Heilongjiang Bayi
Agricultural University
China/Guangdong Medical
College
Disciplines Sample size Participants characteristics Interventions Comparator Duration of
L. Liu et al.
(PBL/LBL) intervention
either 8-10 students.
The teaching process includes laying out problem,
analysis, discussion, evaluation, and summary.
Medicine 30/30 Year two medical students from 1 Using PBL as the educational approach. Using lecturing as the teaching 1 semester
classes Total have 6 class hour. approach. Total have 6 class
The teaching process includes laying out problem, hour.
search information, self-study, discussion, and
summary.
Medicine 78/82 Medical students from 1 class Using PBL as the educational approach. Using lecturing as the teaching 1 semester
The teaching process includes laying out problem, approach.
search information, discussion, and summary.
Medicine 80/47 Medical students from 4 classes Using PBL as the educational approach. Using lecturing as the teaching 1 semester
The teaching process includes laying out problem, approach.
search information, discussion, solving problem,
summary, and evaluation.
Pharmacy 70/70 Pharmacy students from 1 class Using PBL as the educational approach. Using lecturing as the teaching 1 semester
The teaching process includes laying out problem, approach.
search information, class discussion, and summary.
Pharmacy 34/34 Year two pharmacy students from Using PBL as the educational approach. Using lecturing as the teaching 1 semester
2 classes There are 6 groups. approach.
The teaching process includes laying out problem,
group discussion, and supplementation.
Studies in Educational Evaluation 60 (2019) 43–58
L. Liu et al. Studies in Educational Evaluation 60 (2019) 43–58

Table 2
Risk of bias assessment of 34 included randomized controlled articles.
Study ID Randomization Allocation Blinding Incomplete Data Selective Data Other Bias
Concealment Report Report

Antepohl and Herzig (1999) Yes Unclear Yes None None None
Bao et al. (2008) Unclear Unclear Unclear None None None
Cao et al. (2011) Unclear Unclear Unclear None None None
Chen et al. (2012) Yes Unclear Unclear None None None
Cui et al. (2011) Unclear Unclear Unclear None None None
Fu et al. (2016) Unclear Unclear Unclear None None None
Gao (2015) Unclear Unclear Unclear None None None
Guo et al. (2013) Unclear Unclear Unclear None None None
Herzig et al. (2003) Yes Unclear Yes None None None
Huang (2017) Yes Unclear Unclear None None None
Huo et al. (2016) Yes Unclear Yes None None None
Imran et al. (2015) Yes Unclear Unclear None None None
Jia (2013) Unclear Unclear Unclear None None None
Li et al. (2011) Unclear Unclear Unclear None None None
Li et al. (2013) Unclear Unclear Unclear None None None
Li and Xi, (2014) Yes Unclear Unclear None None None
Liu (2014) Yes Unclear Unclear None None None
Luo (2012) Unclear Unclear Unclear None None None
Miao et al. (2013) Unclear Unclear Unclear None None None
Michel, Bischoff, & Jakobs (2002); Michel, Bischoff, Zu Yes Unclear Yes None None None
Heringdorf et al. (2002)
Miller (2003) Yes Unclear Yes None None None
Pan (2014) Unclear Unclear Unclear None None None
Song (2011) Unclear Unclear Unclear None None None
Song et al. (2015) Unclear Unclear Unclear None None None
Wang, Wu et al. (2008), Wang, Wie et al. (2008) Yes Unclear Unclear None None None
Wang, Wu et al. (2008), Wang, Wie et al. (2008) Unclear Unclear Unclear None None None
Wei and Zheng (2013) Unclear Unclear Unclear None None None
Xiang and Yang (2015) Unclear Unclear Unclear None None None
Yang and Jin (2008) Unclear Unclear Unclear None None None
Zhang et al. (2005) Unclear Unclear Unclear None None None
Zhang et al. (2005) Unclear Unclear Unclear None None None
Zhao et al. (2014) Yes Unclear Unclear None None None
Zhao et al. (2015) Unclear Unclear Unclear None None None
Zhou et al. (2012) Unclear Unclear Unclear None None None

studies and subsequently recalculating the pooled estimates for the
remaining studies to ensure that it did not significantly alter the results.
Publication bias was assessed by funnel plot and Egger's test of
asymmetry (Egger, Davey Smith, Schneider, & Minder, 1997), which
were conducted using STATA 13.0. Funnel plot shapes did not reveal
obvious evidence of asymmetry, and all of the P values of Egger's tests
were more than 0.05, providing statistical evidence of the funnel plot
symmetry.
3. Results
3.1. Characteristics of included studies
Thirty-four articles (Antepohl & Herzig, 1999; Bao, Li, & Jiang,
2008; Cao, Li, & He, 2011; Chen et al., 2012; Cui, Zhong, He, & Zhang,
2011; Herzig, Linke, Marxen, Borner, & Antepohl, 2003; Fu, Yang, & Li,
2016; Gao, 2015; Guo, Wang, & Li, 2013; Huang, 2017; Huo, Yang, &
Gong, 2016; Imran et al., 2015; Jia, 2013; Li & Xi, 2014; Li, Chang,
Zhou, & Gao, 2011; Li, Jiang, Zhang, & Wang, 2013; Liu, 2014; Luo,
2012; Miao, Zhou, & Qin, 2013; Miller, 2003; Pan, 2014; Song, 2011;
Song, Huang, Cai, Zeng, & Wang, 2015; Wang, Wu, Wang, & Qian,
2008; Wang, Wei et al., 2008; Wei & Zheng, 2013; Xiang & Yang, 2015;
Yang & Jin, 2008; Zhang, Zhang et al., 2015; Zhang, Zhou, & Shen,
2005; Zhao, Liu, Yan, Sun, & Zhang, 2015; Zhao, Lan, Zhu, & Gong,
2014; Zhou, Luo, Qin, & Cui, 2012), which represented 37 controlled
trial studies, were included in the meta-analysis. The studies were all
published between 1999 and 2017 in English or Chinese. The sample
size ranged from 10 to 189 participants, and the pooled sample size was
4406 (PBL group = 2137, control group = 2269). There were 19 arti-
cles for medicine (Antepohl & Herzig, 1999; Bao et al., 2008; Cao et al.,
2011; Chen et al., 2012; Fu et al., 2016; Herzig et al., 2003; Huang,
2017; Huo et al., 2016; Imran et al., 2015; Li & Xi, 2014; Li et al., 2013;
Song, 2011; Wang, Wu et al., 2008; Wang, Wie et al., 2008; Wei &
Zheng, 2013; Zhang, Zhang et al., 2015; Zhang et al., 2005; Zhao et al.,
2014), nine articles for pharmacy (Cui et al., 2011; Gao, 2015; Jia,
2013; Li et al., 2011; Luo, 2012; Miao et al., 2013; Song et al., 2015;
Zhao et al., 2015; Zhou et al., 2012), and six articles for nursing (Guo
et al., 2013; Liu, 2014; Miller, 2003; Pan, 2014; Yang & Jin, 2008;
Xiang & Yang, 2015). Outcomes were measured by theoretical scores
and questionnaire surveys after PBL teaching. The lengths of the in-
tervention varied between 3 months to one semester. All studies mea-
sured the participants’ theoretical scores at the end of the intervention
using PBL as the teaching method. Table 1 shows the study character-
istics of the 34 included articles.
3.2. Study quality
Table 2 shows the risk of bias assessment of the 34 included articles.
Thirteen articles had a low risk of bias across the six domains (Gao,
2015; Guo et al., 2013; Imran et al., 2015; Jia, 2013; Li et al., 2013;
Luo, 2012; Miller, 2003; Pan, 2014; Song et al., 2015; Wang, Wu et al.,
2008; Wei & Zheng, 2013; Xiang & Yang, 2015; Zhao et al., 2014). The
allocation sequence in one article (Miller, 2003) was adequately gen-
erated by coin toss to determine assignment of groups. In another ar-
ticle (Zhao et al., 2014), the allocation sequence was generated by
stratified random conducted in accordance with the college entrance
theoretical scores. In another article (Imran et al., 2015), the students
were divided into two groups by simple randomization. The allocation
sequence of 10 articles (Gao, 2015; Guo et al., 2013; Jia, 2013; Li et al.,
2013; Luo, 2012; Pan, 2014; Song et al., 2015; Wang, Wu et al., 2008;

50
L. Liu et al.
Fig. 2. Meta-analysis and forest plot of theoretical scores after using PBL compared with traditional lectures.
Wei & Zheng, 2013; Xiang & Yang, 2015) was generated by the pre-
ference of the researchers, in which the students were assigned to either
an experimental group or a control group by the researchers’ decision
or not mentioned in detail. The other 21 articles did not report the
method used, and information was not available even after contacting
with authors of these articles. Due to the nature of the intervention,
blinding of students and teachers to the intervention is not feasible. All
articles reported complete outcome data. We assessed whether the
study was free from selective outcome reporting, which was checked
whether all outcomes mentioned in these reports were adequately re-
ported in the results section. All articles adequately reported the results.
3.3. Effects of interventions on theoretical scores
The effects of PBL methods were evaluated both by data and de-
scription. Thirty-four articles involving 4406 students (PBL group =
2137, control group = 2269) reported theoretical scores. Chen et al.
(2012); Luo (2012), and Zhao et al. (2014) all reported two sets of
outcome data, thus, there were 37 reports of outcome data in the meta-
analysis. This meta-analysis reports the effects based on the synthesis of
theoretical scores, which were more objective than other indexes, such
as questionnaire. Twenty-six studies showed statistically significant
51
Studies in Educational Evaluation 60 (2019) 43–58
differences in pharmacology students’ theoretical scores between the
PBL and the control group, but the remaining eleven studies showed no
significant differences in theoretical scores between the PBL and the
control group. In statistics, the fixed effects model is a statistical model
that represents the observed quantities in terms of explanatory vari-
ables that are treated as if the quantities were non-random. This ap-
proach is in contrast to the random effects model, in which all of the
explanatory variables are treated as if they arise from random causes.
There was high heterogeneity (I2 = 97%, p < 0.00,001), thus, the
random effects model was used. The pooled effect size showed sig-
nificant difference in theoretical scores (SMD = 6.77, 95%
CI = 5.23–8.31, p < 0.00,001) in favor of PBL, compared with tradi-
tional lectures (Fig. 2). The results indicate that PBL has an effect on
improving students’ theoretical scores. The fixed effect model was also
applied to pool the data, and the pooled effects also favored the PBL
group (SMD = 4.99, 95% CI = 4.75–5.24, p < 0.00,001).
Because of the heterogeneity observed between studies reporting
theoretical scores, a sensitivity analysis was carried out to verify the
reliability of the result. Sensitivity analysis was performed by sequential
omission of individual studies. With the exclusion of 11 studies from ten
articles (Gao, 2015; Guo et al., 2013; Jia, 2013; Li et al., 2013; Luo,
2012; Pan, 2014; Song et al., 2015; Wang, Wu et al., 2008; Wei &
L. Liu et al.
Fig. 3. Funnel plot analysis for theoretical scores.
Zheng, 2013; Xiang & Yang, 2015) with inadequate generation of a
randomized sequence from the analyses of theoretical scores, the
pooled effect size still favored the PBL group (SMD = 4.81, 95%
CI = 3.15–6.48, p < 0.00,001) and did not change the effects ob-
served in the primary analysis.
The funnel plot for the 37 studies on theoretical scores analysis is
shown in Fig. 3. The funnel plot's shape is symmetrical. There was no
significant publication bias indicated in the main analysis.
With the exclusion of 11 studies from 10 articles (Antepohl &
Herzig, 1999; Bao et al., 2008; Chen et al., 2012; Herzig et al., 2003;
Miller, 2003; Wang, Wei et al., 2008; Wei & Zheng, 2013; Zhang, Zhou
et al., 2015; Zhou et al., 2012) with no effect of PBL on theoretical
scores, the pooled effect size still favored the PBL group (pooled sample
size was 3241, 1648 for PBL group, 1593 for control group; SMD =
9.07, 95% CI = 7.38–10.76, p < 0.00,001) and did not change the
effects observed in the primary analysis.
3.4. Effects of interventions on questionnaire
Table 3 shows the categories of statements, description levels, and
conclusion of questionnaire for 34 included articles. Excluding 3 Eng-
lish articles (Herzig et al., 2003; Imran et al., 2015; Miller, 2003) and 4
Chinese articles (Liu, 2014, Cui et al., 2011; Yang & Jin, 2008; Zhou
et al., 2012), the questionnaire results show that most students at the
PBL pilot institutions have more enthusiasm for PBL methods, rather
than conventional teaching methods. Because most studies use different
questionnaire contents with variable order statements and with dif-
fering description levels to evaluate each statement, students' learning
ability cannot be analyzed statistically. The questionnaire results show
that the PBL methods are superior to the conventional teaching
methods in improving the students’ outcomes, such as self-study,
learning interest, team spirit, problem solving, analyzing, scope of
knowledge, communication, and expression.
4. Discussion
Our meta-analysis examined the contents of 37 studies from 34
articles that addressed the effects of PBL in pharmacology education
and synthesized the effects and influence of PBL according to theore-
tical scores and questionnaires. The results revealed that PBL had a
52
Studies in Educational Evaluation 60 (2019) 43–58
positive effect on students’ gaining higher theoretical scores in phar-
macology education. The questionnaire results also show that PBL
methods are superior to the conventional teaching methods in im-
proving the students’ outcomes of self-study, learning interest, team
spirit, problem solving, analyzing, scope of knowledge, communication,
and expression. This finding indicates that PBL offers many benefits to
pharmacology education research compared with traditional methods.
These results support findings from research in nursing (Kong et al.,
2014), medical (Tayyeb, 2013), and dental education (Huang et al.,
2013), as well as pharmacy (Zhou et al., 2016), pediatric (Gao et al.,
2016), preventive medicine (Ding et al., 2014), and physical diagnostics
in China (Wang et al., 2016).
The theoretical score for each study included in the meta-analysis is
one of the main objective indexes with which to compare the effects of
PBL with the control group. Since the pooled sample size for English
articles is 405 and for Chinese articles is 4001, more studies outside of
China are required to reevaluate the effect of PBL on pharmacology.
Due to the selected bias of language and countries, the conclusion
should carefully adapt PBL in pharmacology education for other
countries.
Over the last decade, we have seen considerable interest in phar-
macology education calling for better integration of basic and clinical
pharmacology with the use of PBL (Antepohl & Herzig, 1999; Des
Marchais, 1999; Jones, Walley, & Bligh, 1997; Kaufman & Mann, 1997;
Sivam, Iatridis, & Vaughn, 1995; Vestal & Benowitz, 1992) and for the
training of pharmacologists who are “jacks of all trades” and “masters
of their own” (Page, Sutter, & Walker, 1994; Rangachari, 1994a). To
date, the available evidence suggests that the PBL approach of learning
medical pharmacology benefits medical students (Michel et al, 2002,
Sim, Achike, & Geh, 2005; Vestal & Benowitz, 1992), and it has also
been proven promising in the teaching and learning of basic pharma-
cology for non-medical undergraduate life-science students
(Rangachari, 1994b). As Dochy et al. (2003) have mentioned, the dif-
ference in effect size between student expertise levels is remarkable.
Therefore, pharmacists and senior students may be more motivated and
effective when confronting PBL scenarios because they have already
had real-world experience compared with junior students. However,
some studies showed that there was no positive effect on medical
judgment (Hoffman, Hosokawa, Blake, Headrick, & Johnson, 2006) and
that the benefits of PBL was confirmed by a randomized controlled trial
L. Liu et al.
Table 3
The categories of statements, description levels, and conclusion of questionnaire for 34 included articles.
Study ID
Antepohl and Herzig (1999)
Bao et al. (2008)
Cao et al. (2011)
Chen et al. (2012)
Cui et al. (2011)
Fu et al. (2016)
Gao (2015)
Guo et al. (2013)
Herzig et al. (2003)
Huang (2017)
Studies in Educational Evaluation 60 (2019) 43–58
Categories of statements Description levels Conclusion
From high to low Mean score, 5-point-Likert-scale from 1 Positive effects of PBL in terms of use of additional
1 The PBL cases illustrate medical (disagree at all) to 5 (agree completely) learning resources, inter-disciplinarity, team spirit
concepts and learning interest
2 PBL method is worth recommend
and well educates students
3 Important and interest in subject of
this PBL tutorial
4 Adequate teaching ability of the
tutor
5 PBL is an effective self-learning style,
open, and fun
6 The content of the tutorial fits
knowledge level and motivates to use
additional learning resources
7 The group climate facilitates the
learning process and uses additional
learning time
8 The learning outcome of the tutorial
is useful and large
9 Improve communication skills
From high to low Mean score, 5-point-Likert-scale from for PBL has positive effects on learning attitude and
1 Learning attitude learning attitude and Instructional motivation than lecture-based learning
2 Learning motivation Materials Motivation Survey for learning
motivation
From high to low Percentage for positive effect of PBL (agree, PBL method is beneficial for students in clinical
1 Strength communication skills disagree) teaching
between teacher and students
2 Improve classroom climate to
strength understand basic knowledge
3 Improve language expression ability
4 Improve learning interest
5 Improve problem analyzing and
solving ability
From high to low Percentage for positive effect of PBL on PBL promotes asking question, discussion, and self-
1 Question frequency question frequency and self-study time study
2 Discussion and presentation
3 Literature reach and read
4 Self-study time
NA NA NA
From high to low Agreement degree for positive effect of PBL PBL promotes self-study ability, problem solving,
1 Improve self-study ability (agree, neutral, disagree) ability, scope of knowledge, and expression ability
2 Improve problem solving ability
3 Expand knowledge scope
4 Improve comprehensive and
language expression ability
From high to low Percentage for positive effect of PBL (agree, PBL has positive effects on motivation, analyzing
1 Improve self-study ability disagree) skills, scope of knowledge, communication skills, and
2 Improve problem analyzing and team spirit
solving ability
3 Improve knowledge mastering
4 Improve communication skills and
team spirit
From high to low Agreement degree for positive effect of PBL PBL has positive effects on self-study, team spirit,
1 Connect theoretical knowledge with (agree, neutral, disagree) communication, expression, and analyzing,
clinical practice but not for learning efficacy
2 Culture self-study ability
3 Culture team spirit
4 Culture systematic medical thinking
5 Improve learning efficacy
6 Culture communication skills and
expression ability
7 Improve problem analyzing and
solving ability
8 Improve self-study ability
NA NA NA
From high to low Percentage for positive effect of PBL (agree, PBL has positive effects on team spirit, self-study, and
1 Improve team spirit disagree) comprehensive ability
2 Improve literature search and read
ability
3 Improve self-study ability
4 Improve comprehensive analyzing
ability
(continued on next page)
53
L. Liu et al. Studies in Educational Evaluation 60 (2019) 43–58

Table 3 (continued)

Study ID Categories of statements Description levels Conclusion

Huo et al. (2016) From high to low Agreement degree for positive effect of PBL PBL has positive effects on scope of knowledge,
1 Master knowledge (agree, neutral, disagree) analyzing, team spirit, problem solving, and self-
2 Literature search ability study
3 Collaborative and team spirit
4 Problem analyzing and solving
ability
5 Self-study and personal development
ability
6 Communication skills and ability
Imran et al. (2015) NA NA NA
Jia (2013) From high to low Percentage for positive effect of PBL (agree, PBL has positive effects on self-study, motivation,
1 Improve self-study ability disagree) analyzing, and learning climate
2 Improve learning interest
3 Improve degree of satisfaction on
teacher
4 Improve depth of problem
understanding
5 Improve degree of satisfaction on
classroom climate
Li et al. (2011) From high to low Agreement degree for positive effect of PBL PBL has positive effects on self-study, scope of
1 Self-study ability (agree, neutral, disagree) knowledge, team spirit, and analyzing
2 Systematically master theoretical
knowledge ability
3 Team spirit
4 Summary ability
5 Ability of theory connecting with
reality
6 Problem discovering and solving
ability
Li et al. (2013) From high to low Agreement degree for positive effect of PBL PBL has positive effects on self-study, analyzing,
1 Culture self-study ability (agree, neutral, disagree) team spirit, scope of knowledge, and problem solving
2 Culture problem analyzing ability
3 Improve understanding and memory
ability
4 Improve team spirit
5 Improve literature search and read
ability
6 Improve knowledge mastering
7 Improve problem discovering and
solving ability
8 Ability of theory connecting with
reality
9 Improve learning efficacy
Li and Xi (2014) From high to low Percentage for positive effect of PBL (agree, PBL has positive effects on application knowledge,
1 Culture application knowledge disagree) self-study, expression, analyzing, and team spirit
ability
2 Improve self-study ability
3 Practice language expression ability
4 Culture comprehensive analyzing
ability
5 Improve literature search and read
ability
6 Culture team spirit
Liu (2014) NA NA NA
Luo (2012) From high to low Percentage for positive effect of PBL (agree, PBL has positive effects on self-study, motivation,
1 Self-study ability disagree) problem solving, and analyzing
2 Learning efficacy
3 Discover and solve problems ability
4 Improve literature search ability
5 Team spirit
6 Expression ability
7 Logical thinking ability
8 Scientific attitude and method
Miao et al. (2013) From high to low Agreement degree for positive effect of PBL PBL has positive effects on application knowledge,
1 Comprehensive utilizing knowledge (agree, neutral, disagree) scope of knowledge, problem solving, team spirit,
ability and self-study
2 Systematically master theoretical
knowledge ability
3 Ability of theory connecting with
reality
4 Summary ability
5 Team spirit
6 Self-study ability
(continued on next page)

54
L. Liu et al.
Table 3 (continued)
Study ID
Michel, Bischoff, & Jakobs
(2002); Michel, Bischoff, Zu
Heringdorf et al. (2002)
Miller (2003)
Pan (2014)
Song (2011)
Song et al. (2015)
Wang, Wu et al. (2008), Wang,
Wie et al. (2008)
Wang, Wu et al. (2008), Wang,
Wie et al. (2008)
Studies in Educational Evaluation 60 (2019) 43–58
Categories of statements Description levels Conclusion
From high to low Mean score, rated by the students on a scale PBL students rate generated interest in
1 Tutors are motivated and competent of 1–10 (1 statement does not describe my pharmacology, convey knowledge in pharmacology
2 The class increases knowledge in experience at all, 10 statement fully and understanding of medical questions
pharmacology describes my experience) approximately 1 point higher than lecture-based
3 The class enhances overall medical learning students
understanding
4 The class increases interest in
pharmacology
5 The class helps to become a good
physician
Original order Student satisfaction as measured by 8-item, No statistically significant difference between two
1 What extent personal learning goals 7-point Likert scale groups
reaches for this course?
2 What extent the course facilitates
meeting posted course objectives?
3 What extent the teacher helps to
reach learning goals?
4 What extent the teacher
demonstrates knowledge of the
subject matter covered by this
course?
5 What extent enjoys this course?
6 What extent the teacher helps create
a positive learning environment?
7 Overall, what extent is satisfied with
this course?
8 What extent feel that this teaching
method is the best way to teach
pharmacology?
From high to low Percentage for positive effect of PBL (agree, PBL has positive effects on communication, interest,
1 Professional knowledge disagree) self-study, scope of knowledge, and analyzing
communication between teacher and
students
2 Improve learning interest
3 Improve self-study ability
4 Improve knowledge understanding
and memory
5 Improve comprehensive analyzing
ability
From high to low Agreement degree for positive effect of PBL PBL has positive effects on self-study, team spirit,
1 Improve self-study ability (agree, neutral, disagree) communication, and expression
2 Improve team spirit
3 Improve communication skills
4 Improve expression ability
5 Practical ability
From high to low Agreement degree for positive effect of PBL PBL has positive effects on self-study, analyzing,
1 Improve self-study ability (agree, neutral, disagree) problem solving, communication, and team spirit
2 Improve literature search and read
ability
3 Improve summary ability
4 Culture problem discovering and
solving ability
5 Improve communication skills
6 Strength team spirit
From high to low Percentage for positive effect of PBL (agree, PBL has positive effects on scope of knowledge, team
1 Master knowledge disagree) spirit, problem solving, self-study, communication,
2 Collaboration and team spirit and analyzing
3 Problem discovering and solving
ability
4 Self-study and personal development
ability
5 Communication skills and ability
6 Literature search ability
From high to low Agreement degree for positive effect of PBL PBL has positive effects on problem solving, scope of
1 Problem solving with learned (agree, neutral, disagree) knowledge, interest, and expression
knowledge
2 Knowledge understanding and
memory
3 Degree of satisfaction on PBL method
4 Degree of preference class discussion
5 Improve thinking ability
6 Opinion on traditional teaching
method
(continued on next page)
55
L. Liu et al. Studies in Educational Evaluation 60 (2019) 43–58

Table 3 (continued)

Study ID Categories of statements Description levels Conclusion

Wei and Zheng (2013)
Xiang and Yang (2015)
Yang and Jin (2008)
Zhang et al. (2005)
Zhang, Zhang et al. (2015),
Zhang, Zhou et al. (2015)
Zhao et al. (2014)
Zhao et al. (2015)
Zhou et al. (2012)
From high to low Agreement score PBL has positive effects on interest, problem solving,
1 Improve teaching attitude teaching method and level
2 Improve learning interest
3 Culture clinical ability
4 Improve teaching method
5 Improve teaching level
From high to low Percentage for positive effect of PBL (agree, PBL has positive effects on self-study, analyzing,
1 Improve self-study ability disagree) scope of knowledge, analyzing, and learning climate
2 Improve comprehensive analyzing
skills
3 Improve study efficiency
4 Improve literature search ability
5 Active classroom climate
NA NA NA
From high to low Agreement degree for positive effect of PBL PBL has positive effects on self-study, scope of
1 Improve self-study ability (agree, neutral, disagree) knowledge, and problem solving
2 Improve learning efficacy
3 Improve problem analyzing and
solving ability
From high to low Percentage for positive effect of PBL (agree, PBL has positive effects on interest, application,
1 Improve learning interest disagree) communication, learning climate, self-study, and
2 Help to become a good physician problem solving
3 Improve communication between
teacher and students, active
classroom climate
4 Improve self-study, problem
analyzing and solving ability
From high to low Percentage for positive effect of PBL (agree, PBL has positive effects on self-study, application,
1 Improve self-study ability disagree) problem solving, analyzing, scope of knowledge,
2 Better connect knowledge with team spirit, communication, and expression
clinical practice
3 Improve problem analyzing and
solving ability
4 Improve literature search and read
ability
5 Expand knowledge scope
6 Culture team spirit
7 Culture communication skills ability
8 Culture language expression ability
9 Culture student’s courage to present
From high to low Agreement degree for positive effect of PBL PBL has positive effects on learning climate, interest,
1 Active classroom climate and (agree, neutral, disagree) team spirit, communication, expression, self-study,
communication and problem solving
2 Improve learning interest
3 Improve team spirit, communication
skills, language expression ability
4 Improve self-study ability
5 Improve comprehensive analyzing
and thinking ability
NA NA NA

(Mennin, Kalishman, Friedman, Pathak, & Snyder, 1996). Our meta-
analysis confirmed that PBL provides many benefits to pharmacology
education research compared with traditional methods.
In this meta-analysis, there is high heterogeneity, with I2 = 97%.
Thus, the random effects model was used to calculate the effect of PBL
on pharmacology education. There is a greater number of studies car-
ried out in China in the final analysis of this meta-analysis. The bias of
selected country and language will weaken the meta-analysis result and
affect future application of PBL in pharmacology. The educational le-
vels of included students and schools are different, which is one fun-
damental cause for the heterogeneity among those studies. The bias of
other characteristics, such as the different disciplines and lack of uni-
form PBL pedagogy and its examination, may be another source of the
high heterogeneity.
In this meta-analysis, the questionnaire results show that PBL im-
proves students’ outcome in learning interest. PBL students enjoy PBL-
based pharmacology considerably more than courses with a lecture-
based learning method, the same observation of PBL evaluation present
in preventive medicine and pharmacy (Ding et al., 2014; Zhou et al.,
2016). PBL mainly affects students’ focus on the application of knowl-
edge (Gijbels et al., 2005). Pharmacology is usually considered as a
paraclinical discipline that connects basic sciences to clinical sciences.
Pharmacology education for students generally faces at least two dif-
ficulties: learning how drugs are used before learning about the diseases
and learning detailed facts about classes and actions of many drugs in
addition to their mechanisms. Considering the current trend for curri-
culum revision of medical education, traditional lecture-based learning
has been criticized as lacking the ability to connect basic science with
clinical relevance (Lau, 2004). In the PBL model, students learn, with
teachers' facilitation, in a student-centered environment based on a self-
directed, clinically relevant and case-oriented approach, usually in a
small-group tutorial format. Previous studies have also shown that PBL
is more effective in improving students’ attitudes towards their learning
and that students are more satisfied with the PBL method (Shin & Kim,
2013). Although overall student time for PBL courses also seems to be
similar to that of traditional formats, student behavior appears to be
changed from predominantly passive learning during the course and a
short period of intensive self-study before the exam to a more steady

56
L. Liu et al.
and continuous learning throughout the course (Michel, Bischoff, &
Jakobs, 2002; Michel, Bischoff, Zu Heringdorf et al., 2002).
In this meta-analysis, all of the theoretical scores are examined
immediately after intervention with PBL. PBL has a time-based effect on
knowledge tests, since PBL only shows effects for delayed but not for
immediate knowledge tests (Dochy et al., 2003). Future research is
needed to evaluate higher levels of the cognitive domain, such as ap-
plication, analysis, and transfer. Valid assessment techniques and
longitudinal studies are needed to evaluate whether the effects of PBL
endure past training. Whether PBL is indeed beneficial for long-term
learning has been difficult to assess scientifically, largely because ap-
propriate tools are missing (Michel, Bischoff, & Jakobs, 2002; Michel,
Bischoff, Zu Heringdorf et al., 2002). PBL and traditional lecture groups
were similar with regard to knowledge obtained during their initial
training, but the PBL group scored higher on knowledge related to in-
formation that became available after their graduation (Shin, Haynes, &
Johnston, 1993). However, changes in perception and insight are
usually not manifested as measurable changes in behavior or perfor-
mance, and a number of educators have noted the weaknesses of
measuring overt behaviors to assess the deep learning that PBL pro-
motes (Albanese & Mitchell, 1993). Thus, researchers should consider
the use of instruments carefully when assessing the long-term results of
PBL (Belland, French, & Ertmer, 2009).
This study has several limitations. First, we did not include studies
published in languages other than English and Chinese, abstracts from
conference proceedings, or qualitative research. The inclusion of un-
published studies, such as theses and dissertations, may enable the in-
vestigation of publication bias. Second, those who attended the class
were enthusiastic and answered the questionnaire. Third, high hetero-
geneity exists among the 37 studies. Fourth, several studies do not re-
port the detailed essential study design information necessary to carry
out the assessment. Fifth, we only discussed the questionnaire results of
the studies that were included in the meta-analysis. Sixth, 50 articles
reported questionnaire without theoretical scores were excluded in this
meta-analysis and their questionnaire results were not discussed.
Seventh, analysis for moderator of PBL effect will be carried out when
more articles about PBL on pharmacology are available in the future.
Thus, owing to the above mentioned limitations, the present results
should be interpreted with caution, even though they offer encouraging
empirical support for PBL, contribute to evidence-based education, and
may inform the development of useful educational research tools in
pharmacology education.
Despite the above limitations, this study provides an empirically
derived outcome assessment that may prove useful to pharmacology
education science that seeks to develop a more evidence-based ap-
proach. This meta-analysis has shown that PBL methods in pharma-
cology education have positive effects on theoretical scores and ques-
tionnaires outcomes.
Conflict of interest
None.
Funding
There is no funding for this research.
References
Albanese, M. (2000). Problem-based learning: Why curricula are likely to show little
effect on knowledge and clinical skills. Medical Education, 34(9), 729–738. https://
doi.org/10.1046/j.1365-2923.2000.00753.x.
Albanese, M. A., & Mitchell, S. (1993). Problem-based learning: A review of literature on
its outcomes and implementation issues. Academic Medicine, 68(1), 52–81. https://
doi.org/10.1097/00001888-199301000-00012.
Antepohl, W., & Herzig, S. (1999). Problem-based learning versus lecture-based learning
in a course of basic pharmacology: A controlled, randomized study. Medical
Education, 33(2), 106–113. https://doi.org/10.1046/j.1365-2923.1999.00289.x.
57
Studies in Educational Evaluation 60 (2019) 43–58
Bao, M., Li, Y., & Jiang, J. (2008). Application of PBL integrating synthetic discussion in
pharmacology teaching. Chinese Journal of Medical Research, 8(1), 29–32. https://doi.
org/10.1142/9789814719810_0065.
Barrows, H. S. (1986). A taxonomy of problem-based learning methods. Medical
Education, 20(6), 481–486. https://doi.org/10.1111/j.1365-2923.1986.tb01386.x.
Barrows, H. S. (1996). Problem-based learning in medicine and beyond: A brief overview.
New Directions for Teaching and Learning, 1996(68), 3–12. https://doi.org/10.1002/tl.
37219966804.
Belland, B. R., French, B. F., & Ertmer, P. A. (2009). Validity and problem-based learning
research: A review of instruments used to assess intended learning outcomes.
Interdisciplinary Journal of Problem-based Learning, 3(1), 5. https://doi.org/10.7771/
1541-5015.1059.
Cao, B., Li, S., & He, H. (2011). The application of problem-based learning in the clinical
tumor pharmacology teaching. Chinese Journal of Medical Education, 31(3), https://
doi.org/10.3760/cma.j.issn.1673-677X.2011.03.031 405-406+428.
Chen, L., Wang, W., Song, Z., Liu, M., Wang, S., Wu, H., et al. (2012). Comparative study
of application of PBL+LBL teaching method in pharmacology between seven years
and five years of clinical medicine. China Higher Medical Education, 11. https://doi.
org/10.3969/j.issn.1002-1701.2012.11.006 8-9+19.
Collaboration, R. T. C. (2014). Review manager (RevMan). 5.3. Copenhagen: The Nordic
Cochrane Centre: The Cochrane Collaborationhttps://doi.org/10.1002/
9781444316315.
Cui, G., Zhong, P., He, X., & Zhang, S. (2011). Application of PBL teaching method in
pharmacology. China Healthcare Innovation, 6(16), https://doi.org/10.3969/j.issn.
1673-5552.2011.16.0061 90+85.
Des Marchais, J. E. (1999). New pedagogic approaches adapted to the teaching of
pharmacology, the activities centered on the student. Therapie, 54(1), 171–181.
https://doi.org/10.1080/10511970.2015.1025161.
Ding, X., Zhao, L., Chu, H., Tong, N., Ni, C., Hu, Z., et al. (2014). Assessing the effec-
tiveness of problem-based learning of preventive medicine education in China.
Scientific Reports, 4, 5126. https://doi.org/10.1038/srep05126.
Dochy, F., Segers, M., Van den Bossche, P., & Gijbels, D. (2003). Effects of problem-based
learning: A meta-analysis. Learning and Instruction, 13(5), 533–568. https://doi.org/
10.1016/S0959-4752(02)00025-7.
Egger, M., Davey Smith, G., Schneider, M., & Minder, C. (1997). Bias in meta-analysis
detected by a simple, graphical test. BMJ, 315(7109), 629–634. https://doi.org/10.
1136/bmj.315.7109.629.
Fu, Y., Yang, L., & Li, X. (2016). Teaching model of PBL combined with LBL used in
pharmacology for seven years students of Traditional Chinese Medicine. Journal of
Liaoning University of Traditional Chinese Medicine, 18(8), 118–120.
Galvao, T. F., Silva, M. T., Neiva, C. S., Ribeiro, L. M., & Pereira, M. G. (2014). Problem-
based learning in pharmaceutical education: A systematic review and meta-analysis.
Scientific World Journal, 2014(3), 578382. https://doi.org/10.1155/2014/578382.
Gao, J. (2015). Application of PBL combined with medical record discussion teaching
method in pharmacology teaching. Health Vocational Education, 33(1), 69–70.
Gao, X., Luo, S., Mu, D., Xiong, Y., Guanjian, L., & Wan, C. (2016). Effects of problem-
based learning in paediatric education in China: A meta-analysis. Journal of Evidence-
based Medicine. https://doi.org/10.1111/jebm.12190.
Gijbels, D., Dochy, F., Den Bossche, P. V., & Segers, M. (2005). Effects of problem-based
learning: A meta-analysis from the angle of assessment. International Journal of
Central Banking, 75(1), 27–61. https://doi.org/10.3102/00346543075001027.
Goodnough, K. (2006). Enhancing pedagogical content knowledge through self-study: An
exploration of problem-based learning. Teaching in Higher Education, 11(3), 301–318.
https://doi.org/10.1080/13562510600680715.
Guo, S., Wang, Y., & Li, Z. (2013). Application of PBL teaching method in “pharmacology”
teaching for vocational college nursing students. Chinese Nursing Research, 27(9B),
2935–2936. https://doi.org/10.3969/j.issn.1009-6493.2013.26.060.
Herzig, S., Linke, R. M., Marxen, B., Borner, U., & Antepohl, W. (2003). Long-term follow
up of factual knowledge after a single, randomised problem-based learning course.
BMC Medical Education, 3, 3. https://doi.org/10.1186/1472-6920-3-3.
Higgins, J. (2008). Green S. Cochrane handbook for systematic reviews of interventions ver-
sion 5.1. 0. https://doi.org/10.1002/9780470712184.ch20.
Higgins, J. P., & Thompson, S. G. (2002). Quantifying heterogeneity in a meta-analysis.
Statistics in Medicine, 21(11), 1539–1558. https://doi.org/10.1002/sim.1186.
Higgins, J. P., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003). Measuring incon-
sistency in meta-analyses. BMJ, 327(7414), 557–560. https://doi.org/10.1136/bmj.
327.7414.557.
Hmelo-Silver, C. E. (2004). Problem-based learning: What and how do students learn?
Educational Psychology Review, 16(3), 235–266. https://doi.org/10.1023/B:EDPR.
0000034022.16470.f3.
Hoffman, K., Hosokawa, M., Blake, R., Jr, Headrick, L., & Johnson, G. (2006). Problem-
based learning outcomes: Ten years of experience at the University of Missouri-
Columbia School of Medicine. Academic Medicine, 81(7), 617–625. https://doi.org/
10.1097/01.ACM.0000232411.97399.c6.
Huang, B., Zheng, L., Li, C., Li, L., & Yu, H. (2013). Effectiveness of problem-based
learning in Chinese dental education: A meta-analysis. Journal of Dental Education,
77(3), 377–383.
Huang, W. (2017). Application and evaluation of PBL teaching method in pharmacology
teaching of clinical medicine. Heilongjiang Science and Technology Information, 9, 57.
https://doi.org/10.3969/j.issn.1673-1328.2017.09.057.
Huo, Y., Yang, J., & Gong, Z. (2016). Exploration and practice of PBL in pharmacology
experiment teaching. The Journal of Medical Theory and Practice, 29(14), 1977–1978.
Imran, M., Shamsi, M. S., Singh, A., Goel, S., Sharma, P., & Panesar, S. (2015). Problem-
based learning versus lecture-based learning in pharmacology in a junior doctor
teaching program: A crossover study from northern India. International Journal of
Research in Medical Sciences, 3(11), 3296–3299. https://doi.org/10.28918/jupe.
v13i1.700.
Jia, Y. (2013). Application of PBL in the teaching of pharmacology in college. Guide of
China Medicine, 11(25), 288–289. https://doi.org/10.3969/j.issn.1671-8194.2013.
25.226.
L. Liu et al.
Jones, G. L., Walley, T., & Bligh, J. (1997). Integrating clinical pharmacology in a new
problem based medical undergraduate curriculum. British Journal of Clinical
Pharmacology, 43(1), 15–19. https://doi.org/10.1111/j.1365-2125.1997.tb00026.x.
Kaufman, D. M., & Mann, K. V. (1997). Basic sciences in problem-based learning and
conventional curricula: Students’ attitudes. Medical Education, 31(3), 177–180.
https://doi.org/10.1111/j.1365-2923.1997.tb02562.x.
Koh, G. C., Khoo, H. E., Wong, M. L., & Koh, D. (2008). The effects of problem-based
learning during medical school on physician competency: A systematic review.
CMAJ: Canadian Medical Association Journal, 178(1), 34–41. https://doi.org/10.
1503/cmaj.070565.
Kong, L. N., Qin, B., Zhou, Y. Q., Mou, S. Y., & Gao, H. M. (2014). The effectiveness of
problem-based learning on development of nursing students’ critical thinking: A
systematic review and meta-analysis. International Journal of Nursing Studies, 51(3),
458–469. https://doi.org/10.1016/j.ijnurstu.2013.06.009.
Kwan, C. Y. (2002). Problem-based learning and teaching of medical pharmacology.
Naunyn-Schmiedebergs Archives of Pharmacology, 366(1), 10–17. https://doi.org/10.
1007/s00210-002-0561-y.
Kwan, C. Y. (2004). Learning of medical pharmacology via innovation: A personal ex-
perience at McMaster and in Asia. Acta Pharmacologica Sinica, 25(9), 1186–1194.
https://doi.org/10.1504/IJIL.2003.003611.
Lau, Y. T. (2004). Problem-based learning in pharmacology: A survey of department
heads in Taiwan, China. Acta Pharmacologica Sinica, 25(9), 1238–1240. https://doi.
org/10.1097/00132586-198608000-00016.
Li, L., Chang, J., Zhou, Y., & Gao, Z. (2011). Application of problem based learning
teaching method in agricultural colleges of pharmacy in the experiment teaching of
pharmacology of traditional Chinese Medicine. China Medicine and Pharmacy, 1(24),
137–138.
Li, L., Jiang, H., Zhang, L., & Wang, C. (2013). The classroom design of veterinary
pharmacology with PBL-oriented teaching method. Education for Chinese After-school,
5. https://doi.org/10.3969/j.issn.1004-8502(x).2013.05.077 89+105.
Li, W., & Xi, H. (2014). The application and assessment of problem-based learning in
pharmacology teaching in higher vocational education. Science & Technology Vision
(15) 140+171.
Li, W. D., Zhang, Y., Zhang, C. L., & Zhang, X. M. (2004). Pharmacology teaching and its
reform in China. Acta Pharmacologica Sinica, 25(9), 1232–1237.
Liu, L. (2014). A comparative study of PBL teaching and traditional teaching for phar-
macology in vocational nursing. Journal of North Pharmacy, 11(2), 137.
Lloyd, H., Hinton, T., Bullock, S., Babey, A. M., Davis, E., Fernandes, L., et al. (2013). An
evaluation of pharmacology curricula in Australian science and health-related degree
programs. BMC Medical Education, 13, 153. https://doi.org/10.1186/1472-6920-13-
153.
Luo, X. (2012). Practice and exploration of PBL teaching method in pharmacology
teaching. Guide of China Medicine, 10(15), 682–683. https://doi.org/10.3969/j.issn.
1671-8194.2012.15.531.
Mennin, S. P., Kalishman, S., Friedman, M., Pathak, D., & Snyder, J. (1996). A survey of
graduates in practice from the University of New Mexico’s conventional and com-
munity-oriented, problem-based tracks. Academic Medicine, 71(10), 1079–1089.
https://doi.org/10.1097/00001888-199610000-00015.
Miao, C., Zhou, G., & Qin, M. (2013). Application of PBL teaching mode in the experi-
mental teaching of pharmacology in application-oriented colleges. Animal Husbandry
and Feed Science, 34(5), 84–85. https://doi.org/10.3969/j.issn.1672-5190.2013.05.
030.
Michel, M. C., Bischoff, A., & Jakobs, K. H. (2002). Comparison of problem- and lecture-
based pharmacology teaching 23. Trends in Pharmacological Sciences, 4, 168–170.
https://doi.org/10.1016/S0165-6147(00)01940-4.
Michel, M. C., Bischoff, A., Zu Heringdorf, M., Neumann, D., & Jakobs, K. H. (2002).
Problem- vs. Lecture-based pharmacology teaching in a German medical school.
Naunyn-Schmiedeberg's Archives of Pharmacology, 366(1), 64–68. https://doi.org/10.
1007/s00210-002-0570-x.
Miller, S. K. (2003). A comparison of student outcomes following problem-based learning
instruction versus traditional lecture learning in a graduate pharmacology course.
Journal of the American Academy of Nurse Practitioners, 15(12), 550–556. https://doi.
org/10.1111/j.1745-7599.2003.tb00347.x.
Nybo, L., & May, M. (2015). Effectiveness of inquiry-based learning in an undergraduate
exercise physiology course. Advances in Physiology Education, 39(2), 76–80. https://
doi.org/10.1152/advan.00161.2014.
Page, C. P., Sutter, M. C., & Walker, M. J. (1994). Wither, whether and whither phar-
macology. Trends in Pharmacological Sciences, 15(1), 17–19. https://doi.org/10.1016/
0165-6147(94)90128-7.
Pan, G. (2014). The exploratory application of problem-based learning in pharmacology
teaching. China Health Care Nutrition, 5, 3172–3173.
Rangachari, P. K. (1994a). Quality education for undergraduates in pharmacology: A
Canadian experiment. Trends in Pharmacological Sciences, 15(7), 211–214. https://
doi.org/10.1016/0165-6147(94)90313-1.
Rangachari, P. K. (1994b). The teaching of pharmacology: Needs, challenges and re-
sponses for the future. Trends in Pharmacological Sciences, 15(11), 399–402. https://
doi.org/10.1016/0165-6147(94)90081-7.
Schmidt, H. G., Molen, H. T. V. D., Winkel, W. W. R. T., & Wijnen, W. H. F. W. (2009).
Constructivist, problem-based learning does work: A meta-analysis of curricular
comparisons involving a single medical school. Educational Psychologist, 44(4),
227–249. https://doi.org/10.1080/00461520903213592.
Shah, N., Desai, C., Jorwekar, G., Badyal, D., & Singh, T. (2016). Competency-based
medical education: An overview and application in pharmacology. Indian Journal of
Pharmacology, 48(Suppl 1), S5–S9. https://doi.org/10.4103/0253-7613.193312.
Shankar, P. (2011). Ten basic competencies for undergraduate pharmacology education
at KIST medical college, Lalitpur, Nepal. The Australasian Medical Journal, 4(12),
677–682. https://doi.org/10.4066/AMJ.2011.1046.
58
Studies in Educational Evaluation 60 (2019) 43–58
Shankar, P. R., Dubey, A. K., Mishra, P., Upadhyay, D., Subish, P., & Deshpande, V. Y.
(2004). Student feedback on problem stimulated learning in pharmacology: A ques-
tionnaire based study. Pharmacy Education, 4(2), 51–56. https://doi.org/10.1080/
15602210410001727245.
Shin, I. S., & Kim, J. H. (2013). The effect of problem-based learning in nursing education:
A meta-analysis. Advances in Health Sciences Education: Theory and Practice, 18(5),
1103–1120. https://doi.org/10.1007/s10459-012-9436-2.
Shin, J. H., Haynes, R. B., & Johnston, M. E. (1993). Effect of problem-based, self-directed
undergraduate education on life-long learning. Canadian Medical Association Journal,
148(6), 969–976. https://doi.org/10.1108/S2051-2295(2013)0000000014.
Sim, S. M., Achike, F. I., & Geh, S. L. (2005). Many roads lead to Rome–is that true of the
teaching and learning of pharmacology? Lessons from three medical schools in
Malaysia. The Medical Journal of Malaysia, (60 Suppl D), 41–47.
Sivam, S. P., Iatridis, P. G., & Vaughn, S. (1995). Integration of pharmacology into a
problem-based learning curriculum for medical students. Medical Education, 29(4),
289–296. https://doi.org/10.1111/j.1365-2923.1995.tb02851.x.
Smits, P. B., de Buisonje, C. D., Verbeek, J. H., van Dijk, F. J., Metz, J. C., & ten Cate, O. J.
(2003). Problem-based learning versus lecture-based learning in postgraduate med-
ical education. Scandinavian Journal of Work, Environment & Health, 29(4), 280–287.
https://doi.org/10.1111/j.1365-2929.2004.01846.x.
Song, H., Huang, H., Cai, X., Zeng, R., & Wang, Z. (2015). Practice of PBL teaching mode
in the teaching of pharmacology. Journal of Higher Education Research, 38(3),
113–117. https://doi.org/10.3969/j.issn.1672-8874.2015.03.021.
Song, X. (2011). Exploration of problem based learning in the experimental teaching of
pharmacology. Chinese Medicine Modern Distance Education of China, 9(11), 82–83.
https://doi.org/10.3969/j.issn.1672-2779.2011.11.055.
Tayyeb, R. (2013). Effectiveness of problem based learning as an instructional tool for
acquisition of content knowledge and promotion of critical thinking among medical
students. Journal of the College of Physicians and Surgeons–Pakistan : JCPSP, 23(1),
42–46 https://doi.org/01.2013/JCPSP.4246.
Tisonova, J., Hudec, R., Szalayova, A., Bozekova, L., Wawruch, M., Lassanova, M., et al.
(2005). Experience with problem oriented teaching in pharmacology. Bratislavske
Lekarske Listy, 106(2), 83–87. https://doi.org/10.1159/000288352.
Vernon, D. T., & Blake, R. L. (1993). Does problem-based learning work? A meta-analysis
of evaluative research. Academic Medicine, 68(7), 550–563. https://doi.org/10.1097/
00001888-199307000-00015.
Vestal, R. E., & Benowitz, N. L. (1992). Workshop on problem-based learning as a method
for teaching clinical pharmacology and therapeutics in medical school. Journal of
Clinical Pharmacology, 32(9), 779–797. https://doi.org/10.1002/j.1552-4604.1992.
tb03884.x.
Wang, J., Xu, Y., Liu, X., Xiong, W., Xie, J., & Zhao, J. (2016). Assessing the effectiveness
of problem-based learning in physical diagnostics education in China: A meta-ana-
lysis. Scientific Reports, 6, 36279. https://doi.org/10.1038/srep36279.
Wang, S., Wu, Y., Wang, J., & Qian, X. (2008). Application of problem based learning
teaching method in pharmacology. Chinese Journal of Medical Education, 28(2),
58–59. https://doi.org/10.3760/cma.j.issn.1673-677X.2008.02.022.
Wang, W., Wei, M., Liu, M., He, M., Wu, H., Song, Z., et al. (2008). Exploration and
practice PBL bilingual teaching for seven years medical students in pharmacology.
Chinese Higher Medical Education, 7. https://doi.org/10.3969/j.issn.1002-1701.2008.
07.044 89-90+96.
Wei, G., & Zheng, Q. (2013). Exploration on application of problem-based learning in-
teractive teaching method in clinical pharmacology teaching. Chinese Journal of
Clinical Pharmacology, 29(9), https://doi.org/10.3969/j.issn.1001-6821.2013.09.020
704-705+709.
Xiang, S., & Yang, Y. (2015). Analysis on the effect of PBL teaching method applied in
pharmacology teaching of nursing specialty in higher vocational colleges. Hubei
Today, 6, 125–126.
Yang, C., & Jin, W. (2008). The application of problem-based learning in the teaching of
nursing pharmacology. Chinese Medicine Guides, 5(28), 82–83. https://doi.org/10.
3969/j.issn.1673-7210.2008.28.054.
Yaqinuddin, A. (2013). Problem-based learning as an instructional method. Journal of the
College of Physicians and Surgeons–Pakistan : JCPSP, 23(1), 83–85 https://doi.org/
01.2013/JCPSP.8385.
Zhang, C., Zhang, H., Niu, B., Zhang, L., Song, Y., & Yin, Z. (2015). Application of PBL
teaching method centered on case discussion in pharmacology teaching. China Higher
Medical Education, 6, 104–105. https://doi.org/10.3969/j.issn.1002-1701.2015.06.
054.
Zhang, H., Zhou, L., & Shen, C. (2005). Application of problem-based learning in phar-
macology teaching. Northwest Medical Education, 13(4), 415–416. https://doi.org/10.
3969/j.issn.1006-2769.2005.04.041.
Zhang, Y., Zhou, L., Liu, X., Liu, L., Wu, Y., Zhao, Z., et al. (2015). The effectiveness of the
problem-based learning teaching model for use in introductory Chinese under-
graduate medical courses: A systematic review and meta-analysis. PloS One, 10(3),
https://doi.org/10.1371/journal.pone.0120884 e0120884.
Zhao, R., Liu, C., Yan, D., Sun, P., & Zhang, T. (2015). Application effect of problem-based
learning in the teaching process of Traditional Chinese Medicine pharmacology.
China Higher Medical Education, 4, 51–52. https://doi.org/10.3969/j.issn.1002-1701.
2015.04.027.
Zhao, W., Lan, X., Zhu, K., & Gong, Y. (2014). Effect analysis of pharmacology PBL
teaching. Chinese Medicine Guides, 11(7) 135-137+141.
Zhou, J., Zhou, S., Huang, C., Xu, R., Zhang, Z., Zeng, S., et al. (2016). Effectiveness of
problem-based learning in Chinese pharmacy education: A meta-analysis. BMC
Medical Education, 16(1), 23. https://doi.org/10.1186/s12909-016-0546-z.
Zhou, L., Luo, S., Qin, D., & Cui, L. (2012). Application of PBL teaching method in the
course of clinical pharmacology case discussion. China Higher Medical Education, 11,
13–14. https://doi.org/10.3969/j.issn.1002-1701.2012.11.009.