Research papers

The role of basic sciences in a problem-based learning clinical curriculum

P A O'Neill

Background Very little is known about the use of problem-based learning (PBL) during the later years of the undergraduate medical course and how it inuences further acquisition of basic science knowledge. Similarly to many other Faculties, the PBL approach is used at Manchester in years 1 and 2, but more unusually, the curriculum continues to be centred on PBL in the clinical modules. Objectives To explore whether and how basic science learning was continued in year 3 of the PBL clinical curriculum. Methods 10 of the weekly problems from the two core modules in year 3 were analysed to determine: (a) whether the design teams were using basic science objectives in devising the problems, and (b) whether PBL student groups were setting basic science learning objectives. The basic science knowledge of year 3 and 4 students was also measured. Results Similar numbers of objectives were being set by the management groups for each weekly problem (Heart, lung and blood (HLB) module, median 15, range 1120; Nutrition, metabolism and excretion (NME) module, median 13, range 921). In the basic sciences, there was a median of 3 objectives per problem (range 06) in the NME module, but only 1

objective (02) per problem in the HLB module. The objectives set by six PBL groups in each module were analysed. Overall, agreement was reached on 130 occasions (62%) between the design team basic science objectives and those set for themselves by the student groups. In addition, there was a median of 2 (range 18) new basic science objectives brought out by the PBL groups that were not listed by the HLB module design team. In the NME module, there was again a median of 2 new objectives (range 06). The performance of year 3 and year 4 students in the multiple-choice questions progress test was analysed. For the 65 basic science questions, the year 3 mark was 408 123% compared with 571 123% for year 4 (P < 00001). Conclusions (a) The design teams are setting basic science objectives; (b) the working problems are triggering students to set learning objectives in the basic sciences; (c) most of the objectives being set by the design teams are being triggered in the majority of group sessions; (d) the students knowledge of basic sciences increases in years 34. Keywords *Curriculum; *education, medical, undergraduate; problem-based learning. Medical Education 2000;34:608613

Introduction
The new Manchester curriculum, which started in 1994, is integrated and uses problem-based learning in all the core modules in years 14. From year 3 onwards, the students' base is moved from the medical school to one of three teaching hospital sectors. This extensive use of PBL within a clinical environment is unusual.1 In most PBL curricula, the method is conned to the pre-clinical course. Through this approach, clinically
Correspondence: P A O'Neill, Faculty of Medicine, Dentistry and Nursing, University of Manchester, Oxford Road, Manchester M13 9PT, UK

relevant problems are designed to be stimulating to discuss and as vehicles for basic science learning. One potential disadvantage is that students may become more interested in the clinical aspects of a problem and neglect the underlying basic science knowledge, though this has not been formally reported. Another drawback may be that the students, through studying diagnostic problems, do not acquire an appropriate framework for the continued learning of basic science. Nevertheless, in a fully integrated curriculum, students should go on learning about basic science as they progress through the course, though the emphasis may lessen.2,3 In practice, there has been ongoing concern about the role of basic science within the undergraduate

608

Blackwell Science Ltd ME D I C A L ED U C A T I ON 2000;34:608613

Basic sciences and clinical PBL

P A O'Neill Table 1 The design of the curriculum for years 15 Year 1: Year 2 Year 3 Nutrition and metabolism Cardiorespiratory tness Abilities and disabilities Life cycle Basic skills course Main module: Nutrition, metabolism and excretion (NME) Special study module in NME Main module: Heart, lungs and blood (HLB) Special study module in HLB Main module: Families and children (FC) Special study module in FC Main module: Cognition, special senses and locomotion (CSSL) Special study module in CSSL `Options' (research) Elective period Hospital placement Hospital placement Community placement Consolidation period

609

curriculum, with conferences3 and editorials4 stressing that it should not be allowed to decline. In a review of the experience in North American schools, Schmidt5 found that it was much harder to integrate basic science into the clinical curriculum than the converse. Others have also acknowledged these barriers.3 The Manchester PBL curriculum was designed to cover a core of knowledge over years 14 through the use of PBL. This gave the opportunity to evaluate whether basic science learning was continuing as students reached the later years of the course in which they were still using PBL. A series of questions were set in order to examine this. 1 How are basic science themes such as anatomy represented in the objectives for the PBL cases through years 14? 2 Did the multidisciplinary module management groups for year 3 use basic science objectives in designing the PBL cases? 3 In year 3, do the PBL cases trigger basic science learning objectives in the student groups, and are the objectives the same as the ones intended by the module management group? 4 How does the students' knowledge of basic science change from year 3 to year 4?

4 weeks 14 weeks 4 weeks 14 weeks 4 weeks 14 weeks 3 weeks 14 weeks 3 weeks 11 weeks 8 8 8 8 8 weeks weeks weeks weeks weeks

Year 4

Year 5

Brief description of course

The design of the curriculum and the implementation of PBL have been described in detail elsewhere.6,7 The courses in years 14 are divided into core plus special study modules, with the core themes from years 1 and 2 being revisited and developed in years 3 and 4 (Table 1). The overarching themes for year 3 are `Heart, lung and blood' (HLB) and `Nutrition, metabolism and excretion' (NME). A multidisciplinary group manages each module and they are responsible for setting the objectives for the module and the design of the working problems. Each of the core modules in year 3 is 14 weeks in length with one working problem discussed each week. These problems are built around objectives derived from a number of `index clinical situations' which form the core knowledge and skills of our curriculum.8 The working problems are much broader than any single disease; being designed to integrate across behavioural, clinical, pathological and basic sciences, clinical epidemiology and public health. Our students do not have ready access to case objectives set by the design teams, which is unlike the practice at other schools.9

Study setting
Groups of eight students and a tutor meet twice in 7 days; initially for 1 h to discuss the problem and then for 15 h to discuss what they have found out. The latter session is longer to allow the students more time to make the connections to similar patients they have seen. In years 3 and 4, students have long attachments (7 weeks) to relevant clinical rms during the core modules (see Table 1). They also spend 1 day per week in the community under the guidance of a general practitioner tutor. The principal difference from the rst 2 years is that students are constantly exposed to clinical situations and interactions with clinicians and professions allied to medicine. Outside of the tutorials, the students are acquiring clinical skills and gaining ad hoc clinical experience as well as trying to meet the learning goals set in the rst PBL group session. The students can meet their learning objectives by several means. In year 3, the teaching hospitals provide resources week to week that support the working problem. These may include seminars/workshops/lectures, articles, anatomical models, pathological material, radiological images, posters and IT access.

Blackwell Science Ltd ME D I C AL ED U C AT I ON 2000;34:608613

610

Basic sciences and clinical PBL

P A O'Neill

In years 3 and 4, summative assessments have been kept to a minimum. We have introduced the `Progress test' for summative assessment of knowledge.10 This is a multiple-choice examination which encompasses the core knowledge that the student should have acquired by the time of graduation. This form of assessment is introduced in year 3; prior to this the students have integrated end-of-semester exams. For the Progress test, every student (years 35) sits a paper twice per year, which is the same for all students. Each question is linked to an index clinical situation and is `tagged' according to the knowledge it is testing. Thus, the questions in an exam paper that are linked to basic sciences could be readily identied.

3. Are basic science objectives being triggered in the PBL groups?

Methods
The approach used to answering the four questions set out in the introduction was as follows.
1. Basic science themes

The records of the group discussion of six groups from the NME module and six groups from the HLB module were examined. The records covered the same 10 problems as those covered in the analysis of each of the module management groups (i.e. 120 group discussions were examined). The author compared the learning objectives from the group discussion with those used by the module management groups, to determine whether any basic science objectives were set. If this was the case, then the objectives were categorized as being either (a) similar to those intended by the design teams, or (b) different from those of the design teams. Objectives were categorized as being in agreement when broadly interchangeable words appeared (e.g. `Revise the normal cardiac anatomy' was classied as being in agreement with `Learn about the structure of the heart').
4. Growth of basic science knowledge

Anatomy was chosen as the core theme for evaluation as this is the basic science which most often gives rise to concern about its continued place in any undergraduate curriculum. The core database on the Faculty website (http://www.medicine.man.ac.uk) holds all the objectives underlying the PBL cases for years 14. Each objective is coded according to the general theme (e.g. physiology) and detailed objective (e.g. control of gastric acid production). Using the database, the author recorded the number of coded anatomy objectives for each year of the course.
2. Are basic science objectives used in designing the PBL cases for year 3?

The last part of the evaluation was of how the students' knowledge of basic science changed from year 3 to year 4. The results from the basic science questions in the Progress test (mean and standard deviation) were calculated for each year cohort taking the same exam. In this assessment, 65 of the 250 questions were specically tagged as testing basic science knowledge. Comparisons were made using unpaired Student's t tests.

Results
Representation of basic science themes

The objectives used by the module design teams for the year 3 core modules (HLB and NME) were examined. The number of objectives was recorded and also divided into knowledge and skills. Prior to the analysis, a list of key words was made up to determine the categorization of an objective and the subsequent agreement between the design teams and the PBL groups (e.g. structure, anatomy). If the objective could not be coded according to the predetermined list, it was ignored for the study. The problems from weeks 110 were analysed; weeks 1114 were excluded because of the possible interference with the group discussion caused by the proximity of examinations.

A total of 71 anatomy/structural objectives were used by the module management groups over years 14. In year 1, 21 anatomical objectives underpin the PBL cases, compared with 20 in year 2, 17 in year 3 (divided over the 28 cases in the two core modules) and 13 anatomical objectives in year 4.
Use of basic science objectives in designing the PBL cases for year 3

A similar number of objectives were being set by the management group for each problem (HLB, median 15, range 1120; NME, median 13, range 921). In the basic sciences, there was a median of three objectives per problem (range 06) in the NME module, but a median of only one objective (02) in the HLB module. Over the 20 problems, six did not use any objectives

Blackwell Science Ltd ME D I C A L ED U C A T I ON 2000;34:608613

Basic sciences and clinical PBL

P A O'Neill

611

from the basic sciences in their design; ve of these were from the HLB module. In total, there were 35 basic science objectives for these 20 problems.
Triggering of basic science objectives in the PBL groups

Given that six PBL groups discussed each case with its underlying basic science objectives, then the PBL groups could be in agreement with the 35 objectives set by the module design team on 210 occasions (6 35). With perfect agreement, a single objective would be identied by all six PBL groups, which happened on ve occasions. Conversely, on one occasion, no group brought out in the records of their discussions the basic science objective that was intended by the design team. Overall, agreement was reached on 130 occasions (62%). There was a median of 2 (range 18) new basic science objectives brought out by the PBL groups that could not be categorized as being the same as those listed by the HLB design module team. In the NME module, there was again a median of 2 new objectives (range 06). In total, 54 new basic science objectives were generated by the student groups.
Growth of basic science knowledge from year 3 to year 4

In the Progress test, the total mark (from 250 multiple choice questions) for year 3 was 310 86% compared with 455 85% for year 4. For the 65 basic science questions, the year 3 mark was 408 123% compared with 571 123% for year 4 (P < 00001).

Discussion
In an integrated curriculum using PBL in a clinical environment, we found that there were basic science objectives in the dened core content for years 3 and 4, and such objectives were underlying most of the cases designed by the module design teams for year 3. These objectives were brought out by the majority of the student groups who also generated a greater number of new objectives. There was also evidence of a growth in basic science knowledge between years 3 and 4. There are some drawbacks to the study design. The groups studied were based in one teaching hospital, but this should not have resulted in any bias. The analysis was carried out by the author using judgement on what constituted a basic science objective and whether it was the same in the student group discussion as that set by

the module design team. Others have used a more rigorous method to determine agreement between raters11 and reduce possible bias. However, the author used a list of key words drawn up prior to the study to categorize an objective and discarded any objective in which these were not included. In addition, the aim was to look for general agreement rather than precise wording of a complex objective. A further drawback is that when a student group lists a learning objective, this is simply a statement of intent to study a particular area, it does not mean that any work was done. It is likely, though, that learning did take place, given the improvement in the basic science scores in the Progress test, even if only measured over years 3 and 4. Our educational approach is similar to that of Dolmans et al.,12 who suggested that basic science concepts should be presented in the context of a clinical problem, to encourage integration of knowledge. Although these authors were drawing on experience of designing cases for use in a pre-clinical course, the same principle holds for encouraging basic science learning within a clinical environment. In addition, we also adhere to another of their principles in that a case should match one or more of the Faculty learning objectives.12 In Manchester, we want the students to cover the core content,8 including basic science knowledge and understanding, over years 14. Consequently, the rst step in our analysis was to determine whether the design teams (on behalf of the Faculty) were setting basic science objectives. We found some imbalance between the two modules which we may want to correct in future, but it may be that the cases were devised to achieve other, equally important Faculty objectives. For example, in the HLB module, one case is centred on somatization and non-specic chest pain. In this, it is more important to bring out the psychological issues rather than contrive to include basic science. There was concordance between the Faculty and the student objectives in 62% of the possible matches, which was very close to the 64% reported by Dolmans et al.11 in a study of year 2 students. Mpofu et al.13 found a much higher agreement, but their study was on broad themes rather than specic objectives. We also found that the student groups generated a greater number of new basic science objectives compared with the total set by Faculty. The median values were similar in the two modules, but if a greater numbers of cases had been studied, it might be that the HLB module, which contained fewer Faculty basic science objectives, might have been shown to generate more new student objectives. This would also be dependent on whether the design team

Blackwell Science Ltd ME D I C AL ED U C AT I ON 2000;34:608613

612

Basic sciences and clinical PBL

P A O'Neill

was not explicitly listing basic science objectives even though they intended the students to bring these out in the PBL tutorials. It would also be interesting to look at the objectives generated by student groups in year 4 of the course, as, by this stage, the objectives may have become much more focused on clinical issues rather than the basic science mechanisms underlying the case. Other than the case design, another important variable in the effectiveness of student discussion is the expertise of the tutors. Most of our tutors were consultants in the National Health Service rather than academic faculty.6 Given that we have just introduced PBL into a clinical environment, nearly all the tutors were inexperienced in the process of PBL. However, in relation to content expertise, many would have had reasonable knowledge of basic science (e.g. radiologists, pathologists, haematologists, and biochemists). In the literature, there is debate about the relative importance of process vs. content expertise of tutors,1416 though Regehr et al.15 reported that, when using qualied doctors as tutors, no differences between groups were found. In Manchester, given further experience and staff development, it is hoped that our tutors will become more adept at bringing out the objectives that the case is designed to trigger. Regardless of the generation of Faculty or student objectives, important outcomes are whether our students' knowledge and understanding of basic science continues to grow as they proceed through the course. It has been reported that students' knowledge does increase as they go into the clinical clerkships17 and that at the end of a PBL course there is little difference in the level of basic science knowledge of students compared with those completing a traditional course.18 In Manchester, the Progress test results showed that the students' basic science knowledge signicantly increased from year 3 to year 4. However, the test, as it is currently used, simply examines recall of factual information. We are currently looking to develop the test to assess the application of knowledge to problems. Our curriculum continues to develop and the feedback from evaluation is an important part of this process. As part of this, the results of this study have been fed back to the design teams. Year 3 of the new course has run three times, and with each successive cycle we have rened the cases on the basis of extensive student and tutor feedback. A further issue is how to make suitable basic science resources (written material, workshops, seminars, practicals, computer-assisted learning packages, etc.) available to all our students when most are based at least 3 miles from the medical

school in year 3 and, in year 4 (and 5), much further aeld. This is the next stage in the integration of basic sciences throughout the course.

Acknowledgements
I would like to thank Dr Pat McArdle for all her help and support with this project. I would also like to express my gratitude to the Faculty of the Harvard Macy Program which supported the development of the work.

References
1 Foley RP, Polson AL, Vance JM. Review of the literature on PBL in the clinical setting. Teaching Learning Med 1997;9:49. 2 Lowry S. Medical Education. London: BMJ Publishing; 1993. 3 Anderson J. The continuum of medical education. J R Coll Physicians Lond 1993;27:4057. 4 Beaty HN. Changes in medical education should not ignore the basic sciences. Acad Med 1990;65:6756. 5 Schmidt H. Integrating the teaching of basic sciences, clinical sciences and biopsychosocial issues. Acad Med 1998;73:S24 S31. 6 O'Neill PA. Problem-based learning alongside clinical experience: reform of the Manchester curriculum. Educ Health 1998;11:3748. 7 Whitehouse CR. Planning for community-orientated medical education in Manchester. Educ Health 1996;9:4559. 8 O'Neill PA, David TJ, Metcalfe D. The core content of the undergraduate curriculum in Manchester. Med Educ 1999;33:1219. 9 Blumberg P, Michael JA, Zeitz H. Roles of student-generated learning issues in problem-based learning. Teaching Learning Med 1990;2:14954. 10 Boshuizen HPA, Van der Vleuten CPM, Schmidt HG, Machiels-Bongaerts M. Measuring knowledge and clinical reasoning skills in a problem-based curriculum. Med Educ 1997;31:11521. 11 Dolmans DHJM, Gijselaers WH, Schmidt HG, Van Der Meer SB. Problem effectiveness in a course using problem-based learning. Acad Med 1993;68:20713. 12 Dolmans DHJM, Snellen-Balendong H, Wolfhagen IHAP, Van Der Vleuten CMP. Seven principles of effective case design for a problem-based curriculum. Med Teacher 1997;19:1859. 13 Mpofu DJS, Das M, Murdoch JC, Lanphear JH. Effectiveness of problems used in problem-based learning. Med Educ 1997;31:3304. 14 Eagle CJ, Harasym PH, Mandin H. Effects of tutors with case expertise on problem-based learning issues. Acad Med 1992;67:4659. 15 Regehr G, Martin J, Hutchinson C, Murnaghan J, Cuisamano M, Reznick R. The effect of tutors' content expertise on student learning, group process, and participant satisfaction in a problem-based learning curriculum. Teaching Learning Med 1995;7:22532.

Blackwell Science Ltd ME D I C A L ED U C A T I ON 2000;34:608613

Basic sciences and clinical PBL

P A O'Neill

613 18 Verhoeven BH, Verwijnen GM, Scherebier AJJA, Holdrinet RSG, Oeseburg B, Bulte JA, et al. An analysis of progress test results of PBL and non-PBL students. Med Teacher 1998;20:31016. Received 12 January 1999; editorial comments to authors 1 March 1999; accepted for publication 13 August 1999

16 Silver M, Wilkerson L. Effects of tutors with subject expertise on the problem-based tutorial process. Acad Med 1991;66:298300. 17 Glew RH, Ripkey DR, Swanson DB. Relationship between students' performance on the NBME comprehensive basic science examination and the USMLE Step 1: a longitudinal investigation at one school. Acad Med 1997;72:1097102.

Blackwell Science Ltd ME D I C AL ED U C AT I ON 2000;34:608613