Score Card Endoscopy: A Multicenter Study

to Evaluate Learning Curves in 1-Week Courses

Using the Erlangen Endo-Trainer

by a self-assessment. In addition, on days 1, 3, and 5 the test run

for each beginner was recorded on video, with each video assigned an encrypted code number. All the end of the study
week, control assessments of these videos were carried out by
an experienced endoscopist.
Results: Both the self-assessments and the control assessments
showed significant improvements in the endoscopic parameters
tested during the course (days 1 5; all parameters P < 0.001,
Wilcoxon-test). However, it was found that the trainees tended
to give themselves better marks than the marks given by experienced endoscopists.
Conclusion: During 1 week of training, using the model and following the score card, a significant improvement in the learning
curve was achieved in the beginners group for the individual
steps involved in diagnostic upper gastrointestinal endoscopy.
When this approach is used with trainees who are also provided
with the necessary theoretical background, this type of preparation may lead to a better, lower-risk start to supervised practical
endoscopic examinations in patients.

Introduction

and repeat examinations. Discussions in this area have been concerned with how to integrate simulation systems sensibly and
effectively into clinical educational structures in order to shorten
the learning period for assistants and reduce potential risks for
patients.

Simulation models have become established in various fields of

medicine. In the field of endoscopy, interest in using this method
of training and learning new techniques has also been growing
with the development of plastic models, biosimulation models,
and computer models [1 6]. Several methods of training in endoscopy are now available. Particularly, at the start of the learning curve, training with simulation systems can help protect patients from mistakes, complications, long examination times,

On the basis of an approach known as systematic surgical training following evaluable criteria of quality, control, which was
initially presented by Neumann et al. for surgical procedures in
September 1997 [7], an evaluation system for assessing individ-

Institution
Department of Surgery, University of Erlangen-Nuremberg, Germany
2
Department of Internal Medicine, Central City Hospital, University of Munich, Germany
3
Department of Internal Medicine II, Technical University of Munich, Germany
1

Corresponding Author
M. Neumann, M.D. Department of Surgery University of ErlangenNuremberg Krankenhausstrasse 12
91054 Erlangen Germany Fax: + 49-9131-8536328 E-mail: Martin.Neumann@chir.imed.uni-erlangen.de
Submitted 13 April 2002 Accepted after Revision 16 January 2003
Bibliography
Endoscopy 2003; 35 (6): 515520 C Georg Thieme Verlag Stuttgart New York ISSN 0013-726X

Original Article

Background and Study Aims: The present study was carried out
in the context of current discussions concerning ways in which
simulation systems can be integrated sensibly and effectively
into clinical educational structures, in order to shorten the training period for assistants and reduce potential risks for the patient. In our study, a number of centers used a standardized
training approach, in 1 week courses, to investigate the learning
curve improvement that can be achieved with a group of beginners in upper gastroduodenal endoscopy.
Materials and Methods: The multicenter study used the Erlangen Endo-Trainer, with specially prepared biological specimens
from pigs. Using this, the individual steps of diagnostic upper
gastrointestinal endoscopy with biopsy can be carried out following a score-card system. After a theoretical introduction and
a demonstration of the examination by an experienced endoscopist, an initial evaluation score for each participant was obtained
on day 1. On the following days, the program consisted of 2
hours training by a tutor, followed by a test run for each participant. On days 1, 2, 3, 4, and 5 the test run was directly followed

515

Downloaded by: University of Queensland. Copyrighted material.

M. Neumann 1
C. Hahn 1
T. Horbach 1
I. Schneider 1
A. Meining 2
W. Heldwein 2
T. R(sch 3
W. Hohenberger 1

ual performance, progress, and potential for improvement was

developed for gastrointestinal endoscopy, using a biosimulation
model (the Erlangen Endo-Trainer). This system was called
score-card endoscopy. Based on experience gained at numerous
workshops, an attempt was made to sequence the various steps
and components involved in upper gastrointestinal endoscopy,
both the basic skills such as spatial orientation, tactile ability, instrument handling, and flexibility in approaching the object, and
the more specific skills used in therapeutic endoscopy [8].

The aims of this study were:

1. To record the course of the learning curve in beginners in endoscopy, at various centers, during the course of 1 study week,
with daily training units following a structured training program in standardized conditions.
2. To compare the beginners self-assessments with control assessments carried out by an experienced endoscopist.

Materials and Methods

During the study weeks, a training suite was prepared at each of
the endoscopy centers, provided with the Erlangen trainer model, the prepared biological specimen (pig stomach), a video endoscope, a light source, a suction pump, and a video recorder to
tape the trainees performance (Figure 1). An assistant was available for the biopsy. Videotapes and score-card protocols for four
beginners and for the experts were also prepared, and were assigned a code number for each participant.
The Erlangen Endo-Trainer and Training Specimens
The bio-simulation model used (the Erlangen Endo-Trainer) consists of an anatomically simulated torso and head. The torso can
be rotated around the longitudinal axis via two special fittings,
and can be locked in any desired side position.
Inside the torso, specially prepared, cleaned pig stomach is
placed in an anatomically correct position. The connection between the animal tissue and the plastic part of the model (the
head with a mouth, and a pharynx, and larynx, including the
pharyngeal recess, vocal cords and a tracheal orifice) was made

Figure 1 Workstation set up for assessment of upper gastrointestinal

endoscopy, using the score card, with the Erlangen Endo-Trainer, a biological specimen (pig stomach), a video endoscope and a video recorder to tape the trainees performance.

at a level slightly below the larynx. In a modified version for

training in basic techniques, the model is equipped with a transparent top in order to track the illuminated endoscope tip and
endoscope position during the training runs, but this was not
used during the assessment runs. All the steps involved in a
standard diagnostic upper gastrointestinal endoscopy in real
biological tissue can be performed using a special preparation of
pig stomach, consisting of the esophagus, stomach, and duodenum. After the preparation has been cleaned and cut to the appropriate size, it is turned inside out and marked with 15 pointed
methylene blue markers. The markers are target points for the
forceps biopsies. Polyps are created by ligating pieces of mucosa.
In addition, areas of mucous membrane (e. g. in the angulus area,
the gastroduodenal passage, and at the esophagogastric junction) are marked using the diathermy machine to simulate ulcers
for biopsy (Figure 2).
The biological specimens are positioned and fixed in a realistic
position on a preformed plate in the interior of the model.
When the endoscope is introduced into the biologically elastic
esophagus, the entire hollow viscus remains airtight, so that insufflation via the endoscope results in pneumatic distension,
providing the usual good view inside the stomach and the duodenum (Figure 3). Care was taken to use specimens that were
similar in terms of size and fixation onto the internal preformed
plate (e. g. by creating a similar angle at the angulus), so that all
the examinations could be carried out under uniform conditions.
As long as all of the features provided (e. g. a prepared antral ulcer, a fundic polyp, and a blue marking in the antral area for biopsy) were clearly visible in the training specimen, the specimen
was not changed between trainees. On most of the study days,
one prepared pig stomach was sufficient. On the morning of
each day, one stomach was allowed to thaw for 4 5 hours and
was then ready for use.
Components of the Score Card
The basis for training and evaluation during the study weeks was
a score card that was developed to describe the various steps of
upper gastrointestinal endoscopy, and which had been endorsed

Neumann M et al. One-Week Courses Using the Erlangen Endo-Trainer Endoscopy 2003; 35: 515 520

Downloaded by: University of Queensland. Copyrighted material.

Original Article
516

Next, in collaboration with the endoscopy research group in Munich, a protocol for the requirements in diagnostic upper gastrointestinal endoscopy was adapted to follow the individual steps
of a standard examination. At a consensus meeting which included ten invited endoscopists who blindly assessed videotapes
of 15 endoscopists carrying out upper gastrointestinal endoscopy in a training bio-simulation model (the Erlangen Endo-Trainer), the various assessment parameters were then checked
with regard to interobserver variability and their usefulness for
distinguishing between beginners and experienced endoscopists
[9]. The resulting scorecard, checked and evaluated al a consensus meeting with 10 experienced endoscopists, formed the basis
for a subsequent multicenter study, which was carried out under
the sponsorship of the Endoscopy Section of the German Society
for Digestive and Metabolic Diseases (Deutsche Gesellschaft fMr
Verdauungs- und Stoffwechselkrankheiten; DGVS) and the European Society of Gastrointestinal Endoscopy (ESGE) at 14 centers
of endoscopic expertise from August to December 2001.

at an expert consensus meeting. Using this scoring system, each

step involved in upper gastrointestinal endoscopy was assessed
on a scale ranging from 1 (excellent) to 6 (very poor); global
scores for the entire examination and for examination of the
three different organs (esophageal, gastric, and duodenal endoscopy), as well as for completeness of the examination, were included in the evaluation. Details are shown in Table 1. The score
card defined a standard approach to upper gastrointestinal endoscopy, based on expert experience of the way in which a systematic and complete assessment of the upper gastrointestinal
tract should be performed. In addition, the preparation characteristics of the training specimens (polyps, ulcers, and methylene
blue spots for biopsy) were also taken into account. A total of 15
minutes was generally set as the time limit for the examinations,
both for the beginners group and for the expert group.
Implementation and Program for the Study Week
Four beginners at each of the 14 clinical endoscopy centers completed a 5-day structured training week, with daily assessments
according to the score-card parameters. Each beginner carried
out a self-assessment immediately after the end of each test run
on days 1 5. On days 1, 3, and 5, the beginners test runs were
recorded on video and were evaluated after the study week by
the director or assistant medical director of the endoscopy department concerned.
All of the study weeks were conducted according to the same
plan. On day 1, a 30-minute standardized introduction was given
by the director of the endoscopy center concerned, to provide the
theoretical background and practical instructions for a standard
gastroscopy. The four beginners were instructed on the preparation for gastroscopy, the endoscopy tower and its equipment, the

P1

Introduction of the endoscope and passage through the throat

P2

Passage through the esophagus

P3

Complete assessment of the proximal cardiac folds

P4

Passage through the stomach down to the pylorus, along

the lesser curvature

P5

Passage through the pylorus

P6

Complete (circular) assessment of the duodenal bulb

P7

Introduction of the scope into the descending duodenum

P8

Complete assessment of the duodenal folds

P9

Complete visualization of the antrum

P10

Localization/visualization of an antral ulcer

P11

Visualization of the angular fold

P12

Performance of the retroflexion maneuver

P13

Visualization of the gastric fundus and cardia in retroflexion

P14

Localization/visualization of a fundic polyp

P15

Visualization of the gastric body in retroflexion, and of the

lesser curvature

P16

Placement of biopsy forceps on a gastric ulcer

P17

Placement of biopsy forceps on a gastric polyp

P18

Placement of biopsy forceps on blue marking in antral area

and biopsy

P19

Withdrawal through the esophagus

P20

General assessment (complete assessment of the upper

gastrointestinal tract)

P21

Overall mark for the esophagus

P22

Overall mark for the stomach

P23

Overall mark for the duodenum

P24

Overall mark for technique

Original Article

Figure 3 Endoscopic view showing

the cardiac folds in
the training specimen (pig stomach).

Table 1 Score-card parameters for the steps of a gastroscopy examination

517
functioning of the endoscope, the use of an instrument (in this
case, biopsy forceps), and the basic attitude at the start of the examination. An experienced endoscopist then demonstrated a
standard examination, explaining the individual parameters of
the score card. Each of the four beginners then had 15 minutes
in which to use the endoscope themselves and attempt to put
into practice what had just been demonstrated. During this initial phase of free training, before the initial assessment, no hints
or corrections were provided. The first day ended with the initial
assessment for each participant. Each of the four participants
completed a test run lasting up to a maximum of 15 minutes
(with video recording) that followed the score card protocol, although keeping to the precise sequence of the individual parameters was not compulsory, and immediately afterward carried
out a self-assessment.
Study days 2 5 followed the same pattern. On each day, a 2-hour
period of training was provided, with an introduction by an experienced endoscopist, with each participant only using the endoscope independently for around 30 minutes. For the remainder of the time, the performance of other participants, as well as
instructions, hints, and corrections they were given, was observed on the video endoscope monitor. After the training part
of the day, each participant carried out a test run, which was recorded on video for later assessment by the expert endoscopist.
Each participant carried out a self-assessment immediately
afterward.

Neumann M et al. One-Week Courses Using the Erlangen Endo-Trainer Endoscopy 2003; 35: 515 520

Downloaded by: University of Queensland. Copyrighted material.

Figure 2 Endoscopic view showing

a biopsy from an ulcer at the pylorus in
the training specimen (pig stomach).

Table 2 Statistically significant difference (P < 0.05) between selfassessment and specialist-assessment scores for various
steps in gastroscopy. All parameters were scored from 1
to 6
Parameters

Statistical Analysis
The MannWhitney test was used to compare unpaired data
(self-assessment vs. assessment by specialists) and the Wilcoxon
signed rank test was used to compare paired data (grades
achieved on day 1 vs. those achieved on day 5). The paired Students t-test was used for comparison of normally distributed
data (time needed for examination).

Original Article

A P value of less than 0.05 was considered statistically significant. All calculations were carried out using the Statistical Package for the Social Sciences program for Windows, version 10.1
(SPSS Inc., Chicago, USA).

Results
Significant differences between the self-assessments and assessments by specialists were found for a total of 24 parameters
(MannWhitney test, P < 0.05). These significant differences
were all caused by participants awarding themselves better
marks in self-assessment than they received in specialist assessment (Table 2). With regard to the parameter, complete assessment of the proximal cardiac folds, significant over-marking
was observed in self-assessments on days 1, 3, and 5 (Figure 4).

518

When days 1 and 5 were compared, the assessment marks

showed highly significant improvements between days 1 and 5
for all parameters. This was true for both specialist assessment
and self-assessment (Wilcoxon test, all P < 0.001). Table 3 shows
learning progress based on median marks given by experienced
endoscopists on day 1 in comparison with those given on day 5.
The improvement from day 1 to day 5 in general self-assessment
marks is shown in Figure 5.

Self-Assessment
Median
Range

Specialist-Assessment
Median
Range

P1

16

26

0.009

P2

15

25

0.028

P3

16

26

0.007

P4

16

26

0.039

P1

15

26

0.001

P2

14

14

0.009

P3

14

15

< 0.001

P4

1.5

2.5

0.005

P6

15

25

0.003

P9

15

25

0.007

P11

14

24

0.002

P12

15

15

0.002

P13

15

25

0.006

P14

15

15

< 0.001

P16

14

16

0.002

P21

14

14

0.034

Day 1

Day 2

Day 5
P3

13

14

0.001

P13

15

14

0.017

P14

15

15

0.025

P15

15

14

0.007

P17

15

14

0.014

P18

14

14

0.025

P20

15

14

0.023

P22

15

14

0.029

Assessment of cardiac folds

N
(graded 1 6)
7

Day 1

The difference between days 1 and 5 in the mean time required

to complete the examination was also highly significant (paired
Students t-test, P < 0.001), as shown in Figure 6.

Day 3

Discussion

Day 5

5
4
3

It is difficult for beginners in endoscopy on internal hospital

training programs to get a good practical start with patients in
such a way that the examination is completed safely, within a
reasonable time, and without exposing the patient to discomfort
or even risk. This is particularly important in the field of endoscopy because the beginner is the only practitioner holding the
endoscope and any correction involves taking it away from him
or her (in contrast to surgery, in which the senior surgeon can intervene and carry out corrections directly at the open operating
site).

P (Mann
Whitney)

1
0
N = 56 56 56
Selfassessments

Neumann M et al. One-Week Courses Using the Erlangen Endo-Trainer Endoscopy 2003; 35: 515 520

57 57 57
Specialistassessments

Figure 4 Box plots

demonstrating the
improvement in
complete assessment
of the proximal cardiac folds (days 1 5).
This also demonstrates that participants gave themselves better scores
than the specialists
gave them.

Downloaded by: University of Queensland. Copyrighted material.

The control assessments, using the video recordings from days 1,

3, and 5, were carried out by the director or assistant medical director of the endoscopy center during the week after the actual
study week, taking a maximum of 3 hours for the group of beginners.

Table 3 Medians and ranges of specialist-assessment marks for

conducting each step of the examination on day 1 vs.
day 5 (58 participants)

Figure 5 Box plots

showing improvements from day 1 to
day 5 in self-assessment marks for general assessment of
the gastrocsopy.

General assessment
N
(score 1 6)
7
6

Parameter

Day 1
Median Range

Day 5
Median Range

P
(Wilcoxontest)

P1

26

15

< 0.001

P2

25

14

< 0.001

P3

26

14

< 0.001

P4

26

14

< 0.001

P5

26

15

< 0.001

P6

26

14

< 0.001

P7

26

14

< 0.001

P8

16

15

< 0.001

P9

16

15

< 0.001

P10

16

16

< 0.001

P11

16

15

< 0.001

P12

26

14

< 0.001

P13

26

14

< 0.001

P14

26

15

< 0.001

P15

26

14

< 0.001

P16

16

15

< 0.001

P17

16

14

< 0.001

P18

26

14

< 0.001

P19

16

13

< 0.001

P20

26

14

< 0.001

P21

15

14

< 0.001

P22

26

14

< 0.001

P23

26

14

< 0.001

P24

26

14

< 0.001

5
4
3

0
58

58

58

58

Day 2

Day 3

Day 4

Day 5

Self-assessments

Examination time (mean and 95% Cl),

min
16
15

Figure 6 Box plots

showing examination
times in minutes
(means, with 95 %
confidence intervals),
days 1 5.

Original Article

Day 1

14
13
12
11
10
N = 58
Day 1

58

58

58

58

Day 2

Day 3

Day 4

Day 5

519
To make a beginners initial procedures easier, it is important to
reach this practical starting level through a concentrated and
effective preparation phase. Neither the assistant medical staff
nor the physicians can have their routine procedures disturbed
for too long. In everyday practice, therefore, this developmental
stage is often a lengthy one and it can vary widely from one beginner to another, some beginners requiring more prolonged
training.
The systematic instruction system inherent in the use of the
score-card can help the beginner to make progress in a structured fashion and, above all, it provides a standard by which the
progress of learning and the potential for improvement can be
assessed. The score-card can provide guidance, but its use should
not be regarded as obligatory. The standard examination is
sometimes carried out in very different ways in different hospitals. For example, some only introduce the endoscope under direct vision, while others introduce it blindly with finger guidance. Some examiners push the endoscopes themselves, while
others leave this to the assistant staff and keep both hands on the
control wheels. Others require trainees to carry out minor interventions, such as polyp removal or submucosal injection, even
during basic courses.

The study shows that a structured 1-week course, based on a

consensus training protocol, can improve the learning curve for
all beginners within 1 week. A clear improvement is seen particularly between days 3 and 5, confirming that a course comprising five consecutive training days is a very effective way of aquiring competence in the individual steps required for examining
patients.
Another important aspect of this training method is the workplace atmosphere, which corresponds fully with that experienced later with a patient the endoscopy tower, equipment, instruments, and team training with the endoscopy nurses.
In the long run, the development and constant optimization of a
curriculum and the establishment of suitable training guidelines
by specialist societies [10 18] will be indispensable to the provision of further theoretical support for this clinical weeks training.
In conclusion, an internal hospital training week based on this
approach, individually adapted by each hospital, with a theoretical background and instruction provided by the director of endoscopy and endoscopy nurses, should allow most beginners to
progress sufficiently along the learning curve to enable them to
carry out their first supervised endoscopy procedure in a patient.

Neumann M et al. One-Week Courses Using the Erlangen Endo-Trainer Endoscopy 2003; 35: 515 520

Downloaded by: University of Queensland. Copyrighted material.

N = 58

We are grateful to the centers of endoscopic expertise that participated in the study. Our thanks also go to the endoscopy directors who evaluated the video recordings after the study weeks.

Original Article

We are also grateful to the endoscopy research group in Munich

for their intensive collaboration in developing and evaluating the
score card for diagnostic upper gastrointestinal endoscopy,
which provided the basis for the study presented here. Last, but
not least, our thanks are due to Altana Pharma Ltd. in Konstanz,
Germany, which has provided constant and thorough support for
training in endoscopy, and who also vigorously supported the
present study.

References
1

Classen M, Ruppin H. Practical training using a new gastrointestinal

phantom. Endoscopy 1974; 6: 127 131
2
Rey JF, RomaPzyk T. The development of experimental models in the
teaching of endoscopy: an overview. Endoscopy 1995; 27: 101 105
3
Neumann M, Hochberger J, Felzmann T et al. The Erlangen EndoTrainer. Endoscopy 2001; 33: 887 890
4
Hochberger J, Neumann M, Maiss J et al. EASIE Erlangen Active Simulator for Interventional Endoscopy a new bio-simulation model
first experiences gained in training workshops. Gastrointest Endosc
1988; 47 (Suppl.): AB116
5
Grund KE, Straub T, Farin G. Integriertes Ausbildungskonzept fMr die
diagnostische und interventionelle flexible Endoskopie. Z Gastroenterol 1999; 37: 943
6
Bar-Meir S. A new endoscopic simulator. Endoscopy 2000; 32: 898
900

Neumann M. Pilotprojekt systematisches OP-Training nach bewertbaren QualitStskriterien. Mitt Dtsch Ges Chir 1997; 4: 317 320
8
Neumann M, Siebert T, Rausch J et al. Scorecard endoscopy: a pilot
study to assess basic skills in trainees for upper gastrointestinal endoscopy. Langenbecks Arch Surg 2003; 387 (9 10): 386 391
9
Neumann M, Friedl S, Meining A et al. A score-card for diagnostic upper Gl endoscopy: evaluation of interobserver variability in examiners
with various levels of experience. Z Gastroenterol 2002; 40: 857 862
10
American Society for Gastrointestinal Endoscopy Standards of Training and Practice Committee. Methods of granting hospital privileges
to perform gastrointestinal endoscopy. Gastrointest Endosc 1992; 38:
765 767
11
Farthing MJG, Watt RP, Allan RN et al. A national training programme
for gastroenterology and hepatology. Gut 1966; 38: 459 470
12
Conjoint Committee for Recognition of Training in Gastrointestinal
Endoscopy. Information for supervisors: changes to endoscopic training. Sydney: The Conjoint Committee for Recognition of Training in
Gastrointestinal Endoscopy, 1997
13
Rey JF, Lambert R, and the European Society for Gastrointestinal Endoscopy Quality Assurance Committee. ESGE recommendations for
quality control in gastroinestinal endoscopy: guidelines for image
documentation in upper and lower Gl endoscopy. Endoscopy 2001;
33: 901 903
14
American Society for Gastrointestinal Endoscopy. Principles of training in gastrointestinal endoscopy. Gastrointest Endosc 1999; 49:
845 850
15
American Society for Gastrointestinal Endoscopy. Guidelines for advanced endoscopic training. Gastrointest Endosc 2001; 53: 846 848
16
American Society for Gastrointestinal Endoscopy. Methods of privileging for new technology in gastrointestinal endoscopy. Gastrointest Endosc 1999; 50: 899 900
17
Society of American Gastrointestinal Endoscopic Surgeons. Granting
of privileges for gastrointestinal endoscopy by surgeons. Santa Monica, California: Society of American Gastrointestinal Endoscopic Surgeons, 1992
18
Federation of Digestive Disease Societies. Guidelines for training in
endoscopy. Manchester, Massachusetts: Federation of Digestive Disease Society, 1981

520

Neumann M et al. One-Week Courses Using the Erlangen Endo-Trainer Endoscopy 2003; 35: 515 520

Downloaded by: University of Queensland. Copyrighted material.

Acknowledgements