Medical Teacher

ISSN: 0142-159X (Print) 1466-187X (Online) Journal homepage: http://www.tandfonline.com/loi/imte20

Twelve tips for the construction of ethical dilemma

case-based assessment

Tsuen-Chiuan Tsai

To cite this article: Tsuen-Chiuan Tsai (2017): Twelve tips for the construction of ethical dilemma
case-based assessment, Medical Teacher, DOI: 10.1080/0142159X.2017.1288862

To link to this article: http://dx.doi.org/10.1080/0142159X.2017.1288862

Published online: 16 Feb 2017.

Submit your article to this journal

View related articles

View Crossmark data

Full Terms & Conditions of access and use can be found at

http://www.tandfonline.com/action/journalInformation?journalCode=imte20

Download by: [Fudan University] Date: 17 February 2017, At: 12:51

MEDICAL TEACHER, 2017
http://dx.doi.org/10.1080/0142159X.2017.1288862

TWELVE TIPS

Twelve tips for the construction of ethical dilemma case-based assessment

Tsuen-Chiuan Tsai
Department of Pediatrics, Kaohsiung Medical University Hospital, Kaohsiung Medical University, Kaohsiung, Taiwan

ABSTRACT
Ethical dilemma case-based examination (ethics Script Concordance Test, eSCT) is a written examination that can be deliv-
ered to a large group of examinees for the purpose of measuring high-level thinking. As it accommodates for diverse
responses from experts, ethics SCT allows partial credits. The framework of ethics SCT includes a vignette with an ethical
dilemma and a leading question, which asks the examinee to “agree” or “disagree”, plus the shifts of prior decision by add-
ing new information. In this article, the following tips for constructing this type of examination are provided: use “true”
dilemmas, select an appropriate ethical issue, target high-level cognitive tasks, list key components, keep a single central
theme, device quality scoring system, be important and plausible, be clear, select quality experts, validate, know the limita-
tion, and be familiar with test materials. The use of eSCT to measure ethical reasoning ability appears to be both viable and
desirable.

Introduction
In measuring physicians’ abilities regarding ethics reason-
ing, much has been written about case-based assessment,
which involves a case with a single answer. Ideally, this
answer safeguards the fundamental principles of medical
ethics, i.e. autonomy, beneficence, nonmaleficence, and
justice (Beauchamp & Childress 2012). However, ethical
problems characterized by complexity, uncertainty, and
ambiguity may cause difficulty in gathering a single deci-
sion with respect to ethical practices, even among experts.
Ethical dilemmas may emerge when several different ethics
decisions are all accepted in balancing the conflicts
between moral and ethics principles. There is a need for
ethical dilemmas which do not have a single solution to
still be assessable.
Further, the single answer case-based assessment tends
to give credits solely based on the choice of a decision.
However, ethical decisions are governed by people’s
“justification”, which explains why someone holds a belief,
and how the reasons direct their behaviors when facing
challenges. For physicians, it is important to develop a
deep understanding of ethical issues and the influencing
factors in order to “justify” their decisions to deliver proper
care congruent with an individual patient’s needs. Without
a deep understanding of the interaction between
“justification factors”, the “intuition of decision” may lead to
ethical errors, especially when the context changes and
extended time elapses. With the rapid growth of globaliza-
tion, coupled with increasing migration of patients and
health care providers, the cross cultural factors further chal-
lenge physicians when it comes to ethics problem solving.
Tsai found that experts cannot be differentiated from novi-
ces when judged on solely the correctness of the ethical
decision(s), while they can be differentiated by reasoning
quality (i.e., knowledge, justification, reasoning strategies
and decision) (Tsai et al. 2009). The assessment of how the
Practice points
Ethics Script Concordance Test is:

To assess reasoning abilities on ethical dilemmas
in a high-cognitive level.

Based on the answers from local experts.

Important and feasible.

A shift from gut decision to reasoning process.
factors of justification modify physicians’ decisions has
become more important than ever.
Currently, the majority of measurements of people’s rea-
soning quality are based on interview (Patenaude et al.
2003; Dufrene & Glosoff 2004; Woloschuk et al. 2004; Tsai
et al. 2009), observation (Smith et al. 1994), or written crit-
ical reflections (Grace et al. 2016), techniques that are cost
and time-expensive. In addition, the abilities to solve ethical
problems are “case-specific”, namely that performance on
one problem is not a good predictor of performance on
other problems. To increase such test’s psychometric qual-
ity, broader sampling of cases is required, making an inter-
view/performance-based examination not feasible,
especially when it is delivered to a large group of examin-
ees. To address these challenges, the theory and methodol-
ogies derived from the Clinical Script Concordance Test
(SCT) (Charlin et al. 2000), have been applied to develop a
new assessment tool called the ethics SCT (Tsai et al. 2012).
The clinical SCT is a written examination, originally
designed to address clinical problems involving complexity,
uncertainty, and ambiguity, that measures high-level think-
ing (e.g. the illness script) that physicians apply to solve
clinical problems. The SCT allows multiple answers for scor-
ing and awards partial credits. The various responses of
concordance between examinees' and the experts’

CONTACT Tsuen-Chiuan Tsai tsaitc2003@gmail.com 100, Shih-Chuan 1st Road, Kaohsiung, 80708, Taiwan, R.O.C
ß 2017 Informa UK Limited, trading as Taylor & Francis Group
2 T.-C. TSAI

responses is calculated. The SCT has been successfully regular medication. There are only one or two episodes per
implemented in assessing clinical reasoning in many fields, year. During a hospital visit, Dr. Chen found out that about
such as gynecology (Charlin et al. 2002), urology (Sibert a month ago Mr. Chuang started to serve the public as a
et al. 2006), radiology (“Script Concordance Tests (SCTs)”), taxi driver. Mr. Chuang said that he was laid off, and had
and surgery (Nouh et al. 2012) . The psychometric proper- difficulty in finding a regular job. Dr. Chen promised not to
ties of clinical SCTs have been shown to have high reliabil- disclose this information to anyone, but urged Mr. Chuang
ity and construct validity (Lubarsky et al. 2011). to find a different job as soon as possible. Do you agree
This article outlines the essential components of ethics with what Dr Chen did?
SCT and provides tips in writing this type of test materials
for ethical dilemmas. First, a general framework of a case Part A: w Agree w Disagree
with a question is presented, and then followed by descrip- Part B:
tions of more specific strategies and tips. If the following new infor- My opinion My opinion My opinion
mation was added to the would would not would
case … change change strengthen
A general framework of the cases in eSCT 1. Mr. Chuang has five chil-
dren who all are finan-
An ethics SCT case is composed of a short vignette with an cially depended on him.
ethical dilemma and a leading question. The examinees are 2. Mr. Chuang is willing to
sign a statement indicat-
asked to select the option that best reflects their judgment ing that he consents to
toward the decisions or the management choices made by be fully responsible for
the person in the case. The examinees first choose whether all possible consequen-
ces of his condition.
they “agree” or “disagree” with this decision or manage- 3. Mr. Chuang is a gang-
ment choice. The case is then supplemented with add- ster who insists for his
itional details and the examinees are asked whether this right of confidentiality
and his freedom to
new information would alter their original response. choose the job.

A clinical vignette with ethics dilemma …

A behavioral choice, followed by a leading question … . Scoring system for the sample case
Part A: w Agree w Disagree
Part B: (For each item, please check the blank box that best reflects your Eleven ethical experts were requested to provide responses
answer.) for scoring keys. For new information item 1, the answers
If the following new My opinion My opinion My opinion chosen by the experts were as follows:
information was added would would not would
to the case … change change strengthen
No of experts’
New information item 1 Part A Part B responses Scores
New information item 2
New information item 3 Agree My opinion would change 0 0/6 ¼ 0
Agree My opinion would not change 4 4/6 ¼ 0.67
Agree My opinion would strengthen 0 0/6 ¼ 0
The initial decision provided in “Part A” plus responses Disagree My opinion would change 1 1/6 ¼ 0.17
to new information “Part B” is considered as one test item. Disagree My opinion would not change 6 6/6 ¼ 1
For each test item, there are six options, as follows: Disagree My opinion would strengthen 0 0/6 ¼ 0
Most of the experts (six) selected “Disagree” and “My opinion would not
change”, and the examinees who choose these two options will be
Part A Part B awarded a score of 1.
Agree My opinion would change
Agree My opinion would not change
Agree My opinion would strengthen Strategies and tips
Disagree My opinion would change
Disagree My opinion would not change This section discusses the strategies and tips for construct-
Disagree My opinion would strengthen
ing the ethics SCT.
Scoring system
Tip 1
The standards are based on the responses provided by
10–20 experts. For each test item (Part A þ Part B), the Select an appropriate ethical issue
option selected by most experts is given a value of 1, the First select a specific ethical issue, around which a case
answers not selected at all are given a value of 0. This allows based on a real patient can be constructed. The ethical
for partial scores for the answers chosen by only some issues should pertain to the duties and conduct of health
experts, with the score calculated with the following for- care professionals, paralleling with the career ladder of the
mula: “number of experts who selected a particular option” examinees. The following are examples of ethical issues
divided by “number of experts who selected the most popu- that are appropriate for medical graduates/generalists: truth
lar option” (see the following sample case). Given being telling, confidentiality, end-of-life ethics, and futile treat-
blank in Part A, none of the responses in Part B can be ment. At the same time, ethics dilemmas in genetics, organ
awarded with scores. donation and transplantation, and cloning are relevant for
specialists/scientists. Regarding hospital administrators and
Example case clinical leaders, the relevant issues are balancing care qual-
Mr. Chuang, age 45, has had epilepsy since his childhood. ity and efficiency; improving access to care; and allocating
In recent years, he has been almost free of seizure under limited medical resources.

Tip 2
Use the cases with “true” dilemmas
Present a case with “true” dilemmas, with the considerations
of cultural context and their cognitive bases, assuring that
the ethical issue scenarios are both complex and ambiguous.
For instance, euthanasia is accepted as an “ethical” practice
and supported by legal bodies, while being not acceptable
in other countries. Solving such a case should be challeng-
ing, and reaching consensus, even among the experts,
should be difficult. Incidentally, the cases with the same
answer from all experts could be used for test of the single
response multiple choice question (SRMCQ). Scenarios are
chosen where, in a specific context, ethical conflicts arise
among the ethical principles, or between principles and
norms of proper professional conducts, or involve the com-
peting interests among stake holders. Examples of such eth-
ical dilemmas include reporting an incompetent or impaired
colleague; owning up after making a medical error; and pro-
longing futile care for a dying patient.
Tip 3
Target high-level cognitive tasks
The case with ethics dilemmas is to assess how examinees
maximize benefits for patients, balancing the conflicts
among stake holders, and how they justify all the related
components for what is in their mind a correct decision.
Ethics SCTs are designed to specifically assess decision-mak-
ing skills rather than recall of factual knowledge. While
knowledge is important for effective problem-solving in
ethics dilemmas, the challenge posed by eSCT is the appli-
cation of knowledge to the resolution of the problem. For
example, eSCT should not assess examinees’ ability “to
select an essential component for a valid consent”; rather,
eSCT should assess test takers’ ability “to recognize and val-
idate relevant information around a specific patient’ and ‘to
act appropriately based on a decision’. Therefore, the lead-
ing question should ask for a judgment on the correctness
of a decision, or selection of an appropriate behavior for a
patient/stakeholder. The tasks in ‘leading question’ require
the integrated efforts in collecting relevant information, cat-
egorizing this information as either well understood or
ambiguous, and justifying information for its significance.
The following example illustrates questions that test only
knowledge rather than ethics decisions and actions.
A 50-year-old gentleman who is unconscious and in shock
due to a severe car accident needs immediate blood transfu-
sion. He is soon found to be a Jehovah’s Witness. Members
of this denomination refuse blood transfusions. In order to
provide best care for the patient, based on the law, do you
think it is correct to comply with consent from the wife?
What are the essential requirements for a valid consent?
Tip 4
Develop a list of key components
To enable examinees to make informed decisions, provid-
ing sufficient information in the case is of vital importance.
It is helpful to first identify key components that are essen-
tial in solving any ethics problem. The key components
include knowledge of ethical principles and laws,
MEDICAL TEACHER 3
sociocultural experiences and values of the norms, and con-
textual factors. However, not all information is equally
important for the resolution of a problem, and any irrele-
vant concepts may unnecessarily distract examinees during
limited testing time. To address this issue, the author sug-
gests using think-aloud interviews or brainstorming ses-
sions to extract relevant attitudes and beliefs from ethical
experts. By using a scheme-inductive approach, the compo-
nents of critical steps or “joints” can then be presented in a
diagram, based on which case authors can select appropri-
ate and comprehensive information for constructing a task-
oriented scenario. Figure 1 presents such a sample diagram
regarding “futile treatment”. Given that the leading ques-
tion is about a choice of whether to continue holding a
brain dead patient in ICU, all information regarding the
diagnosis of brain death, family’s readiness to make such a
decision, and cost assessment must be provided.
Tip 5
Provide a single central theme
Ensure that the case stem has a central theme that describes
a single ethics dilemma. The information within the scenario
is arranged logically to first describe “the problem to be
assessed”, followed by the new information. The examinees’
thoughts on the link between the “new information” and the
prior decision are then assessed. Similarly, the new informa-
tion of modifiers should be added one by one on each item.
In general, the stem provides a broad picture that asks
for content-specific gut answer, while new information
adds modifiers tailored to individual situations those tests
examinees’ abilities on contexture-specific justification. The
contextual information which has an influence on ethics
decision include: physical/family circumstances, gender,
age, and cross- sociocultural factors. Dilemmas regarding
ambiguity are expected to appear on both levels of justifi-
cation, or at least on the contexture-specific justification.
Tip 6
Device a scoring system that is understandable
It is recommended that the scoring scheme is based on
maximum two options in Part A and three options in Part
B, which generates six choices in total for each item. The
author had conducted several trials with five options in
Part A and/or Part B, and found the scoring system to be
too complicated to be understandable. A complicated scor-
ing system also resulted in difficulties in gathering data
from participants and the calculation of scores. By asking
for yes/no options and providing three options containing
additional information, the trial participants can “solve”
each case in about 2 min, making the entire ethical
dilemma case-based examination, which typically consists
of 15–30 cases, feasible.
Tip 7
The case dilemmas and modifier factors must be
important and plausible
Both the case dilemmas and the modifier factors must refer
to important and plausible real-life issues in medical ethics.
4 T.-C. TSAI
Figure 1. A sample diagram based on information provided by ethical experts and illustrating how experts approach the issue of “futile treatment of a brain
dead patient”. Numbers 1, 2, and 3 indicate the following critical “joints” that lead to a decision: diagnosis of brain death, family’s readiness, and cost
assessment.
Problems that are not typically encountered in medical
practice should be avoided. In summary, it is advisable to
use real-life cases and construct each story around real
patients.
Tip 8
Be clear
The description of case information as well as the roles of
case characters, and the examinees should be as clear as
possible. Also, the stem and the leading question should
clearly indicate the target behaviors/decisions and the
tasks assigned to the examinees. In order for the students
to process information easily, the case should be written
clearly and in a logical sequence. As eSCT is not to assess
examinees’ ability on clinical problem solving, the case
should provide the diagnosis, plan of treatment and/or
outcome, but not the “raw” information regarding signs
and symptoms of the patients. Instead, examinees could
act as the third party, and are asked to provide ethical
decision, or judge the ethical behaviors that the case char-
acters make. Examinees can also play a role in the case,
and be asked to respond to the presented problems.
Samples of leading questions may include the following:
“Do you agree with the decision made by the doctor?” or
“Given you are the doctor, do you agree with the request
from the parents?”
Tip 9
Select qualified experts
The expert panel generates responses that are used to estab-
lish scoring keys for judging the examinees’ performance.
The selection of qualified experts, thus, becomes critically
important for assuring a quality examination. The ethical
experts should be individuals who have served as ethical
consultants, members of ethics committees, and/or teach-
ers of ethics courses for at least several years. An expert is
someone who makes ethical decisions and adopts ethical
behaviors consistent with contemporary standards.
Expertise in ethics is derived from repeated exposure to
ethical problems and drawing conclusions from the opin-
ions of the majority who provide the most defensible justi-
fication for making decisions. In the eSCT, the expert panel
should include between 10 and 20 members. Each expert
is requested to provide answers independently for
each item.
Tip 10
Conduct validation and piloting
Constructing ethical dilemma case-based examination can
be challenging, even when one adheres to the guidelines
provided in this paper. It is strongly recommended to

present the draft set of test items to others with a request
for comments and criticism. Before implementation, invite
experts to validate the cases for the reasonableness, cor-
rectness, understandability, and relevancy. A pilot testing
can be conducted among experts to confirm the
ambiguity/diversity of cases and a “true” nature of ethical
dilemmas. Cases that yield only a single answer from the
expert panel should be discarded from the eSCT set.
Tip 11
Be aware of the limitations
The paper-based eSCT has limitations in assessing people’s
performance on ethics decision. Individual values/beliefs
may govern how people behave and occasionally create
gaps between the “logic-based decision” and “final actions”.
People’s daily behaviors may not reflect the results of their
“ethics justification”. The case-based examination cannot
differentiate the (un)conscious deviations of final actions
from the correct decisions. Therefore, the examination can
determine a person’s potential to do the “right thing”, but
not his/her actual actions. An example of this limitation
would include writing a prescription for a family member
or friend, despite knowing that the behavior is definitely
inappropriate.
Tip 12
Test development team must have sufficient
knowledge and understanding of the concepts being
tested
A test must reflect the learning objectives that are intended
to be assessed. To ensure test’s content validity, the test
development team should first prepare a blueprint to help
guide the selection of test items. The blueprint specifies
the ethical issues and the number of cases/items prior to
selecting and writing the items. The test authors should be
aware of the appropriate depth and complexity of the con-
cepts that will be used to test students. In addition, only
faculty members who are fully trained in a particular ethical
issue are capable of writing quality test items. Finally, the
ethical experts who validate test items and/or generate
standard keys should also have sufficient domain expertise.
Conclusions
Measuring physicians’ capacities on ethical reasoning is
important. Ethical dilemmas those do not have a single
solution even among experts still should be assessed.
Ethical dilemma case-based examination (ethics SCT) is
designed to measure high-level thinking of those physi-
cians applied to solve ethical dilemmas, and awards partial/
full credits for all the plausible answers. As a written exam-
ination, eSCT has the advantages of easy test administra-
tion, being accessible to a large group. By using local
experts to generate standards for scoring, the test results
can be best adjusted to regional “norms”. Therefore, eSCT
can be applied meaningfully across socio-cultural bounda-
ries and over time periods.
Constructing ethical dilemma case-based examination is
not easy. This article introduces framework of ethics SCT,
MEDICAL TEACHER 5
which includes a short vignette with an ethical dilemma
and a leading question, then asking for “agree or not”, plus
the shifts of prior decision by adding new information.
More than the framework, a quality eSCT is constructed
based on the deep understanding of knowledge, factors
those influence peoples’ justification and the ways how
people use on ethical decision making. Similar with other
examinations, test items should be made clear, targeting
important and plausible issues.
Although the construction of an ethical dilemma case-
based assessment seems tedious and expensive, the use of
the examination to measure ethical reasoning ability
appears to be both viable and desirable.
Glossary
Ethics reasoning: The ability to identify, assess, and develop
arguments on resolving ethical problems, which represents
interplay among: medical and ethical knowledge, problem-solv-
ing skills, and the justification for the use of appropriate princi-
ples, laws or values to rationalize a decision or to modify
actions.
Tsai TC, Harasym PH. 2010. A medical ethical reasoning model
and its contributions to medical education. Med Educ.
44:864–873.
High-level thinking (cognition): The cognitive processes that
presuppose not only the recall of factual knowledge but also
its application, such as, comparison, logic reasoning, planning,
and decision-making.
https://cogsci.uni-osnabrueck.de/NBP/PDFs_Publications/
AWH_MSatRoC_2013_PK_KUK TCK.pdf
Acknowledgements
I would like to thank Professor Bernard Charlin for commending on
improving the test framework, Professor Lambert Schuwirth and
Professor Pawel Kindler for improving this article. I also want to
acknowledge Dr Ching-Ju Shen, Nan-chieh Chen, and Dr William Fu-
Hsiung Su for inviting participants.
Disclosure statement
The author reports no conflicts of interest. The author alone is respon-
sible for the content and writing of the article.
Funding
This work was supported by Ministry of Science and Technology in
Taiwan [NSC 102-2511-S-214-003].
Notes on contributor
Tsuen-Chiuan Tsai, MD, PhD, is now an Associate Dean and Professor,
Kaohsiung Medical University, College of Medicine, Kaohsiung City,
Taiwan.
References
Beauchamp TL, Childress JF. 2012. Respect for autonomy, nonmalefi-
cence, beneficence, justice, in: principles of biomedical ethics’.
7th ed. Oxford (UK): Oxford University Press.
Charlin B, Desaulniers M, Gagnon R, Blouin D, der Vleuten C. 2002.
Comparison of an aggregate scoring method with a consensus scor-
ing method in a measure of clinical reasoning capacity' Teach Learn
Med. 14:150–156.
6 T.-C. TSAI
Charlin B, Tardif J, Boshuizen HPA. 2000. Scripts and medical diagnostic
knowledge: theory and applications for clinical reasoning instruction
and research. Acad Med. 75:182–190.
Dufrene RL, Glosoff HL. 2004. The Ethical Decision-Making Scale-revised
Ph.D. thesis. Measure Eval Counsel Dev. 37:2–14.
Grace S, Orrock P, Vaughan B, Blaich R, Coutts R. 2016. Understanding
clinical reasoning in osteopathy: a qualitative research approach.
Chiropr Man Therap. 24:6.
Lean more about the Script Concordance Tests (SCTs). 2016. University
of Montreal, Department of Medicine [cited 2016 February 20].
Available from: https://www.cpass.umontreal.ca/recherche-et-devel-
oppement/script-concordance-tests-scts/learn-more-about-the-sct.
html
Lubarsky S, Charlin B, Cook DA, Chalk C, van der Vleuten CPM. 2011.
Script concordance testing: a review of published validity evidence.
Med Educ. 45:329–338.
Nouh T, Boutros M, Gagnon R, Reid S, Leslie K, Pace D, Pitt D, Walker R,
Schiller D, MacLean A, et al. 2012. The script concordance test as a
measure of clinical reasoning: a national validation study. Am J Surg.
203:530–534.
Patenaude J, Theophile N, Diane F. 2003. Changes in students’ moral
development during medical school: a cohort study. CMAJ.
168:840–844.
Sibert L, Darmoni SJ, Dahamna B, Hellot M-F, Weber J, Charlin B. 2006.
On line clinical reasoning assessment with Script Concordance test in
urology: results of a French pilot study. BMC Medical Education. 6:9.
Smith SR, Balint JA, Krause KC, Moore-West M, Viles PH. 1994.
Performance-based assessment of moral reasoning and ethical judg-
ment among medical students. Acad Med. 69:381–386.
Tsai T-C, Chen D-F, Lei S-M. 2012. The ethics script concordance test in
assessing ethical reasoning. Med Educ. 46:527–527.
Tsai Tsuen-Chiuan Peter H, Harasym S, Coderre K, McLaughlin T.
Donnon. 2009. Assessing ethical problem solving by reasoning
rather than decision making. Med Educ. 43:1188–1197.
Woloschuk W, Peter H, Harasym W. Temple 2004. Attitude change
during medical school: a cohort study. Med Educ. 38:522–534.