Documente Academic
Documente Profesional
Documente Cultură
Performance-monitoring frameworks (PMFs) have metric may diverge from other PMF quality indicators
increasingly been adopted by the health service sector in because higher patient satisfaction scores have been
an attempt to improve efficacy and efficiency of care.1,2 In associated with both higher overall health care expendi-
Ontario, a prominent PMF is the quality-based procedures tures and increased mortality.8 Herein, we sought to assess
(QBP) system. Here, expert advisory panels develop the current state of patient satisfaction in an emergency
practice recommendations for a given procedure and ophthalmology on-call clinic setting using the modified
outline indicators to monitor quality improvements in a Judgements of Hospital Quality Questionnaire (JHQQ)
modified version of the Balanced Scorecard (BSC) struc- by Ware. In addition, the effect provision of anticipated
ture initially proposed by Kaplan and Norton.3 Reim- wait times had on satisfaction ratings was evaluated.
bursement has historically been tied to the quality
indicators in an attempt to drive systemwide
improvements. METHODS
Appropriate quality indicators and metrics that suffi- Study design and eligibility
ciently discriminate patient satisfaction are salient issues Patients presenting to the Hamilton Regional Eye
for the QBPs. The current QBPs for ophthalmic care Institute, Hamilton, Ont., for an emergency ophthalmol-
emphasize patient satisfaction and strive to place “the ogy on-call consultation from St. Joseph Hospital Emer-
patient/user at the center of the care delivery” and include gency Department and St. Joseph Hospital Urgent Care
“patients’ values, preferences and expressed needs in the Center in Hamilton between November 1, 2015, and July
care they receive.”4 Patient satisfaction, however, is a 31, 2016, were asked to participate in this study. Patients
complex outcome and is influenced by a plethora of were included in this study if they were 18 years or older at
factors. Uncertainty, cultural perceptions, and lack of the time of the appointment, proficient in English, and
perceived control over circumstances appear to modify both willing and able to give informed consent for study
satisfaction ratings.5–7 Moreover, the patient satisfaction participation. Potential study participants were excluded if
they had a mental or physical disability that precluded appointment had on perceived satisfaction as captured
accurate survey completion. If a patient presented with by survey responses, the Mann-Whitney U test was
poor vision and was willing to participate in the study, a performed. For all questions, the exact, 2-sided p-value
masked trained research assistant aided the patient in was calculated, with the standard alpha error of o0.05
completing study documentation. This study received considered significant.
ethics approval from the local institutional review board As a secondary study outcome, we sought to determine
(REB#0498) and adhered to the tenants of the Declara- what effect wait time to see a physician; patient age, sex,
tion of Helsinki. education, and mobility; and number of health care
Study participants were randomized to either a control providers seen at the appointment had on patient satisfac-
or intervention cohort. The control cohort received tion. The wait time satisfaction response was dichotomized
preappointment information that included the visit time as scores at or above the 50th percentile and scores below
and date, clinic location, and physician name if known at the 50th percentile. A score at or above the 50th percentile
the time. This was consistent with the Hamilton Regional corresponded to a survey response of “excellent.” To
Eye Institute’s standard of practice at the time. The preserve degrees of freedom, mobility was dichotomized
intervention cohort received identical preappointment as no help required or help required, and the number of
information with the addition of details regarding antici- health care providers was dichotomized as one provider
pated wait times, a descriptor of a typical patient encoun- seen or more than one provider seen. A forward logistic
ter at the clinic, and a suggested list of items patients could regression model was then performed; only those variables
bring with them to the appointment should they so that achieved statistical significance were included in the
choose. A copy of study intervention material is provided model. The odds ratios (ORs) of significant variables with
in Appendix 1. The intervention specifically addressed 95% confidence intervals (95% CIs) were calculated and
uncertainty regarding wait times, explaining the reasons reported. The p-value of the Hosmer-Lemeshow goodness-
behind potentially longer-than-expected wait times, and of-fit test was reported for the model. Study participants
introduced the concept of possible triaging among were asked to assess their satisfaction with the preappoint-
patients at the eye clinic. Randomization was performed ment information as either satisfied or not satisfied. The
in blocks of 4 with an allocation ratio of 1:1 between study Fisher exact test was used to compare responses between
cohorts. The ophthalmologists evaluating study partici- intervention and control cohorts.
pant were blinded to cohort assignment at the time of Analysis was performed on SPSS software (IBM, version
evaluation. 22.0). Post hoc, a histogram of patient wait time satisfac-
After the patient–physician encounter, study partici- tion scores by intervention, control, and all study partic-
pants were given a copy of the modified JHQQ. The ipants was generated on SPSS. If a given participant’s
questionnaire is available for reference in Appendix 2. The survey was missing a response to the primary research
JHQQ is a validated metric9 that assesses patient satisfac- question, the entire survey was excluded from analysis.
tion and has historically been applied to an English- However, if a response option other than the primary
speaking ophthalmic population.6 The JHQQ assesses endpoint was left blank, the survey was still included in
satisfaction on using a 5-point Likert response scale, with the primary analysis but was excluded in the analysis of the
response options consisting of “poor,” “fair,” “good,” “very missing domain.
good,” and “excellent” for most domains.
RESULTS
Statistical analysis From November 1, 2015, to July 31, 2016, 50 patients
Participant demographic information is summarized as consented to study participation. One of the 50 patients
means and standard deviations for continuous variables or failed to provide a survey response to the primary outcome
frequency with associated percentages for categorical and was subsequently removed from analysis. The study
measures. Survey responses were scored on a 5-point cohort consisted almost equally of male (46.9%) and
Likert scale, with 1 being the lowest or most disagreeable female (53.1%) participants. The mean age of study
score and 5 being the highest or most agreeable score. This participants was 54.4 ± 18.0 years. Total wait time to
was true of all questions except the final question, which see the doctor upon arrival at the eye clinic averaged 20.5
assessed patient satisfaction on a 3-point scale. We treated ± 23.6 minutes. Subsequent demographic information is
the survey scores as ordinal data. For ordinal data, the provided in Table 1. The survey response to question 2,
preferred measures of central tendency and dispersion are which reads, “Compared to your expectations, the wait
the median and interquartile range, respectively. Survey was poor, fair, good, very good, or excellent,” served as the
responses were reported using this convention. To assess dependent variable for the primary research question. The
the influence that informing the patient of typical wait median response was 4.00 for the intervention cohort and
times for an emergency on-call ophthalmology clinic 4.50 for the control cohort, meaning that the median
Patient Wait-Time Satisfaction Intervention(N=25) Patient Wait -Time Satisfaction control (N=24)
finding consistent with previous research.5 Despite there problems.10 Patient satisfaction scores have been demon-
being no difference in patient satisfaction between the strated to correlate positively with all-cause mortality and
intervention and control cohorts, intervention patients higher health care expenditures.8 It is now known that, in
reported appreciating the anticipated wait time informa- part, these contradictory findings can be explained by the
tion. Of patients in the intervention group, 92.0% (23/25) fact that the experience a patient has at a given encounter
reported that information provided regarding wait times within a given health care system accounts for only a
and visit details was useful. Meanwhile, only 12.5% (3/24) fraction of variation in satisfaction scores. In a study of 21
of control patients reported the same (p o 0.001). The European Union countries, Bleich et al. found that
overall distributions of satisfaction scores were highly approximately 10% of all variability in patient satisfaction
skewed to the left (Fig. 1) in both control and intervention scores could be accounted for by patient encounter
cohorts, with 85.7% (n ¼ 42) of all study participants experience with the health care system.11 Meanwhile,
reporting a satisfaction score of either “very good” or other known factors explained some 17.5% of satisfaction
“excellent.” variability, whereas the majority of satisfaction variability
Thus, although patients reported that having knowledge can be attributed to as-yet-unknown determinants.11
of anticipated wait times was useful and that increases in Like many measures of patient satisfaction, the JHQQ
wait times decreased their satisfaction with care, neither uses a Likert scale. These metrics suffer from ceiling and
factor resulted in a statistically meaningful difference in floor effects, and their discriminatory ability has been
satisfaction scores between intervention and control called into question.12,13 The divergence between patient-
cohorts using the JHQQ. Apparent contradiction in reported relevance of anticipated wait time knowledge and
seemingly corollary proxies for patient satisfaction and actual wait times satisfaction suggests that the JHQQ scale
satisfaction with care itself is not a new phenomenon. itself had poor discriminatory ability in assessing patient
Donelan et al. reported high care satisfaction across experience with an on-call ophthalmology care system.
patients from 5 nations who described the health care This explanation is further corroborated by extensive left
systems of their individual countries as having significant skewing of survey scores in both study cohorts (see Fig. 1).
Table 2 (continued )
The high satisfaction of study participants may be (n ¼ 42) of survey respondents scored their satisfaction
attributable to Bleich’s as-yet-unknown factors. Instead with wait times as “very good” or “excellent.” There was
of factors salient to the provision of care modulating no detected difference in satisfaction with care between
satisfaction scores, factors innate to the individual may be patients who received anticipated wait time information
responsible. and those who did not, despite there being a difference in
Study findings should also be interpreted in light of wait the proportion of patients who found the preappointment
time experienced by the population under investigation information useful. This may be explained, in part, by the
and sample size. Average wait time for study participants fact that the patient satisfaction metric used has limited
was 20.5 ± 23.6 minutes. Given that this wait time is discriminatory ability in our population. Although pre-
consistent with those of other services,14 wait times appointment information on estimated appointment wait
experienced by patients in the present study may be time had little effect on patient satisfaction with the
consistent with their preconceived notions of what con- ophthalmic visit, 92.0% of patients reported that the
stitutes a reasonable wait. The study sample size was 50. In information was useful, suggesting that the material may
light of the post hoc findings that the JHQQ had limited have some utility in improving the overall patient experi-
discriminatory ability in our ophthalmic population, a ence. Patient satisfaction with care has many determinants,
larger sample size is required to establish equivalence of and accurately assessing the construct is influenced by
satisfaction scores between study arms. numerous parameters, some of which are intrinsic to the
Future research exploring patient satisfaction in an individual. For this reason, using patient satisfaction
emergency ophthalmology on-call service clinic should measures in guiding health care system reform needs to
seek to evaluate the discriminatory ability of the scale be done with caution.
under investigation. Other disciplines have found that
scales with a lower number of response categories have
poorer validity, reliability, and discrimination power than APPENDIX
scales with more response categories.15 A wider scale may, Supplementary data
in part, overcome the poor discriminatory ability of the Supplementary data associated with this article can be
JHQQ scale in this investigation. As patient satisfaction found in the online version at http://dx.doi.org/10.1016/j.
metrics increasingly play a relevant role in the provision of jcjo.2017.08.002.
care and systemic changes on an administrative level, it is
paramount that outcome indicators reflect the intended
construct they are designed to measure in a valid and
discriminatory manner. REFERENCES
7. Leung V, Vanek J, Braga-Mele R, et al. Role of patient choice in 15. Preston CC, Colman AM. Optimal number of response categories
influencing wait time for cataract surgery. Can J Ophthalmol. in rating scales: reliability, validity, discrimination power, and
2013;48:240-5. respondent preferences. Acta Psychol. 2000;104:1-15.
8. Fenton JJ, Jerant AF, Bertakis KD, et al. The cost of satisfaction:
a national study of patient satisfaction, health care utilization,
expenditures, and mortality. Arch Intern Med. 2012;172:
405-11.
9. Ware JE, Hays RD. Methods for measuring patient satisfaction with Footnotes and Disclosure:
specific medical encounters. Med Care. 1988;26:393-402.
10. Donelan K, Blendon RJ, Schoen C, et al. The cost of health system The authors have no proprietary or commercial interest in any
change: public discontent in five nations. Health Aff. 1999;18: materials discussed in this article.This article includes online-only
206-16. material. Appendices 1 and 2 can be found on the CJO web site
11. Bleich SN, Ozaltin E, Murray C. How does satisfaction with the at http://pubs.nrc-cnrc.gc.ca/cjo/cjo.html. They are linked to this
health-care system relate to patient experience? Bull World Health article in the online contents of the xxx 20xx issue.
Organ. 2009;87:271-8.
12. Moret L, Nguyen JM, Pillet N, et al. Improvement of psychometric From the *Hamilton Regional Eye Institute, Research Unit,
properties of a scale measuring inpatient satisfaction with care: a Division of Ophthalmology, Department of Surgery, McMaster
better response rate and a reduction of the ceiling effect BMC Health University, Hamilton, Ont; †Wayne State School of Medicine,
Serv Res. 2007;7:197. Detroit, MI.
13. Howorka K, Pumprla J, Schlusche C, et al. Dealing with ceiling
baseline treatment satisfaction level in patients with diabetes under Originally received Feb. 20, 2017. Final revision Jul. 8, 2017.
flexible, functional insulin treatment: assessment of improvements in Accepted Aug. 8, 2017.
treatment satisfaction with a new insulin analogue. Qual Life Res.
2000;9:915-30. Correspondence to Joshua Barbosa, BHSc, Division of Oph-
14. Downey LV, Zun LS, Burke T. Comparison of Canadian triage thalmology, Department of Surgery, Hamilton Regional Eye
acuity scale to Australian emergency mental health scale triage for Institute, 2757 King Street East, Hamilton, Ont. L8G 5E4;
psychiatric patients. Int Emerg Nurs. 2015;23:138-43. joshuawilliambarbosa@gmail.com