J Robotic Surg (2015) 9:179186

DOI 10.1007/s11701-015-0513-4

ORIGINAL ARTICLE

Virtual reality robotic surgery simulation curriculum to teach

robotic suturing: a randomized controlled trial
Daniel J. Kiely1,2 Walter H. Gotlieb3 Susie Lau3 Xing Zeng4 Vanessa Samouelian2

Agnihotram V. Ramanakumar5 Helena Zakrzewski1 Sonya Brin3

Shannon A. Fraser6 Pira Korsieporn7 Laura Drudi8 Joshua Z. Press9

Received: 6 March 2015 / Accepted: 28 April 2015 / Published online: 16 May 2015
Springer-Verlag London 2015

Abstract The objective of this randomized, controlled
trial was to assess whether voluntary participation in a
proctored, proficiency-based, virtual reality robotic sutur-
ing curriculum using the da Vinci Skills SimulatorTM
improves robotic suturing performance. Residents and at-
tending surgeons were randomized to participation or non-
participation during a 5 week training curriculum. Robotic
suturing skills were evaluated before and after training
using an inanimate vaginal cuff model, which participants
This manuscript was presented at the Society of Gynecologic
Oncology 45th Annual Meeting on Womens Cancer, Tampa, FL,
USA, March 23, 2014 and at the Society of Gynecologic Oncology of
Canada 35th Annual General Meeting. June 13, 2014. Niagara Falls,
ON, Canada.
Condensation: In this randomized controlled trial, learners
randomized to participation in a virtual reality robotic suturing
curriculum showed a trend towards greater improvement when scored
on their ability to suture an inanimate vaginal cuff model using the da
Vinci Surgical System.
ClinicalTrials.gov: NCT01811095, www.clinicaltrials.gov.
sutured for 10 min using the da Vinci Surgical System.
Performances were videotaped, anonymized, and subse-
quently graded independently by three robotic surgeons. 27
participants were randomized. 23 of the 27 completed both
the pre- and post-test, 13 in the training group and 10 in the
control group. Mean training time in the intervention group
was 238 136 min (SD) over the 5 weeks. The primary
outcome (improvement in GOALS? score) and the sec-
ondary outcomes (improvement in GEARS, total knots,
satisfactory knots, and the virtual reality suture sponge 1
task) were significantly greater in the training group than
the control group in unadjusted analysis. After adjusting for
lower baseline scores in the training group, improvement in
the suture sponge 1 task remained significantly greater in
the training group and a trend was demonstrated to greater
improvement in the training group for the GOALS? score,
GEARS score, total knots, and satisfactory knots.
Keywords Randomized controlled trial
Patient
simulation
Computer simulation
Graduate medical
education
Robotics
Operative surgical procedures

Electronic supplementary material The online version of this

article (doi:10.1007/s11701-015-0513-4) contains supplementary
material, which is available to authorized users.
5
& Daniel J. Kiely Epidemiology, McGill University, Montreal, Canada
danieljameskiely@gmail.com 6
General Surgery, Jewish General Hospital, McGill
1
University, Montreal, Canada
Experimental Surgery, McGill University, Montreal, Canada 7
2
Obstetrics and Gynecology, McGill University, Montreal,
Gynecologic Oncology, University of Montreal, Montreal, Canada
Canada 8
3
Vascular Surgery, McGill University, Montreal, Canada
Gynecologic Oncology, Jewish General Hospital, McGill 9
University, Montreal, Canada Division of Oncology and Pelvic Surgery, Pacific
4
Gynecology Specialists, Seattle, USA
Gynecologic Oncology, Royal Victoria Hospital, McGill
University, Montreal, Canada

123
180
Introduction
Robotic suturing is an important skill for the aspiring
robotic surgeon, which must be performed with precision
and efficiency. Robotic suturing skills may possibly be
developed outside of the operating room through surgical
simulation. Interest in simulation is on the rise due to
several current trends including increasing emphasis on
patient safety and operating room efficiency, increasing
complexity of surgery, decreased trainee work hours, and
greater medico-legal risk [1, 2].
Multiple studies have demonstrated the face, content,
construct, and concurrent validity of the da Vinci Skills
simulator [37]. Predictive validity refers whether perfor-
mance with the simulator can predict future performance of
the task in reality or in another simulation, with or without
intervening training [8, 9]. An additional form of validity
merits definition: training validity. Training validity refers
to the extent to which training with the simulator improves
ability to perform a task. This may be considered a subset
of predictive validity which explicitly requires training
between baseline and final assessment. Training validity
has three basic subtypes, depending on outcome: type 1,
the outcome is the same simulation task used for training;
type 2, the outcome is a different simulation, ideally one
which is considered more realistic; and type 3, the outcome
is the actual real-life task which is the goal of training. To
our knowledge, there are no studies published to date
assessing the strongest type of training validity for robotic
surgery, type 3. Hung et al. [10] studied type 2 training
validity for the da Vinci Skills SimulatorTM where the
outcome was ability to operate using the actual da Vinci
robot on models constructed from ex vivo animal tissue in
three exercises: partial nephrectomy, cystotomy and cys-
totomy repair, and bowel resection. The training group
performed ten weekly training sessions with each session
including the completion of 17 pre-selected virtual reality
exercises using the da Vinci Skills SimulatorTM [10].
Although the training group did not show significantly
greater scores for final performance on the ex vivo animal
tissue exercises than the control group, greater performance
was achieved by the subset of participants who started with
scores in the lower 50th percentile [10]. Similarly, Vaccaro
et al. performed a randomized controlled trial assessing the
effect of virtual reality training (9 exercises, completed to
proficiency or at least 10 attempts) on ability to perform an
inanimate task involving incising and suturing chicken
breasts (again, type 2 training validity) [11]. Both control
and intervention groups improved, but the only statistically
significant difference between groups was in the time to
incision [11]. There were no significant differences in total
time, suturing time, objective structured assessment of
123
J Robotic Surg (2015) 9:179186
technical skills or global rating scales [11]. There was a
strong trend to higher pass rate on the global rating scale in
the training group [11].
Our study aimed to contribute further to this literature
by evaluating type 2 training validity of the da Vinci
Skills SimulatorTM where the outcome was ability to suture
an inanimate model of the vaginal cuff. Our intervention
was a focused, proficiency-based, virtual reality robotic
suturing curriculum. We chose a proficiency-based cur-
riculum because this allows tailoring the amount of
simulation training required per trainee to achieve a pre-set
goal for skill. Increasingly, proficiency-based curriculums
are being recognized as the gold standard in the simulation
literature [1217].
Materials and methods
Ethics and eligibility criteria
Ethics approval was obtained from the McGill Faculty of
Medicine Institutional Review Board. Residents or at-
tending surgeons in the department of Obstetrics and
Gynecology, Urology, General Surgery, or Cardiac Surgery
were eligible. The study was conducted in the spring of
2013. Funding was obtained from the Jewish General
Hospital, Department of Gynecologic Oncology Research
Fund and by a Masters Award from the Canadian Insti-
tutes of Health Research.
Randomization
A random sequence for 40 participants was generated by an
associate not otherwise involved in the study using an
online random number generator with randomly permuted
blocks of 4 or 6 with a 1:1 ratio of training group to control
group within each block. The associate then wrote the
group allocation (train or control) on an index card, cov-
ered the card with carbon copy paper, and sealed both in a
sequentially numbered, opaque envelope to ensure alloca-
tion concealment [18]. Randomization occurred just prior
to the first pre-test of the study.
Pre- and post-test
Ability to perform robotic suturing was evaluated using an
inanimate vaginal cuff model, with assessments done at
baseline and then 5 weeks later after completing the
simulation curriculum (or continuing with routine clinical
duties). The study pre-test session included the following:
(1) a 10 min robotic suturing task involving suturing an
J Robotic Surg (2015) 9:179186
Fig. 1 The da Vinci Surgical System set-up with training arms and the inanimate vaginal cuff model
inanimate model with the actual da Vinci Surgical System
and (2) completion of the suture sponge 1 task using the da
Vinci Skills SimulatorTM. For suturing the inanimate
model, we used the model of the vaginal cuff described by
Finan [19], constructed from a beer huggie to represent
the vaginal cuff, a block of wood for positioning, and
balloons to represent the bladder and rectum, and to distend
the vaginal cuff. Prior to the pre-test, participants were
given a verbal overview of how to use the da Vinci sur-
gical system. They were then instructed to suture closed the
vaginal cuff model using figure of eight stitches with
square knots composed of a double throw and then three
single throws. This task was recorded using the camera of
the da Vinci Surgical System. The post-test was identical
to the pre-test. The vaginal cuff model along with the set-
up used on the testing day is shown in Fig. 1. Supple-
mentary file 1 shows a demonstration of the task of su-
turing the vaginal cuff model.
Training intervention
The training group was assigned to a proficiency-based
robotic surgery simulation curriculum which took place
over a 5 week period between pre- and post-test sessions,
and was completed in addition to usual clinical duties or
residency training. We used the da Vinci Skills Simulator
tasks and the goals of the training curriculum were to
181
achieve the completed score (equivalent to attaining all
component scores C60 % and overall score green
checkmark) non-consecutively on 2 occasions for the
following tasks: camera targeting 1, camera targeting 2,
suture sponge 1, suture sponge 2, suture sponge 3 and to
complete the horizontal suturing defect (HSD) task ten
times. Participants in the training group set their own hours
for training. Given the voluntary nature of the study, there
was no obligation to train and participants were free to try
other simulation exercises outside the curriculum, although
participants were made aware of, and encouraged to
achieve all the goals of the proficiency curriculum. Par-
ticipants were informed that, on average, it was estimated
that it would take 5 h to attain all the goals of the profi-
ciency curriculum and that the suggested training schedule
was 1 h per week for 5 weeks. The study coordinator (DK),
at the time experienced in robotic virtual reality simulation
but not actual robotic surgery, was present for all training
sessions and offered suggestions for improvement.
Outcomes
The primary outcome was skill in performing the task of
suturing the vaginal cuff model. This was assessed by three
blinded raters (two gynecologic oncologists and one gyne-
cologic oncology fellow, all experienced in robotic surgery)
using the GOALS? score. This score is composed of the five
123
182 J Robotic Surg (2015) 9:179186

domains of the GOALS score developed for assessing skill in Assessed for eligibility
laparoscopy which include autonomy, efficiency, tissue (n = 30)

handling, depth perception, and bimanual dexterity [20] plus

Excluded (n = 3)
two additional metrics developed specifically for robotics,
Not meeting inclusion criteria
precision and awareness of camera and instruments [10]. To (n = 1)
blind evaluators, all videos were coded and then rearranged Lost to follow-up
(n = 2)
in random order using a random number generator by a
member of the study team not involved in evaluations.
Scoring of the videos began only after completion of all
testing. Prior to scoring, the study coordinator explained the Randomized (n = 27)

domains of the GOALS? score to the evaluators and en-

couraged them to try to use the entire range of available Allocated to Allocated to control
intervention (n = 14) (n = 13)
scores from 1 to 5, as appropriate. Although we were not
aware of the GEARS score when designing our trial, the Completed pre-test (n=13) No da Vinci Skills Simulator

Allocation
training (n=12)
GOALS? includes 7 domains, 6 of which form the GEARS Participated in proficiency-
based curriculum but did not Engaged in some training
score [21]. Thus, our data allowed us to calculate the GEARS achieve all goals (n = 8) with da Vinci Skills Simulator
outside of study (n=1)
score as a secondary outcome of the study. Completed proficiency-based
The secondary outcomes were the number total knots and curriculum (n = 5)

satisfactory knots performed during the inanimate model

suturing task and scores in the virtual reality suture sponge 1 Lost to follow up Lost to follow up (n = 3)
simulator task on the pre-test and post-test dates. For scoring Follow up (n =0) (unable to attend post-
of total knots, if a knot was partially completed at the 10 min test)

stop time, it was scored as follows: 0.4, if the first double

throw was completed and cinched down and then, an addi-
tional 0.2 for each single throw cinched down. Satisfactory
knots were defined as knots that the rater would not cut out
and re-suture during live surgery. The maximum score for
satisfactory knots was equal to the total knots score. If a
partial knot was judged satisfactory, the score for that partial
knot (e.g. 0.4) was retained as part of the satisfactory knot
score. Evaluators also assessed whether there was an injury
to the bowel or bladder as assessed by perforation of the
balloons representing those structures. Following comple-
tion of the post-test, participants from obstetrics and gyne-
cology, as well as one gynecologic oncologist and one
gynecologic oncology fellow who also tried the task, were
asked to complete a questionnaire assessing the realism (face
and content validity) of the inanimate vaginal cuff model as a
tool for learning robotic suturing.
Sample size calculations
Sample size calculations for the trial were performed with
PASS 11 [22] using a non-parametric sample size calcu-
lator assuming a mean improvement of scores and a stan-
dard deviation equal to 10 % of the maximum GOALS?
score in the training group versus no improvement in
scores in the control group, with a power of 0.08 and a
p value of 0.05 as the cut-off for statistical significance.
This calculation indicated the need for a sample size of 38,
with 19 participants in each group. Due to logistic con-
straints, the trial was designed to stop after the pre-set final
Analyzed (n = 13) Analyzed (n = 10)
Excluded from analysis:
(n = 1) (Absent from pre-
test)
Fig. 2 CONSORT diagram
testing date, regardless of whether the target sample size
was obtained.
Statistical analysis
Statistical analysis was conducted using SPSS. Ordinal
data (GOALS? score, GEARS score, and their compo-
nents) were analyzed using non-parametric tests (Wilcoxon
signed rank test for within group comparisons and Wil-
coxon rank sum test for between group comparisons). The
GOALS? and GEARS score were treated as continuous
data for multivariate regression. Continuous data (virtual
reality simulator scores, total knots, and satisfactory knots)
were analyzed using t tests.
Results
Our consort diagram is shown in Fig. 2. A total of 23 par-
ticipants contributed results for analysis based on attendance
at both the pre- and post-test sessions, 13 in the training group

123
J Robotic Surg (2015) 9:179186 183

Table 1 Baseline characteristics of the participants

Training group (n = 13) Control group (n = 10) p value

Training level 8 residents 9 residents Not

5 attendings 1 attendings applicable
Training level of residents 2 fourth year residents 1 fourth year resident Not
1 third year resident 6 third year residents applicable
3 second year residents 1 second year resident
2 first year residents 1 first year resident
Specialty 10 obstetrics and 5 obstetrics and Not
gynecology gynecology applicable
3 general surgery 3 general surgery
0 urology 2 urology
0 cardiac surgery 0 cardiac surgery
Mean age (range) 35 (2655) 30 (2734) 0.11
Mean number of robotics cases as primary console surgeon (range) 0 (0) 0 (0) 1.00
Mean number of robotics cases with some time on console (range) 0.7 (06) 1.2 (06) 0.97
Mean number of robotics cases as assistant (range) 0.8 (010) 3.3 (010) 0.12
Mean hours trained with laparoscopic simulators prior to study 11.3(080) 22.9 (160) 0.22
(range)
Mean hours trained with robotic simulators prior to study (range) 0.85 (03) 2.6 (020) 0.33
Mean baseline simulator score, suture sponge 1 (range) 49.0 (385) 74.5 (4494) 0.009
Mean baseline GOALS? score (range) 15.1 (7.3331.0) 21 (12.029.7) 0.020

and 10 in the control group. Table 1 shows the baseline
characteristics of the participants. Despite a rigorous ran-
domization protocol, there were imbalances between groups
likely due to the small size of the trial and the lack of
stratification by baseline scores. The training group, on av-
erage, tended to be older, contained more attending physi-
cians, had less robotic surgery experience, less robotic
simulation experience, and significantly lower pretest scores
for both the virtual reality simulator task (suture sponge 1)
and the inanimate model task (GOALS? score).
The mean total time spent training with the da Vinci
Skills Simulator over the 5 week study period was 238 min
among the training group participants (range 15600 min).
Five of the 13 study participants achieved all the training
goals of the proficiency curriculum, with a mean time of
272 min (range 175600 min). The three shortest training
times were among participants who did not complete the
curriculum. One participant in the control group performed
approximately 3 h of robotic simulation training with the
da Vinci Skills SimulatorTM during the study period
outside of the study protocol. One participant in the
training group and two participants in the control group
assisted at robotic surgery during the study period, in-
cluding one participant in the control group who had the
opportunity to suture the vaginal cuff in live surgery twice
during the study period.
The training group showed statistically significant im-
provements between pre-test and post-test in the GOALS?
score (p = 0.002), the GEARS score (p = 0.002), total
knots (p \ 0.001), satisfactory knots (p \ 0.001), and the
suture sponge 1 score (p \ 0.001) (Table 2). The control
group also showed statistically significant improvements in
the GEARS score (p = 0.041), total knots (p \ 0.001) and
satisfactory knots (p = 0.004). Unadjusted analysis com-
paring the magnitude of the improvement in the training
group to that of the control group showed statistically
significantly greater improvement in the training group for
the GOALS? score, the GEARS score, total knots, satis-
factory knots, and the suture sponge 1 score (Table 3).
At baseline, the training group had significantly lower
GOALS? scores (p = 0.020) and GEARS scores (p = 0.023)
than the control group. However, at the post-test, no statisti-
cally significant differences were found between groups in the
GOALS? scores (p = 0.455) or the GEARS scores
(p = 0.598), suggesting that participation in the curriculum
allowed the training group to catch up to the control group.
Training group participants with baseline GOALS?
scores less than 14 out of 35 tended to have greater im-
provements (mean 8.5, n = 8) than those with scores
greater than 14 (mean 3.0, n = 5) with a p value of 0.067.
A multivariate analysis was then conducted (Table 4),
controlling for baseline scores with a regression model
containing pre-score and group allocation as independent
variables and improvement in score as the dependent
variable. In this analysis, group allocation (training or
control) was no longer a statistically significant predictor of

123
184 J Robotic Surg (2015) 9:179186

Table 2 Pre-scores and post-scores in the training and control groups

Type of score Group Pre-score Post-score p value
(post-score vs. pre-score)
Average Range Average Range

GOALS? score (max 35) Training group 15.1 7.3331.0 21.4 1229.7 0.002*
Control group 20.6 13.330.3 22.8 16.731.3 0.052
GEARS score (max 30) Training group 12.7 6.3326.7 18.4 9.6726.3 0.002*
Control group 17.4 1126 19.4 13.327 0.041*
Total knots Training group 1.65 05.6 3.38 1.935.73 \0.001*
Control group 2.45 0.934.87 3.42 1.936.4 \0.001*
Satisfactory knots Training group 1.04 05.07 2.73 1.134.4 \0.001*
Control group 2.02 0.934.2 2.87 15.6 0.004*
Suture sponge 1 score (max 100) Training group 49.0 385 90.7 77100 \0.001*
Control group 74.5 4494 77 6394 0.599
* Statistically significant at a p value \0.05

Table 3 Differences between

Type of score Average difference in scores (post-test minus pre-test) p value
post-scores and pre-scores in the
training and control groups Training group Control group

GOALS? score ?6.4 ?2.2 0.037*

GEARS score ?5.7 ?2.0 0.027*
Total knots ?1.73 ?0.97 0.029*
Satisfactory knots ?1.69 ?0.85 0.041*
Suture sponge 1 ?41.7 ?2.5 \0.001*
* Statistically significant at a p value \0.05

Table 4 Multivariate linear

Outcome variable (post-test minus pre-test) Independent variable 1 Independent variable 2
regression models to control for
pre-score while assessing the Group allocationa Pre-score
impact of group allocation on
improvement in score Standardized beta p value Standardized beta p value

Change in GOALS? score ?0.23 0.26 -0.48 0.02

Change in GEARS score ?0.26 0.20 -0.42 0.05
Change in total knots ?0.32 0.12 -0.37 0.07
Change in satisfactory knots ?0.22 0.26 -0.50 0.02
Change in suture sponge 1 score ?0.298 0.001 -0.765 \0.001
a
Training group coded as 1, control group coded as 0

change in GOALS? score (p = 0.26), change in GEARS
score (p = 0.20), change in total knots (p = 0.12), or
change in satisfactory knots (p = 0.26). Group allocation
did remain a highly statistically significant predictor of
change in suture sponge 1 score, even after controlling for
baseline score (p = 0.001).
Face and content validity of the vaginal cuff model were
supported by surveys completed by 13 obstetrics and gy-
necology participants, one gynecologic oncologist, and one
gynecologic oncology fellow. Their input showed median
ratings of four out of a maximum five for the domains of:
shape of the vaginal cuff, resistance to suture of the vaginal
cuff, extent of capturing essential anatomy, extent of cap-
turing essential surgical challenges, utility for training, and
utility for assessment. However, ratings for proximity of
the bowel and bladder, appearance of the bowel, and ap-
pearance of the bladder were lower: median ratings 3, 2.5,
and 3.5, respectively, suggesting that this aspect of the
model was less realistic.
Comparison of the scores provided by the three
evaluators demonstrated high internal consistency for the
GEARS score, GOALS?, efficiency, total knots, and sat-
isfactory knots (Kendalls coefficient of concordance 0.85,
0.84, 0.75, 0.93, 0.83, respectively). Statistically significant

123
J Robotic Surg (2015) 9:179186
correlations were found between the GOALS? score and
the suture sponge 1 score on the same day, providing
evidence of concurrent validity (Spearmans q 0.510,
p value \0.001). Suture sponge 1 pre-test score was sig-
nificantly correlated with GOALS? score at the post-test
(Spearmans q 0.451, p value 0.031), providing evidence of
predictive validity, with or without training. Strong corre-
lations were found between the GOALS? score and total
knots (Spearmans q 0.875, p value \0.001), and between
GOALS? score and satisfactory knots (Spearmans
q 0.853, p value \0.001). No statistically significant dif-
ferences were found in injury to bowel or bladder between
groups or within groups at pre- versus post-test.
Discussion
Our results are in agreement with those of two other ran-
domized controlled trials assessing virtual reality simula-
tion using the da Vinci Skills Simulator which suggest: a
trend to improve in ability to perform inanimate tasks
robotically, less than perfect translation from virtual reality
to inanimate tasks, and greatest benefit for participants with
minimal to no prior robotic simulation or robotic surgery
experience [10, 11]. Rather than reflecting inherent
limitations of the da Vinci Skills Simulator, which remains
a platform with great flexibility and potential, this may
rather indicate limitations of the current virtual reality
exercises. Nevertheless, the gap in translation between
virtual reality and inanimate model tasks provides support
for the development of inanimate robotic surgical skill
curricula [23] and advanced inanimate robotic surgical
simulations [24, 25] as a possible intermediate step be-
tween virtual reality and live surgery on patients.
The main limitations of our study include baseline im-
balances in pre-test scores between groups and recruitment
which did not attain our target sample size. In addition, there
were some training group participants with short training
times who did not complete the curriculum as well some
contamination of the control group by live robotic surgery
experience or simulation training (off protocol) during the
study period. All of these weaken the strength of our ultimate
conclusion of no significant difference between groups in our
primary outcome, improvement in the GOALS? score.
The strengths of our study are rigorous outcome assess-
ment with blinded video review as well as the possibility of
comparing virtual reality and inanimate model scores at
baseline and conclusion. These aspects allowed us to
establish that virtual reality training translated well into
improved performance of the virtual reality task (highly
significant after adjustment) but not so well into performance
of the inanimate task (not significant after adjustment). In
addition, the curriculum appeared to be most beneficial for
185
participants with lower baseline GOALS? scores. This
could have important implications for development of future
training curricula which are more appropriately tailored
based on the participants pre-training skill level.
The high inter-observer agreement for the GEARS score
provides further evidence regarding the validity of this
scale by an independent group. Furthermore, the high inter-
observer agreement for total knots and satisfactory knots
and their correlation with the GEARS score suggest these
individual measurements may provide simpler substitutes
for the GEARS scale in the limited setting of assessing
suturing ability. Finally, our study provides some valida-
tion of the vaginal cuff model as a training tool with face
and content validity.
In conclusion, participation in a well-defined virtual
reality training curriculum results in a trend of improved
ability to perform robotic suturing as assessed using an
inanimate task. However, statistical significance was not
demonstrated for the primary outcome, improvement in the
GOALS? score. Although our trial has limitations, taken
in the context of two other randomized controlled trials
with similar results [10, 11], it suggests the need to develop
improved virtual reality exercises and supports inanimate
models as a worthwhile complement to virtual reality.
Conflict of interest Authors Kiely, Gotlieb, Lau, Zeng, Samoue-
lian, Ramanakumar, Zakrzewski, Brin, Fraser, Korsieporn, Drudi, and
Press declare that they have no conflict of interest.
Informed consent Informed consent was obtained from all study
participants.
Compliance with ethical standards All procedures performed in
this study were in accordance with the ethical standards of the McGill
Faculty of Medicine Institutional Review Board and with the 1964
Helsinki declaration and its later amendments.
References
1. Reznick RK, MacRae H (2006) Teaching surgical skillschan-
ges in the wind. N Engl J Med 355:26642669
2. Liss MA, McDougall EM (2013) Robotic surgical simulation.
Cancer J 19:124129
3. Hung AJ, Zehnder P, Patil MB, Cai J, Ng CK, Aron M et al
(2011) Face, content and construct validity of a novel robotic
surgery simulator. J Urol 186:10191024
4. Alzahrani THR, Alkhayal A, Delisle J, Drudi L, Gotlieb W,
Fraser S, Bergman S, Bladou F, Andonian S, Anidjar M (2013)
Validation of the da Vinci Surgical Skill Simulator across three
surgical disciplines: a pilot study. Can Urol Assoc J 7:e520e529
5. Lee JY, Mucksavage P, Kerbl DC, Huynh VB, Etafy M, McDougall
EM (2012) Validation study of a virtual reality robotic simulator
role as an assessment tool? J Urol 187:9981002
6. Hung AJ, Jayaratna IS, Teruya K, Desai MM, Gill IS, Goh AC
(2013) Comparative assessment of three standardized robotic
surgery training methods. BJU Int 112:864871
123
186
7. Abboudi H, Khan MS, Aboumarzouk O, Guru KA, Challacombe
B, Dasgupta P et al (2013) Current status of validation for robotic
surgery simulatorsa systematic review. BJU Int 111:194205
8. Fried GM, Feldman LS, Vassiliou MC, Fraser SA, Stanbridge D,
Ghitulescu G et al (2004) Proving the value of simulation in
laparoscopic surgery. Ann Surg 240:518525 (discussion 258)
9. Carter FJ, Schijven MP, Aggarwal R, Grantcharov T, Francis NK,
Hanna GB et al (2005) Consensus guidelines for validation of
virtual reality surgical simulators. Surg Endosc 19:15231532
10. Hung AJ, Patil MB, Zehnder P, Cai J, Ng CK, Aron M et al
(2012) Concurrent and predictive validation of a novel robotic
surgery simulator: a prospective, randomized study. J Urol
187:630637
11. Vaccaro CM, Crisp CC, Fellner AN, Jackson C, Kleeman SD,
Pavelka J (2013) Robotic virtual reality simulation plus standard
robotic orientation versus standard robotic orientation alone: a
randomized controlled trial. Female Pelvic Med Reconstr Surg
19:266270
12. Sroka G, Feldman LS, Vassiliou MC, Kaneva PA, Fayez R, Fried
GM (2010) Fundamentals of laparoscopic surgery simulator
training to proficiency improves laparoscopic performance in the
operating rooma randomized controlled trial. Am J Surg
199:115120
13. Dulan G, Rege RV, Hogg DC, Gilberg-Fisher KM, Arain NA,
Tesfay ST et al (2012) Proficiency-based training for robotic
surgery: construct validity, workload, and expert levels for nine
inanimate exercises. Surg Endosc 26:15161521
14. Dulan G, Rege RV, Hogg DC, Gilberg-Fisher KK, Tesfay ST,
Scott DJ (2012) Content and face validity of a comprehensive
robotic skills training program for general surgery, urology, and
gynecology. Am J Surg 203:535539
15. Palter VN, Graafland M, Schijven MP, Grantcharov TP (2012)
Designing a proficiency-based, content validated virtual reality
curriculum for laparoscopic colorectal surgery: a Delphi ap-
proach. Surgery 151:391397
123
J Robotic Surg (2015) 9:179186
16. Rosenthal ME, Ritter EM, Goova MT, Castellvi AO, Tesfay ST,
Pimentel EA et al (2010) Proficiency-based fundamentals of la-
paroscopic surgery skills training results in durable performance
improvement and a uniform certification pass rate. Surg Endosc
24:24532457
17. Scott DJ, Ritter EM, Tesfay ST, Pimentel EA, Nagji A, Fried GM
(2008) Certification pass rate of 100% for fundamentals of la-
paroscopic surgery skills after proficiency-based training. Surg
Endosc 22:18871893
18. Schulz KF, Grimes DA (2002) Allocation concealment in ran-
domised trials: defending against deciphering. Lancet 359:614618
19. Finan MA, Rocconi CM (2010) A novel method for training
residents in robotic hysterectomy. J Robot Surg 4:3339
20. Vassiliou MC, Feldman LS, Andrew CG, Bergman S, Leffondre
K, Stanbridge D et al (2005) A global assessment tool for
evaluation of intraoperative laparoscopic skills. Am J Surg
190:107113
21. Goh AC, Goldfarb DW, Sander JC, Miles BJ, Dunkin BJ (2012)
Global evaluative assessment of robotic skills: validation of a
clinical assessment tool to measure robotic surgical skills. J Urol
187:247252
22. PASS: Power analysis and sample size software. http://www.
ncss.com/software/pass/. Accessed 6 Feb 2015
23. Arain NA, Dulan G, Hogg DC, Rege RV, Powers CE, Tesfay ST
et al (2012) Comprehensive proficiency-based inanimate training
for robotic surgery: reliability, feasibility, and educational bene-
fit. Surg Endosc 26:27402745
24. Marecik SJ, Prasad LM, Park JJ, Pearl RK, Evenhouse RJ, Shah
A et al (2008) A lifelike patient simulator for teaching robotic
colorectal surgery: how to acquire skills for robotic rectal dis-
section. Surg Endosc 22:18761881
25. Kiely DJ, Gotlieb WH, Jardon K, Lau S, Press JZ (2015) Ad-
vancing surgical simulation in gynecologic oncology: robotic
dissection of a novel pelvic lymphadenectomy model. Simul
Healthc 10:3842