1

Research Attachment SSC Student registration number: ********* Word count: 2726

2 Introduction In the modern era, the internet is being used more and more as a source of information for the public, students and even medical professionals about many aspects the medical 'ou(ube as ran)ed as the &rd most visited orld !1"2#$ In 2%1&,

ebsite on the internet!&"*# and has a broad range of

videos available to vie ers$ +any of these videos are intended to be educational, ho ever their ,uality and accuracy can be e-tremely varied !.#$ (his presents a uni,ue challenge to anyone hoping to learn from them$ (his research pro/ect loo)s at videos available on ebsites such as 'ou(ube that aim to inform the

vie er about randomised control trials 012(s3$ It attempts to collect the best of the videos that are revie ed and combine them to create a 4digital handboo)5 0in the form of a anyone anting to learn about 12(s$ ebsite3 to be used by

The research Question(s) (his pro/ect attempted to address & main ,uestions: 1$ 6re there accurate, high ,uality videos available on the internet to teach people about 12(s7 2$ 2an e gather the best videos together into a single learning platform and ill it be useful7 &$ Is there scope for further video creation and if so, in hat areas7

A literature review In recent years a lot of research has gone into loo)ing at the content available on 'ou(ube and has investigated the ,uality of the videos that are available in regards to a number of different topics !2,."
8#

$ I carried out a literature revie

using +9:;I<9 to ascertain

hat methods had been used in

previous studies that e-plored a similar issues$ (his proved very useful in designing the pro/ect$ =n a side note, as the internet is so rapidly evolving and changing, I decided to limit my revie to only papers published ithin the last & years so as to ensure the research is relevant$

& (he first paper loo)ed at 'ou(ube videos as a source of information on inflammatory bo el disease and concluded that although the videos and clearly stressed there ere popular they ere a poor source of patient education

as a need for healthcare providers to create high ,uality, educational

videos to ma)e use of 'ou(ube as an information platform and to 4counteract any misleading information5 presented in other videos !2#$ 6 second study yielded relatively similar results, finding that out of the 1%% videos revie ed on the sub/ect of sinusitis, only *.> of them provided some useful information!6#$ (he remaining ..> ere deemed to contain little to no useful information and of these videos, 27> of them ere actually thought to contain 4potentially misleading or even as an unreliable source of

dangerous information5$ (his report therefore concluded that the site

information on sinusitis!6#$ (he study also sho ed that the videos uploaded by medical professionals or videos that came from health information than those ebsites proved to contain more useful information

hich came from individual 'ou(ube contributors!6#, supporting the suggestion made by

the first paper that there needed to be more healthcare professionals ma)ing use of 'ou(ube as a source of patient information$ ?urthermore, t o separate studies ere carried out loo)ing into videos on 'ou(ube pertaining to

paediatric tonsillectomy and adenotonsillectomy and both came to relatively similar results despite being carried out almost 1% months apart$ =ne concluded that out of the 1.6 videos revie ed, only 2.$6> ere /udged to contain very@moderately useful information !.#$ (he other study found that only 6 out of the .. videos that ere revie ed scored highly in both accuracy and audio@visual ,uality !7#$ Aoth these studies sho that there is ,uality content available on 'ou(ube but that patients loo)ing for information about paediatric adenotonsillectomy ill 4generally encounter lo ,uality

information5!7#$ (his sho s a need for careful scrutiny of the content on 'ou(ube and advice to patients and health"information consumers as to hich of the videos are misleading$ I finally loo)ed at a systematic revie of methods for studying 'ou(ube videos about consumer

health$ It loo)ed at a total of && studies and highlighted several ea)ness in some of the studies that

* have already been reported and in doing so it provides valuable advice to people loo)ing to carry out a study involving the revie ing of 'ou(ube content!1%#$

The ethical perspectives of the research being undertaken 6lthough * of the reports involved in the systematic revie carried out by Sampson et al!1%# of

similar pro/ects to this one did claim to have received ethical approval from the I1A, the vast ma/ority of them deemed it unnecessary and so in discussion as decided that ethical consideration as not re,uired$ ith the supervisor of the pro/ect, it

The research methodolog 6s the revie ing process involved some degree of personal opinions about the videos, it inappropriate to use a positivist methodology$ (herefore it as decided, having discussed as

ith our

supervisor, that a Bnet in the evidenceB approach ould be more suitable$

Research methods (his revie of the videos on 'ou(ube aimed to cover a broad range of topics that are important to

e-plore hen learning about 12(s$ In order to ensure that this as achieved, the chapter titles from t o boo)s ere used to act as a guide$ (he first of hich as BCo to design analyse and report as a boo) by +achin and

cluster randomised trialsB by Walters and 2ampbell and the second

?ayers entitled B1andomiDed 2linical (rials: :esign, Eractice and 1eportingB$ (hese titles covered everything from the simple practicalities of hat the phases of a clinical trial are to the far more

technical aspects such as 2=<S=1( guidelines and statistical analysis$ Proforma Creation (he first step revie as to design a proforma for revie ing videos$ It ith the pro/ectBs supervisor there as decided from the literature

and consultation

ere several important aspects of the

. videos that ould need to be included in the proforma$ Aasic elements of a video ere included

such as the topic of the video, video F1; and an overall rating of the video$ It

as also deemed

important to include . further pieces of informationG video length, video source, video style, audio" visual ,uality and comple-ity$ Ay including these factors it enabled the study to loo) at different elements of the videos and note any patterns as to hat made a video score highly overall$ (he final category as the videoBs ran) hich as included for administrative purposes$ 0Sho n belo is an e-ample of a completed proforma for one of the videos e revie ed3

7 Search Criteria (he method for searching for videos to revie as designed to reflect the ay in hich someone

interested in learning about 12(s might also approach the tas)$ (he first step

as to directly type

the topic of the chapter into 'ou(ubeBs search bar$ (hen all of the videos on the first page of results ere revie ed$ 9ach page on average contained 2% videos ho ever often several of the videos ould not directly related to the topic being searched for and here hence e-cluded and not ould be applied to

revie ed$ In the absence of any suitable videos the same search method HoogleBs video searching tool$ 6ny videos that

ere not revie ed during the 'ou(ube search that ere then also

sho ed up on the first page of results during this search 01% results per page3 revie ed in an attempt to find as many ,uality videos as possible$ Calibration of Results 6s there ere t o people or)ing on this pro/ect the topics

ere split evenly bet een the t o ea)nesses presented by the as that very

revie ers$ Co ever, before starting the revie ing process, one of the systematic revie fe

by Sampson et al regarding previous pro/ects similar to this one

of them described any calibration e-ercises to train the individual revie ers to appreciate the as therefore suggested by the pro/ectBs supervisor that a BAland"

same ,ualities in the videos!1%#$ It

altman plotB of the overall video ratings 0out of 1%%3 should be created for a sample of 12 randomly selected and independently revie ed videos$ (he plot is displayed belo $

(his sho ed that the mean difference bet een the ratings

as lo , at &$16,

ith Aen on average ent above J@"1%

scoring videos higher than =llie but the differences bet een the results never hich indicates &$16 J@" 8$82

e had no hugely differing opinions about videos$ (he limits of agreement to be hich sho s that there as random variation that as affecting the results$ (he ere then discussed

e-treme results 0 here the t o revie ers differed the most in their opinions3 loo)ing specifically at

hat caused the differences in the scores$ (his enabled both revie ers to be

confident in their ability to rate videos accurately and fairly$ Gathering our Results (he aim of the pro/ect as to compile a list of the best videos that ere revie ed$ (he Bbest videoB

as defined as the video that had the highest overall rating that e anted to locate any areas useful content in the future$ It hich there

as revie ed for each topic$ (hen

as a lac) of high ,uality videos to facilitate creation of

as therefore important to establish some e-clusion criteria as some

topics had videos that dealt ith the topic, but ere not good enough to be featured in the database$

8 6 video as e-cluded if: It scored belo .% overall rating If the content as deemed to be inaccurate or misleading If the audio"visual ,uality of the video scored belo .

0<A: =ne e-ception made hen a video as of poor audio ,uality but as still very informative3 If the comple-ity of the video scored belo . as added to a separate list

If no video survived the e-clusion criteria for a topic, then the topic hich ill serve as a guide as to

here future content should be created to create a comprehensive

collection of videos about 12(s$ The creation of the website :ue to time restraints it as not possible to create the ebsite before the riting of this report$

Co ever ith the database of the best scoring videos complete, e hope to create the ebsite in the near future but it ill re,uire further input and video creation before it is a complete guide to 12(s$

The results so far !<A: It is important to note that these results are only for the videos and topics revie ed by myself$# It as possible to find videos that survived the e-clusion criteria 0and hence ere eligible to be

included in the database3 for 22 out of a possible .% topics that to note that the ma/ority of these successful videos dealt surrounding 12(s

ere covered 0**>3$ It is important

ith the comparatively BsimpleB topics ere poorly represented in the ere useful in teaching people ere suitable on the

hilst the more in"depth and technical aspects ere found that

database$ (his is sho n in the fact that 12 videos

about the phases of 12(s and survived the e-clusion criteria, topic of random"effects logistic regression$ 6dditionally it

hilst no videos

as clear that videos created by medical professionals and academic establishments

ere the most useful, ith 16 out of the 22 videos 072>3 that made it into the database coming from

1% either of these t o sources$ Shorter and more concise videos, in general, also scored better than longer videos ith 18 out of 22

0I6>3 lasting less than 1% minutes$ (he median length as 22& seconds$ 02alculating a mean of the video lengths as not done due to the range in successful videos caused by t o anomalies$3 (he style of the video sho ed little effect on the rating of the video, ith 1& of the 22 videos 0.8>3 being of videos of intervie s@lectures@tal)s and the remaining 8 0*1>3 being classified as voice" over video presentations$

!iscussion of the research pro"ect and a summar of its potential impact =verall, this pro/ect as a solid but by no means perfect loo) at the videos that ere available

online to educate people about 12(s$ 6s this as a relatively short"term pro/ect, it as not possible to atch every video on every topic and instead it as necessary to limit the search to only the first

page of videos$ (his leaves open the possibility that there are videos available on 'ou(ube that ere not revie ed and could be better than the ones that ere found$ +oreover, it is possible that there are e-cellent videos in e-istence that are hosted on other ebsites other than 'ou(ube that do not get the attention that they deserve due to the dominance of 'ou(ube over the video"sharing mar)et$ (he difficulty that is encountered hen attempting to broaden the

search to include videos from other ebsites is found in the fact that as Hoogle o ns 'ou(ube, any search carried out through Hoogle seems to give a very high priority to results found on 'ou(ube and videos from other video"sharing sites are rarely found$ 6 possibility to improve this search method ould have been to use a different search engine from the beginning but even this ould

not be perfect$ Fpon reflection it is also evident that the search criteria and method used by this pro/ect (he systematic revie method to revie as ea)$

by Sampson et al!1%# highlights that some pro/ects used a Bsno ballingB

videos$ (his involves loo)ing at videos that 'ou(ube recommends after a video

11 has been atched$ Ay adopting this search strategy, a more comprehensive revie of all the

available videos ould have been achieved$ 6n issue videos that as also encountered over the course of the pro/ect, here it became apparent that the ere becoming increasingly

ere coming from searches on both Hoogle and 'ou(ube

more technical$ (o ards the end of the pro/ect I cross"analysed the results found from the same search in Hoogle from one computer 0on hich I searched for all of the videos previously3 and a

separate, independent computer that had not been used in the pro/ect$ (he results of the searches ere different and upon investigation I found that Hoogle and 'ou(ube both tailor their results in accordance ith your searching history as ell as a number of other factors !11#$ (here are ays to

prevent this hich should be implemented if further research is done into this topic$ (he pro/ect suffered from numerous limitations but the results support the findings of several other similar pro/ects that loo)ed at educational videos on different topics !2,."8#$ It suggests the best videos are created by medical and academic professionals and provides further evidence to support the fact that there are useful videos on 'ou(ube but in its current state it cannot be deemed to be a reliable source of information$ (he results of this report should also enable medical professionals to create effective videos in the future by loo)ing at hat currently ma)es a video successful on 'ou(ube$ It is also hoped that the ea)nesses e-posed ith this pro/ect ill help future researchers to create

stronger and more robust studies that attempt to deal ith the revie ing of internet videos$

Conclusion 1$ 6re there accurate, high ,uality videos available on the internet to teach people about 12(s7 (he pro/ect has sho n that there are accurate, high",uality videos available on 'ou(ube and other video"sharing ebsites that are useful in learning about 12(s but there is still a need for further

videos before a comprehensive guide can be created$ In its current form, the database of videos that

12 e ere able to collect is enough to teach someone the basic points about a 12( but ould not

offer much help to someone ho as loo)ing to carry out a 12( of their o n$ 2$ 2an e gather the best videos together into a single learning platform and ill it be useful7 (his pro/ect sho s that it is a challenge to properly revie all available videos on the internet about a particular topic$ (he process is very time consuming and re,uires a very robust search method for the results to be reliable$ &$ Is there scope for further video creation and if so, in hat areas7 'es$ (here ere many areas found here there as little to no useful videos on certain topics$ It is

clear from current literature that the internet is a vital source of information in the modern age !1"2# and thus it ould be advisable for medical professionals and academics to actively ma)e use of

po erful information platforms such as 'ou(ube in providing high ,uality videos to those people ho ish to )no more about 12(s$

1& #ibliograph 1$ 2ooley :, +ancuso 6, Weiss ;, 2oren K$ Cealth"1elated Internet Fse 6mong Eatients of =steopathic Ehysicians$ The Journal of the American Osteopathic Association 2%11G $$$(%): *7&"*I2 2$ +u)e ar S, +ani E, Wu L, ;opeD 1, Shen A$ 'ou(ubeM and inflammatory bo el disease$ Journal of Crohn's and Colitis 2%1&G &('): &82"*%2 &$ http:@@ *$ http:@@ $ale-a$com@topsites 6ccessed 1&$%&$2%1* $clic)top1%$com@2%1&@%I@top"1%"most"popular" ebsites"of"2%1&@ 6ccessed

1&$%&$2%1* .$ Strycho s)y K, <ayan S, ?arro)hyar ?, +ac;ean K$ 'ou(ube: 6 good source of information on pediatric tonsillectomy7 International Journal of Pediatric Otorhinolar ngolog 2%1&G &&((): 872"87. 6$ Aiggs (, Aird K, Carries E, Salib 1$ 'ou(ube as a source of information on rhinosinusitis: the good, the bad and the ugly$ The Journal of !ar ngolog " Otolog 2%1&G $)&(%): 7*8"7.* 7$ Sorensen K, EusD +, ArietD)e S$ 'ou(ube as an information source for pediatric adenotonsillectomy and ear tube surgery$ Otorhinolar ngolog 2%1*G &%($): 6."7% I$ Williams :, Sullivan K, Schneiders 6, 6hmed = et al$ Aig hits on the small screen: an evaluation of concussion"related videos on 'ou(ube$ #ritish Journal of Sports $edicine 2%1*G *%: 1%7"111 8$ Syed"6bdul S, ?ernandeD";u,ue ;, Kian W, ;i ' et al$ +isleading Cealth"1elated Information Eromoted (hrough Nideo"Aased Social +edia: 6nore-ia on 'ou(ube$ Journal of $edical Internet Research 2%1&G $'()): e&% 1%$ Sampson +, 2umber K, ;i 2, Eound 2 et al$ 6 systematic revie consumer health 'ou(ube videos, of methods for studying International Journal of Pediatric

ith implications for systematic revie s$ PeerJ 2%1&G $:

1* e1*7 11$ http:@@ $channeldigital$co$u)@seo"articles@sorry"you"are"not"really"on"page"1"of"

google$html 6ccessed 26$%&$2%1*

1. Annotated #ibliograph

+ukewar S, +ani -, .u /, 0ope1 R, Shen #2 3ouTube4 and inflammator bowel disease2 Journal of Crohn's and Colitis )5$67 &(')8 69):*5)

(his study attempted to find the top 1%% 'ou(ube videos relating to IA:$ It states that previous studies have sho n that the ma/ority of patients ith IA: use the internet as a source of information about their condition$ (he aim of the study therefore, as to assess the source, accuracy and content of the 1%% videos and then collect and assess the demographic information of the vie ers$ =n 27th +arch 2%11, the study analysed the top .% relevant videos found by searching ith the

)ey ords 42rohnBs :isease5, 4Inflammatory Ao el :isease5 and 4Flcerative 2olitis5$ (his process as then repeated a ee) later on & rd 6pril 2%11$ (he study notes that none of the top .% videos

found during the first search reappeared in the top .% results of the second study$ (his is indicative of a ea)ness in the study and consideration should have gone into repeating the searching process

on several further occasions to see if this trend continued$ 6ny videos that ere not in 9nglish, lasted for over 2% minutes or had a poor sound ,uality ere

e-cluded from the process$ 6lthough this saved time, e-cluding videos that e-ceeded 2% minutes is a limitation to the study as there are many very informative and useful videos available of lectures and tal)s that often provide high ,uality information but seldom last for less than 2% minutes$ (he study analysed the videos on over &% variables$ (his as a strength of the study as it provided a broad range of results that could determine factors that made a video popular$ (he results of the study sho ed many things, most notably that videos depicting patientBs e-perience ere the most popular, but not necessarily the most informative$ (he study concludes by calling for more healthcare professionals@hospitals to create videos that ma)e use of the information gathered during the study as to hat made a video about IA: popular on 'ou(ube$

16 #iggs T, #ird ;, <arries -, Salib R2 3ouTube as a source of information on rhinosinusitis8 the good, the bad and the ugl 2 The Journal of Laryngology & Otology )5$67 $)&(%)8 &*9:&'*

(he study carried out in 2%1& by Aiggs et al analysed a sample of videos pertaining to sinusitis found on the video"sharing to try and ascertain ebsite, 'ou(ube$ It assessed 1%% videos and the aim of the study as

hether or not 'ou(ube as a reliable and accurate source of information for a

lay"person to learn about rhinosinusitis$ ?irstly, 'ou(ube as searched using the )ey" ord 4sinusitis5 on the 17 th and 1Ith of 6ugust 2%12 ere then included in the study$ (his presents a slight limitation to the

and the first 1%% results study, as it

ould have almost certainly been better to carry out the search on a single day as the ould li)ely be different due to the ever"changing and

results of searching on separate days evolving nature of the

ebsite$ ?ollo ing this, t o of the authors independently vie ed and

assessed all of the videos, documenting important information about them and grading as BusefulB, Bnot"usefulB, Bslightly misleadingB, Bcompletely misleadingB or Buseful but only to a health professionalB$ (heir results ere then compared and any differing opinions ere discussed$ =verall, the study found that *.> of the videos revie ed contained some useful information$ =f the remaining ..> of videos, 27> of them ere found to be misleading$ It also found that the mean as &6&% in comparison to the Bcompletely misleadingB

number of vie s for the top"graded videos

videos hich had a mean of I%81$ (he number of Bli)esB received sho ed a similar trend$ (he authors conclude by suggesting that the situation could be improved by enabling professionals to highlight inaccurate videos for removal or to implement a method for directing lay"people to credible resources and a ay from inaccurate ones$ (his as a carefully planned and ell thought"out study$ 2onsideration as given to calibration and ,uality control of the scoring system and a ide variety of factors ere assessed$ It also supports

evidence found in previous studies improving generalisability of the results$

17 Str chowsk ;, =a an S, >arrokh ar >, +ac0ean ;2 3ouTube8 A good source of information on pediatric tonsillectom ? International Journal of Pediatric Otorhinolaryngology )5$67 &&(()8 9&):9&'

(his study loo)ed at the videos available on 'ou(ube intended to educate the vie er about paediatric tonsillectomy ith a particular emphasis on the source of the videos$ In total it analysed

1.6 videos$ (his is one of the larger sample siDes that have been assessed in the literature that has investigated 'ou(ube as a source of medical information to lay"people$ (his is one of the main strengths of the study and enables the results to be deemed more reliable$ 6lso by revie ing every single available video on 'ou(ube about the sub/ect, rather than /ust the top hundred or so results, the study ensures that it provides a more complete and comprehensive vie of the videos available$ (he videos videos found ere found by searching 'ou(ube on &1 st Kuly 2%12 using & different search terms$ (he ere then revie ed independently by t o separate researchers and classified on a

scale of Bvery usefulB to Bnot"usefulB or BmisleadingB$ 6ny differences in decisions by the t o revie ers ere resolved through discussion$ (he source of the video as also recorded as being

from Bpatient e-perienceB, BphysicianB, Bsurgical techni,ueB or Bne s reportB$ (he results found that 2.$6> of the videos ere moderately or very useful$ (he videos created by

physicians ere normally rated as being moderately or very useful but only 18> of the videos came from this source$ 66> of the videos ere sourced from patient e-periences and as such as the

most popular$ Co ever the report points out that .%> of the videos that ere classed as Bnot"usefulB or BmisleadingB ere from this source$ (his study ma)es use of a similar rating system for videos that several other studies carried out previous to this one have used$ (his improves the ,uality and generalisability of the evidence produced by the study as although the studies loo)ed at videos on different medical issues, it is easier to compare results and /udge 'ou(ube as a source of health"information as a hole$

1I Sorensen ;, -us1 +, #riet1ke S2 3ouTube as an information source for pediatric adenotonsillectom and ear tube surger 2 International Journal of Pediatric

Otorhinolaryngology )5$*7 &%($)8 (':&5

Sorensen et al conducted this study loo)ing at 'ou(ube as a source of information regarding paediatric adenotonsillectomy and ear tube surgery$ (he pro/ect the methods that patients might use as designed to try and replicate ould be

hen trying to learn about these sub/ects$ It

impossible to completely achieve this as 'ou(ube is constantly changing$ +ore videos are uploaded every day and hence t o different searches carried out on t o different days by t o different computers ould produce t o different search results$ Co ever the pro/ect did attempt to

counteract this by performing the search of 'ou(ube on t o separate days to try and give a broader range of videos to revie that the patients might encounter and furthermore by using 'ou(ubeBs

default search settings to ensure that their results ere as similar to the results found by a lay"person as they could manage$ ( o researchers then independently revie ed all of the videos that ere selected and loo)ed at

many different aspects of the videos such as audio"visual ,uality, accuracy of content and comprehensiveness$ (hey did not state that they loo)ed at video source hich is commonly

assessed among pro/ects similar to this one$ (he report also doesnOt mention any calibration e-ercises to ensure the researchers are revie ing videos to the same standard hich should have

been carried out before the videos ere revie ed $ (heir search only yielded a total of .. videos on adenotonsillectomy and *7 about ear tube surgery suggests the possibility that some videos hich is a relatively small sample siDe$ (his

ere missed by the search that could have influenced the ould have improved the search method and

results$ Eerhaps employing a Bsno ballingB strategy!1%# given a more complete set of results$ (he results of this study found that very fe

of the videos that

ere revie ed

ere found to have

18 both accurate information and high audio"visual ,uality, ith only 6 of the 1%2 videos scoring

highly in both the categories$ (he study therefore concluded that someone trying to use 'ou(ube as a source of information about either pediatric adenotonsillectomy or ear tube surgery li)ely encounter lo ,uality information$ ould most