Sunteți pe pagina 1din 15

AnalyzingTwitterSentiment

ofthe2016PresidentialCandidates
DelennChin,AnnaZappone,JessicaZhao

SECTION1:TASKDEFINITION

1.1Introduction
Electionsempowercitizenstochoosetheirleaders.Itgivesallanopportunityforequal
voiceandrepresentationinourgovernment.Democracyisgovernmentforthepeople,
andbythepeople,whichmeansgovernmentleadersaredeterminedbyparticipationin
elections.Asweapproachthe2016Novemberpresidentialelection,thepublic
sentimenttowardscandidateswillinfluencethefutureleaderofourcountry.Weare
interestedinhowthepublicviewsthetopelectioncandidates,namelyDonaldTrump,
HillaryClinton,BenCarson,andBernieSanders.Feelingstowardscandidatesfluctuate
quicklyasinterviews,debates,responsestoglobalevents,andotherissuescometo
front.Toachievealarge,diversedatasetofcurrentpublicopinionsonthecandidates,
wedecidedtouseTwitter.Twitterprovidesuswithliveaccesstoopinionsaboutthe
electionacrosstheglobe.Weareanalyzingrecenttweetsregardingthetopcandidatesin
the2016electiontopredictthepublicsentimentstowardseachcandidate.Ourproject
attemptstosolvetherealworldproblemofunderstandingthecurrentsentiment
towardselectioncandidatesbasedoffthepublicsliveopinionsandemotionsrather
thanoffofsmaller,localizedpollstypicallydonebymainstreammediacorporations.
Furthermore,weareaimingforalarger,morerelevantassessmentofsentiments
towardscandidatesthanthemediaandpollshave.Weareexpandingonsimplypositive
ornegativesentimenttowardsacandidatebyclassifyingtweetsintofiveemotions
happy,sad,fear,laughter,andangry.

1.2Scope
Wefocusedontweetsregardingthepresidentialelectionanditscandidates.We
assessedthesentimenttowardseachcandidatebasedoffrecenttweets.Wedownloaded
over300,000tweetsusingkeywords.Ourkeywordsforthesearcheswerepolitics,
politicalcandidates,orthefullnameofa2016presidentialcandidate.Ofthe
approximately300,000tweetsdownloaded,weobtainedalmost3,000tweetsthat
containedanemojiinthislist:

1.3Input
Togetourinputdata,wedownloadedtweetsusingtheTwitterAPIfoundfromasearch
ofourkeywordsorthefullnameofa2016presidentialcandidate.Wethenusedonly
thetweetsthatcontainanemojiinthe1.2listabove.Forbothtestingandtraininginput,
weusedasinputthetextoftweetsstrippedoftheemojis.Weareusingthetextofthe
tweetstrippedoftheemojisin1.2toavoidoverfittingtoemojisandtobaseour
predictionoffthetextsentiment.Beforestrippingtheemoji,wecreateamappingofthe
tweettotheclassinwhichthefirstemojiofthetweetbelongsforeithertrainingor
assessingclassificationaccuracy.

TrainingInput
Weused2,000ofourdownloadedtweetstotrainourclassifier.

TestingInput
Weusedtheremainingtweetstotestourclassifier.Weranourclassifieronat
least150tweetspercandidate,withover900totaltweetstested.

1.4Output
TrainingOutput
Ouroutputisthesentimentstringthatthefirstemojiinthetweetismappedto.
Forexample,atweetIloveHillary!

wouldbemappedtoHappysince

isthefirstemojiandanelementoftheHappylist.

TestingOutput
OuroutputisthesentimentstringthatthetweetmostexpressesamongHappy,
Sad,Angry,Laughter,andScaredafterfittingourclassifiertothetraining
data.

1.5Evaluation
Toassessourprojectssuccess,wecalculatetheaccuracyofourpredictedsentimentsby
comparingtheoutputtedclassificationsforthetesttweetswiththecorrectsentiments
thatwedetectedbyusingtheemojis.

SECTION2:INFRASTRUCTURE

Asfortheinfrastructureforourmachinelearningproject,wecollecteddataand
processeditforsentimentclassification.Weoutlineourchallengesanddecisionsbelow.
2

2.1DataSource:
Thefirstchallengearoseindeterminingourdatasource.Electionsentimentschange
quickly,ascandidatesdropoutorgainpopularitybasedoneventsanddebates.We
decidedtouseTwittertohandlethetemporalnatureofelectionsandstillbeableto
collectrelevantdataaboutthecandidates.AfterdeterminingTwitterasourdatasource,
westillfacedchallengeswithourvolumeofdata.Whileweultimatelywereabletoget
almost3,000tweetswithourchosenemojisfortheproject,wewereshockedathowfew
tweetshademojis.Evenwhenwedownloadedtweetsoverdays,wewouldendupwith
lessthan1/100ofthosetweetshavingemojisinour1.2list.

2.2ClassifySentiment:
Wethenhadtodeterminehowwewouldextractthesentimentfromatweet.Since
handclassifyingthousandsoftweetswouldbeincrediblytimeconsuming,wedecided
touseemojisasouroracleandonlyconsidertweetswithemojis.Thephenomenainthe
datathatwearecapturingishowTweettextcorrelatestoemojis.Weselectedemojis
relevanttofeelingsaboutthecandidatesandthencategorizedthembycreatinga
mappingofemojistosentimentsasfollows:

2.3ProcessText
Finally,weprocessedourtweetinputtext.Wefoundideasandcodeexamplesfrom
otherTwittersentimentsprojects,citedbelow.Weconvertedallofourtexttolowercase,
removedhashtagsbeforewords,andremovedextraspacesinthetext.SinceTwitteris
verycasual,peopleoftenincludemultiplesofthesamelettersinaword,suchassooooo
funnnyyy.Tohandlethesecases,wereplacedmorethantwoofthesameconsecutive
letterswithjusttwoofthatletter.WealsoignoredallURLsandusernamessinceURLs
andusernamesdonttypicallyrelatetotheemotionexpressedinthetweet.
Furthermore,wefoundalistofcommonwordsthatdonotexpresssentiment
(Janardhana).Thelistincludeswordslikebe,at,the,etc.Allwordsinatraining
tweetfoundonthislistwereignoredandnotaddedtothefeaturevectorwithaweight.

WhileusingTwitterandonlyusingtweetswithemojisbiasourresults,wefeltthese
tradeoffswerenecessarytopursuesuchalargescale,temporalproject.Twitterbiases
3

ourprojecttowardsanalyzingfeelingsofTwittersyounger,moretechsavvyaudience
andopinionsthatcanbeexpressedusingstrictwordlimits.Similarly,onlyusingtweets
withemojismayfurtherbiasourprojectstowardsyouthandmoreemotionalrather
thanserioustweets.

SECTION3:APPROACH

3.1Baseline:
Asourlowerbound,wesetourbaselineasrandomlypredicting1ofthe5sentiments,
whichgives20%accuracy.

3.2Oracle:
Asourupperbound,weusedthecorrectsentimentclassificationbasedonthefirst
emojiandseeinghowwellourclassifierdoespredictingthesentimentexpressedbythe
emoji.Thismeanswedefinethecorrectsentimentofthetweetasthefirstemojiinthe
tweet.Wecalculatehowwellourclassificationalgorithmperformsrelativetothis
oracle.

3.3FeatureExtraction:
Weusedthebagofwordsmodeltocreateoursparsefeaturevector.Everydistinctword
fromourtrainingtweetsisaddedtoafeaturevectorexcludingURLs,usernames,and
wordsonourlistofcommonwordsthatdonotexpresssentimentasdescribedin2.3.

3.4Modelling:
Thereweremanychallengesandthingstoconsiderinhowtobestmodelourtask.We
realizedthattherearemanyfeelingsthegeneralpublicholdstowardscandidates,sowe
wantedtoaccountformorethansimplypositiveornegativesentimentsachieved
throughbinaryclassificationorevenarangebetweenpositiveandnegativethrough
linearregression.Multiclassclassificationallowsustocategorizethetweetsinto
multipledistinctclassestohandleamorespecificrangeofemotions.Wemodelledthe
taskasamulticlassclassificationproblemtoaccountforanarrayofdifferentemotions
thatpeoplecouldrealisticallyfeelregardingtheelectioncandidates.

WetestedthreealgorithmsSupportVectorMachines,NearestNeighbors,andNaive
Bayesclassification.Thereweremanytradeoffsbetweenaccuracyandefficiency.The
tradeoffsbetweenaccuracyarevisibleinFigure1below.


FIGURE1

NaiveBayes
NaiveBayesperformedalmostaswellasSVMonclassifyingtweetswith49.02%
accuracy.WechosetouseBernoulliasoureventmodelfortheNaiveBayes
classifierbecauseourfeaturevectorisusingabagofwordsmodel,wherethe
valuesarebinary.WeconsideredusingtheMultinomialeventmodelwhich
countswordoccurrences.However,tweetsaresuchshorttextsthatrarely
containmultipleoccurrencesofthesameword,thusBernoulliwasadequatefor
ourproject.Adisadvantagewouldbethatconditionalindependenceoftendoes
notholdinrealityfortext,yetthismodelperformedfairlywellforourproject.

SupportVectorMachine
WeseethatSVMwasourmostaccurateclassifier.Wechosetouseanmulticlass
SVMwithaonevsonescheme,whichcreatesonebinaryclassifierforevery
differentpairofclasses,andthenusesavotingmechanismtoultimatelychoose
oneclass.OnedrawbackofSVMisthatifthesizeofthefeaturevectorislarger
thanthenumberoftrainingsamples,ittendstooverfitonthetrainingdata,
whichlowerstheaccuracyonthetestingdata.AnotherdrawbackofSVMoccurs
withtiesinvoting,whichcouldposeanissuetoourprojectduetothesmallsize
ofourdataset.Nonetheless,SVMcorrectlyclassified446tweets,withabout
49.22%accuracy.
5

NearestNeighbors
Wesetour
k
valueto6,whichgaveusmostaccurateresults.The
k
Nearest
Neighborsmodellooksatthe
k
closestneighborstothegivensampleandusesa
votingmechanismtoclassifythegivensampleintotheclasswiththemajorityof
votes.Thisisanexampleoflazylearning,becausenothingisdonewiththe
trainingdatauntilthemodeltriestoclassifythetestdata.WhileKNNisoneof
thesimplestmachinelearningalgorithms,itdidnotperformaswellonour
dataset.WethinkKNNsloweraccuracyof45.99%couldrelatetothesizeofthe
datasetanddifferencesbetweenwordsinourtrainingdatasetandtestingdataset
resultinginworseplotting.Wefoundthatthetrainingtweetsweaddedfrom
keywordsofpoliticsandpresidentialcandidatesuseddifferent,moreformal
lexiconsthanthatofthetweetsfromapresidentialcandidatesnameasthe
keyword.

Overall,thereweremanytradeoffsbetweentheNaiveBayes,SupportVectorMachines,
andNearestNeighborsalgorithms.Intermsofefficiency,NaiveBayeswasthemost
efficientintermsoftime,whiletheKNNalgorithmwastheslowestbecauseittakes
longertoclassifyeachtestdatapoint.Intermsofaccuracy,SupportVectorMachines
wasmostaccurateand
k
NearestNeighborswasleastaccurate.

SECTION4:LITERATUREREVIEW

Weweresurprisedtofindthattherehasnotbeenmuchworkrelatingtoemojisand
sentimentclassification.Mostworkiscenteredonthepolarityofsentimentratherthan
adiversityofsentiments.WefoundasimilarIndonesianstudythatusedhashtags,
emojis,emoticons,andadjectivesintweetsforemotiondetectionoffiveemotions
happy,angry,fear,sad,andsurprise(Arifinetal.).Sincetheywereclassifyingmultiple
emotions,theysawthesameimbalancethatweencounteredwithsomeemotions
havingalmostdoublethenumberoftweetsthanothers.Ourmodelsusedsimilar
algorithms,thoughtheyusedmatrixmanipulationbeforeusingnearestneighbors.We
foundanothersimilarstudydoneinIndiathatdoesemotionestimationusingdirect
words,emojis,andsmileys(Yeoleetal.).WhiletheresearchersconsideredusingSVM,
theyultimatelyusedasimilarapproachtoSVMofArtificialNeuralNetworkswith
Jaccardsimilarity.Ourprojectsmodellingandapproachseemscomplementarytothe
Indiaemotionresearch.Intheiruseofemojistoestimatesentiments,ourworkis
similartoboththeseexistingworks.However,Yeolesstudyusedemojisonlyin
predictingtheemotionandusedanexistingclassifiedAmazonsentimentdatasetfor
6

trainingwhereasweusedthefirstemojiasouroracletocreateourownsentiment
datasetfortraining.Furthermore,wethinkourprojecthadamorerealworld
applicationasweappliedoursentimentclassificationtotheelectionandonly
consideredtweetsrelatingtospecificcandidates.Wefoundaninterestingproject
relatingtousingemojistoobtaintheemotionalstateofuserstothenbetterpredict
musicrecommendationsfortheuser(Ferwerda,Schedl).Whilethisprojectalsohada
realworldapplicationoftheiremotionalclassification,ourprojectisstillfairly
orthogonaltotheirapplication.

SECTION5:ERRORANALYSIS

Ourmainquestionswithoursystemrelatedtolanguageforourfeaturevector.Since
Tweetsuseauniquedialect,muchofourinfrastructurerelatedtoprocessingtextand
streamliningtweetsforimprovedclassification.Wedesignedourexperimentstotest
ourhypothesisofhowtweetsshouldbelinguisticallyprocessedtomaximize
classificationaccuracy.Asourcontrolaccuracy,weusedSupportVectorMachinesaswe
hadachievedourhighestclassificationaccuracywiththisalgorithm.

FIGURE2

5.1Doesignoringpunctuationimproveourclassification?
Inourprocessing,wehadtrimmedpunctuationfromtweetsbecauseweassumeditwas
notveryrelatedtosentiment.Wefoundthatignoringpunctuationdoesimproveour
classification,butonlyslightly.ForHillaryClinton,DonaldTrump,andBenCarson,
includingpunctuationyieldedequallyeffectiveorbetterclassification.However,itdida
muchworsejobclassifyingwithpunctuationforBernieSanders.Weweresurprisedthat
keepingorignoringpunctuationdoesnothaveahugeimpactonouraccuracy.

5.2DoesignoringURLsandusernamesimproveourclassification?
WehypothesizedthatignoringURLsandusernameswouldimproveourclassification
accuracyastheyseemunrelatedtosentiment.However,weweresurprisedtofindthat
includingURLsandusernamesactuallyimprovedclassificationforDonaldTrumps
tweets,thoughitdidnotfortheothercandidates.Overall,ourhypothesiswascorrect
andignoringURLsandusernamesgaveushigherclassificationaccuracy.

5.3Doesignoringcommonwordsunrelatedtosentimentimproveour
classification?
Weusedalistofwordsthatarecommon,butdonotdirectlyexpressorrelateto
sentimentfromotherTwittersentimentprojects.Ignoringthesewordsthatdidnt
directlycorrelatetosentimentsveryslightlyimprovedourclassification,butthe
influencewasnotnearlyaslargeasweexpectedittobe.

Ourmaintakeawayfromourexperimentswasthatourprocessingofwhatwethought
wereextraneousfeaturesinthetextdidnothaveaslargeofanimpactonour
classificationresultsasweexpected.

SECTION6:DATAANALYSIS

6.1Overview
Weconstructedmapsfromthesentimentsofthetweetswecollected,usingmetadatafor
eachtweettofindthelocationsthatusersinputtedintotheirprofile.TwittersAPIdoes
haveanentryforlatitudeandlongitudecoordinates.However,itrequirestheuserto
enablethissetting,andwefoundthatlessthan1%ofusershadenabledthisfeature.We
thereforeusedtheuserinputtedlocation.Thisrequireshandlabellingeachlocationto
determinethestatetheuserlivedbecausetheuserinputtedlocationwasinavarietyof
formats.Somepeopleusedareacodes,universitynames,andairportabbreviations,
amongotherthings,todescribetheirlocation,sowelookeduptheirinputted
8

informationonGoogleandusedthefirstresult.Therewere,ofcourse,somelocations
whichwewereunabletoascertainthestateandsomelocationsthatwerentintheUS,
whichweignored.Withourhandlabeleddata,wemadeaheatmapforeachcandidate
whichshowstheconcentrationoftweetsfromeachstaterelativetothetotalnumberof
tweetswehaveaboutthatcandidate.Lastly,foreachstatewithatleastonetweet,we
addedtheemojirepresentingthedominantemotionofthetweetsfromthatstate.

6.2AnalysisofMaps
Wefoundthatforstateswithhighconcentrationsoftweets,therewasalwaysone
emotionthatreceivedmorethanthreequartersofthetweetsentiments.Evenforthe
lightbluestatesfromwhichweonlygotonedatapoint,itisfairlylikelythatthis
emotionwouldhavereceivedmajoritysentimentifwehadgottenmoredatapointsfor
thatstatesinceoneemotiontypicallydominatedsostrongly.Sincewedidnotscalethe
numberoftweetsbystatepopulation,California,Texas,NewYork,andFlorida
consistentlyhavethemosttweets.Thishindersourabilitytocomparecandidates
popularitybetweenstatesbasedonnumberoftweets,sinceCaliforniawillalwayshave
moretweetsaboutanycandidatethanIdahowhichhadzerotweetsforeverycandidate.

Ifwedisregardretweets,laughterwasthemostcommonemojifromtheuniquetweets
foreverycandidate.Interestingly,thisisnotobviousjustfromlookingattheAppendix
mapsbecauseforstateswherelaughterwas
not
thedominantsentiment,therewerestill
manynumberoflaughtertweetseventhoughtheywereultimatelyoutweighedbya
largernumberoftweetsexpressingadifferentsentiment.Essentially,eveniflaughter
didntdominateastatessentiment,itspopularityasasentimentwasstillrelatively
high.Inaddition,ourmapcountsincluderetweets,soformanycases,therewasone
tweetofaparticularsentiment,forexample,asadtweet,thatwasretweetedmany
times.Forourmaps,wecountedeachretweettocapturepopularsentimentandshow
wherethatsamesentimentwasfelt.ThisismostapparentwhenlookingatDonald
Trumpsmap,whichshowsahugemajorityofthesadsentiment,causedbyrelatively
fewsadtweetsthatwerethenretweetedmanytimes.DonaldTrumpsmapisalsoan
exampleofthetimelinessoftweetsingeneral,sincewecollectedallhisdataontheday
oftheSanBernardinoshootings.Wecollectedsomeothercandidatesdataonthisday
also,butdidnotgetnearlyasmuchofaneffect.WesuspectthatDonaldTrumpdidor
saidsomethingextremelyinappropriateinrelationtothisevent,whichcausedafewsad
tweetswhichwerethenmassivelyretweeted.Anotherexampleoftimelinessandan
eventspotentialimpactwasvisibleafterBenCarsonmispronouncedHamasas
hummusataRepublicanJewishCoalitionforuminD.C.Thisresultedinahugespike
inlaughtertweetsandretweetstowardsBenCarson.

APPENDIX

MapKey:
Happy:
Sad:

Angry:

Laughter:
Scared:

BenCarsonSentimentMap

10

HillaryClintonSentimentMap

BernieSandersSentimentMap

11


DonaldTrumpSentimentMap

TestDataStatistics

DonaldTrump
Keywordqueries:trump,donaldtrump
Totaltweetscollectedfortestingdata=28902
Uniquetweetsoutofthosecollected=16507
Totaltweetswithemojisusedfortesting=256
Accuracy:
KNN:0.4453125
SVM:0.4453125
Includepunctuation:0.4453125
IncludeusernameandURLs:0.453125
Includeallwords:0.453125
NaiveBayesBernoulli:0.44921875
BenCarson
Keywordqueries:bencarson
Totaltweetscollectedfortestingdata=21416
Uniquetweetsoutofthosecollected=13869

12

Totaltweetswithemojisusedfortesting=152
Accuracy:
KNN:0.473684210526
SVM:0.532894736842
Includepunctuation:0.539473684211
IncludeusernameandURLs:0.532894736842
Includeallwords:0.532894736842
NaiveBayesBernoulli:0.532894736842
HillaryClinton
Keywordqueries:hillaryclinton,hillary
Totaltweetscollectedfortestingdata=53974
Uniquetweetsoutofthosecollected=29347
Totaltweetswithemojisusedfortesting=194
Accuracy:
KNN:0.448453608247
SVM:0.505154639175
Includepunctuation:0.505154639175
IncludeusernameandURLs:.5
Includeallwords:0.494845360825
NaiveBayesBernoulli:0.5
BernieSanders
Keywordqueries:bernie,berniesanders
Totaltweetscollectedfortestingdata=40797
Uniquetweetsoutofthosecollected=25065
Totaltweetswithemojisusedfortesting=305
Accuracy:
KNN:0.472131147541
SVM:0.485245901639
Includepunctuation:0.468852459016
IncludeusernameandURLs:0.468852459016
Includeallwords:0.47868852459
NaiveBayesBernoulli:0.47868852459

13

WorksCited

Aly,Mohamed."SurveyonMulticlassClassificationMethods."
Cs.utah.edu
.N.p.,n.d.
Web.8Dec.2015.
Arifin,AgusZ.,YuitaA.Sari,EvyK.Ratnasari,andSitiMutrofin."EmotionDetectionof
TweetsinIndonesianLanguageUsingNonNegativeMatrixFactorization."
InternationalJournalofIntelligentSystemsandApplications(IJISA)
6.9
(2014):5461.
IJISA
.MECSPublisher,Aug.2014.Web.29Nov.2015.
Ferwerda,Bruce,andMarkusSchedl."EnhancingMusicRecommenderSystemswith
PersonalityInformationandEmotionalStates:AProposal."(2011):n.pag.
JKU
ComputationalPerception
.JohannesKeplerUniversity,2014.Web.29Nov.
2015.
Janardhana,Ravikiran."HowtoBuildaTwitterSentimentAnalyzer."
Ravikiranj.net
.
Nikola,08May2012.Web.06Dec.2015.
Kim,Jinho,ByungsooKim,andSilvioSavarese."ComparingImageClassification
Methods:KNearestNeighborandSupportVectorMachines."
Wseas.us
.N.p.,
n.d.Web.8Dec.2015.
Nakhaeizadeh,Gholamreza.
StatisticalDataMining
.N.p.:n.p.,n.d.
UlmUni
.Ulm
University,2007.Web.8Dec.2015.
"PropertiesofNaiveBayes."
TheStanfordNaturalLanguageProcessingGroup
.
CambridgeUniversityPress,07Apr.2009.Web.09Dec.2015.

14

Tai,KaiS."CS229FinalProjectSentimentAnalysisofTweets:BaselinesandNeural
NetworkModels."(n.d.):n.pag.StanfordUniversity,13Dec.2013.Web.12Nov.
2015.
Yeole,AshwiniV.,P.V.Chavan,andM.C.Nikose."OpinionMiningforEmotions
Determination."
InnovationsinInformation,EmbeddedandCommunication
Systems(ICIIECS)2015InternationalConference
(2015):15.
IEEEXplore
.20
Mar.2015.Web.29Nov.2015.
Zhang,Hao,AlexanderC.Berg,MichaelMaire,andJitendraMalik."SVMKNN:
DiscriminativeNearestNeighborClassificationforVisualCategoryRecognition."
(n.d.):n.pag.Web.8Dec.2015.

SourcesandOutsideLibraries
TwitterAPI
GoogleMapsAPI
Tweepy
Scikitlearn
AmMap

15

S-ar putea să vă placă și