8/19/13

Artificial Intelligence and What Computers Still Don't Understand : The New Yorker

HardtoCrack:TheGovernmentsEncryptionConundrum Main HomophobiainRussiaFindsaNewMedium August16,2013

WhyCantMyComputerUnderstandMe?
Postedby GaryMarcus Print More

Share
Close Reddit LinkedIn Email

www.newyorker.com/online/blogs/elements/2013/08/why-cant-my-computer-understand-me.html?printable=true&currentPage=all

1/6

8/19/13

Artificial Intelligence and What Computers Still Don't Understand : The New Yorker

HectorLevesquethinkshiscomputerisstupidandthatyoursis,too.SiriandGooglesvoicesearchesmaybeabletounderstandcannedsentenceslikeWhat moviesareshowingnearmeatsevenoclock?,butwhataboutquestionsCananalligatorrunthehundredmetrehurdles?thatnobodyhasheardbefore? Anyordinaryadultcanfigurethatoneout.(No.Alligatorscanthurdle.)ButifyoutypethequestionintoGoogle,yougetinformationaboutFloridaGators trackandfield.Othersearchengines,likeWolframAlpha,cantanswerthequestion,either.Watson,thecomputersystemthatwonJeopardy!,likely wouldntdomuchbetter. Inaterrificpaperjustpresentedatthepremierinternationalconferenceonartificialintelligence,Levesque,aUniversityofTorontocomputerscientistwho studiesthesequestions,hastakenjustabouteveryoneinthefieldofA.I.totask.Hearguesthathiscolleagueshaveforgottenabouttheintelligencepartof artificialintelligence. LevesquestartswithacritiqueofAlanTuringsfamousTuringtest,inwhichahuman,throughaquestionandanswersession,triestodistinguishmachines frompeople.Youdthinkthatifamachinecouldpassthetest,wecouldsafelyconcludethatthemachinewasintelligent.ButLevesquearguesthattheTuring testisalmostmeaningless,becauseitisfartooeasytogame.Everyyear,anumberofmachinescompeteinthechallengeforreal,seekingsomethingcalledthe LoebnerPrize.Butthewinnersarentgenuinelyintelligentinstead,theytendtobemorelikeparlortricks,andtheyrealmostinherentlydeceitful.Ifaperson asksamachineHowtallareyou?andthemachinewantstowintheTuringtest,ithasnochoicebuttoconfabulate.Ithasturnedout,infact,thatthewinners tendtouseblusterandmisdirectionfarmorethananythingapproximatingtrueintelligence.Oneprogramworkedbypretendingtobeparanoidothershave donewellbytossingoffonelinersthatdistractinterlocutors.ThefakeryinvolvedinmosteffortsatbeatingtheTuringtestisemblematic:therealmissionof A.I.oughttobebuildingintelligence,notbuildingsoftwarethatisspecificallytunedtowardfixingsomesortofarbitrarytest.
www.newyorker.com/online/blogs/elements/2013/08/why-cant-my-computer-understand-me.html?printable=true&currentPage=all 2/6

8/19/13

Artificial Intelligence and What Computers Still Don't Understand : The New Yorker

Totryandgetthefieldbackontrack,Levesqueisencouragingartificialintelligenceresearcherstoconsideradifferenttestthatismuchhardertogame, buildingonworkhedidwithLeoraMorgensternandErnestDavis(acollaboratorofmine).Together,theyhavecreatedasetofchallengescalledthe WinogradSchemas,namedforTerryWinograd,apioneeringartificialintelligenceresearcheratStanford.Intheearlynineteenseventies,Winogradasked whatitwouldtaketobuildamachinethatcouldansweraquestionlikethis: Thetowncouncillorsrefusedtogivetheangrydemonstratorsapermitbecausetheyfearedviolence.Whofearedviolence? a)Thetowncouncillors b)Theangrydemonstrators Levesque,Davis,andMorgensternhavedevelopedasetofsimilarproblems,designedtobeeasyforanintelligentpersonbuthardforamachinemerely runningGooglesearches.SomearemoreorlessGoogleproofsimplybecausetheyareaboutmadeuppeople,who,bydefinition,havefewGooglehits: JoanmadesuretothankSusanforallthehelpshehadgiven.Whohadgiventhehelp? a)Joan b)Susan (Tomakethingshardertogame,analternativeformulationsubstitutesreceivedforgiven.) OnecantsimplycountthenumberofWebpagesinwhichpeoplenamedJoanorSusangaveotherpeoplehelp.Instead,answeringthisquestiondemandsa fairlydeepunderstandingofthesubtletiesofhumanlanguageandthenatureofsocialinteraction. OthersareGoogleproofforthereasonthatthealligatorquestionis:alligatorsarereal,buttheparticularfactinquestionisntonethatpeopleusuallycomment on.Forexample: ThelargeballcrashedrightthroughthetablebecauseitwasmadeofStyrofoam.WhatwasmadeofStyrofoam?(Thealternativeformulation replacesStryrofoamwithsteel.) a)Thelargeball b)Thetable * Samtriedtopaintapictureofshepherdswithsheep,buttheyendeduplookingmorelikegolfers.Whatlookedlikegolfers? a)Theshepherds b)Thesheep Theseexamples,whichhingeonthelinguisticphenomenonknownasanaphora,arehardbothbecausetheyrequirecommonsensewhichstilleludesmachines andbecausetheygetatthingspeopledontbothertomentiononWebpages,andthatdontendupingiantdatasets.
www.newyorker.com/online/blogs/elements/2013/08/why-cant-my-computer-understand-me.html?printable=true&currentPage=all 3/6

8/19/13

Artificial Intelligence and What Computers Still Don't Understand : The New Yorker

Morebroadly,theyareinstancesofwhatIliketocalltheLongTailProblem:commonquestionscanoftenbeansweredsimplybytrawlingtheWeb,butrare questionscanstillstymiealltheresourcesofawholeWebfullofBigData.MostA.I.programsareintroubleifwhattheyrelookingforisnotspelledout explicitlyonaWebpage.ThisispartofthereasonforWatsonsmostfamousgaffemistakingTorontoforacityintheUnitedStates. Thesameproblemcomesupinimagesearch,intwoways:manykindsofpicturesarerare,andmanykindsoflabelsarerare.Therearemillionsofpicturesof catslabelledcatbutaGoogleImagesearchforscubadiverwithachocolatecigaretteyieldsalmostnothingofrelevance(dozensofpicturesofcigars,pin upgirls,beaches,andchocolatecakes)eventhoughanyhumancouldreadilysummonamentalimageofanappropriatelyadorneddiver.Ortakethephrase righthandedman.TheWebisfilledwithpicturesofrighthandedmenengagedinunmistakeablyrighthandedactions(likethrowingabaseball),whichany humanworkinginaphotoarchivecouldrapidlysortout.Butveryfewofthosepicturesarelabeledassuch.Asearchforrighthandedmaninsteadreturnsa grabbagofsportsstars,guitars,golfclubs,keychains,andcoffeemugs.Somearerelevant,butmostarenot. Levesquesaveshismostdamningcriticismfortheendofhispaper.ItsnotjustthatcontemporaryA.I.hasntsolvedthesekindsofproblemsyetitsthat contemporaryA.I.haslargelyforgottenaboutthem.InLevesquesview,thefieldofartificialintelligencehasfallenintoatrapofserialsilverbulletism, alwayslookingtothenextbigthing,whetheritsexpertsystemsorBigData,butneverpainstakinglyanalyzingallofthesubtleanddeepknowledgethat ordinaryhumanbeingspossess.Thatsagargantuantaskmorelikescalingamountainthanshovelingadriveway,asLevesquewrites.Butitswhatthe fieldneedstodo. Inshort,Levesquehascalledonhiscolleaguestostopbluffing.Asheputsit,Thereisalottobegainedbyrecognizingmorefullywhatourownresearch doesnotaddress,andbeingwillingtoadmitthatotherapproachesmaybeneeded.Or,toputitanotherway,tryingtorivalhumanintelligence,without thinkingaboutalltheintricaciesofthehumanmindatitsbest,islikeaskinganalligatortorunthehundredmetrehurdles. IllustrationbyArnoldRoth Keywords elements technology techpages

Newsletters

ThisWeek:LinkstoarticlesandWebonlyfeaturesinyourinboxeveryMonday. Cartoons:AweeklynotefromtheNewYorker'scartooneditor. Daily:What'snewtodayonnewyorker.com.

www.newyorker.com/online/blogs/elements/2013/08/why-cant-my-computer-understand-me.html?printable=true&currentPage=all 4/6

8/19/13

Artificial Intelligence and What Computers Still Don't Understand : The New Yorker

ReceiveallthelatestfakenewsfromTheBorowitzReport. IunderstandandagreethatregistrationonoruseofthissiteconstitutesagreementtoitsUserAgreement,andPrivacyPolicy. Print More

Share
Close Reddit LinkedIn Email

GaryMarcus
Bio Allposts RSS Jul23

MorrisHalle,theManWhoHiredNoamChomsky
Jun27

DARPA'sNewChallenge:BuildaDisasterZoneRobot
Jun19
www.newyorker.com/online/blogs/elements/2013/08/why-cant-my-computer-understand-me.html?printable=true&currentPage=all 5/6

8/19/13

Artificial Intelligence and What Computers Still Don't Understand : The New Yorker

TheProblemwiththeNeuroscienceBacklash
GaryMarcusisaprofessoratNewYorkUniversity. @GaryMarcus Follow OurNewsletters Togetmoreof TheNewYorker'ssignaturemixofpolitics,cultureandthearts: SubscribeNow

www.newyorker.com/online/blogs/elements/2013/08/why-cant-my-computer-understand-me.html?printable=true&currentPage=all

6/6