THESEMIOTICGESTURE

OleKhl
CentreofSemiotics,UniversityofAarhusRoyalAcademyofMusic,Aarhus.

ABSTRACT
Musicalmeaningisfluid.Thesamepieceofmusiccanmeandifferentthingstodifferentpeople,and thesamepersoncanexperience a piece ofmusicdifferentlyin different contexts. This does notmean, however, thatthe relationship between music asperceived structureandmusicasexperiencedcontentisabsolutelyarbitrary.Wecanshareamusicalexperience,wecanidentifywithspecific musicalstyles,and,mostimportantly,weseeminallculturesandatalltimestousemusicasanindispensablepartofourmost meaningful moments, as a device for sharing and bonding. So, although musical meaning cannot be pinpointed in any specified manner,likethemeaningoflanguage,thereisstillanamountofstablesubstanceinmusicalcommunication,whichcanbedefined. Themostimportant,stableelementinamusicalsemanticsistheprimarysignificationfrommusicalphrasetogestureandfrommu sicalgestureto emotionalcontentandsocialbelongingness.

Introduction
The musical gesture epitomizes human expressivity. It represents an implied level of communication, in which a musical phrase signifiesagesture.Inthisway,gesturebecomesthekeytotheunderstandingofmusicalmeaning.Toborrowanelegantformulation from Colwyn Trevarthen,music isaudible gesture(Trevarthen,2000: 172). Our perception seemsto extract certain shapes and patternsfromthesurfaceofthemusicalstream,whicharesubsequentlyrepresentedinthemindasinternalisedgesture.Thisisno surpriseforthoseofus,whohappentoexperiencemusicinthisway,andthereisfurtherconfirmationfromneurosciencethatsoma tosensorycentersinthebrainareactivewhenweareengagedinmusicalactivities.Itis,perhaps,alittlemoresurprisingthatoneof theprimaryvoicesagainsttheideaofextramusicalmeaningthatofEduardHanslickalsocouldbeinterpretedinsupportofthis idea.WhatelseshouldHanslickmeanbyhisfamousstatementthatmusicisTnendbewegteFormen,whichhasbeenvariously translatedas dynamicsoundpatterns bySusanneLanger(Langer,1942:225)astonallysoundingformbyRobertHatten(Hatten, 2004: 224)andas sonicallymovingformsbyThomasGrey(Grey,2006)? In what follows, I shall discussthepsychological and cognitive perspectives of thisstrange phenomenon of metaphoric mapping fromthesounddomaintothebodydomain.Ishallthenpleadforareorientationofmusicalsemiotics,whereIwillbearguingthat thegesturerepresentsadenotationallevelinmusic.Finally,Ishallproposethatthemusicalgesture,asasign,representsthelink betweenmusicassound,ontheonehand,andanintersubjectivelyfoundedsocialandemotionalcontentontheother.

GestureasExpressiveSharing
TheviewofmusicalgesturethatIamadvocatinghere,isinspiredbytheworkofdevelopmentalpsychologistslikeColwynTre varthenandDanielStern(Trevarthen2000Stern1998).Accordingto their accountofthedevelopmentofsocialand cognitiveskills, ouruniquehumanabilitytohavealanguageandaculturebeginswiththeprimordialexperienceoftheintersubjectivesharingof emotionandsensationbetweeninfantandcareperson.Itseemsthatourearliest,dyadiccommunicationcombinesgestureswithvo calisationsandtouch,thusunfoldinginseveralmodalitiessimultaneously:visually,somatically,andaurally. Thereisampleclinicalevidencethatperceptionattheearlieststagesofconsciousnessisnotmodalityspecific(Stern,1998:57ff. Trevarthen,1994Trevarthen,2000).Aperceivedgestaltinthemindofababyisthoughtnottobetiedtothevisual,theauditiveor thesensorymotoricmodality,butrathertoberepresentedamodally.Whenaninfantisengagedintheexchangeofgestures,vocali sationsandfacialexpressionswithacareperson,thereisnodistinctionfortheinfantbetweenthedifferentmodesofcommunication (somatic,auditive,andvisual).Theyare,accordingtothistheory,representedinthemindoftheinfantasaunified,amodalgestalt. Sterndescribesthepropertiesofsuchagestaltinthefollowingterms: Theexperimentsoncrossmodalcapacitiessuggestthatsome propertiesofpeopleandthings,suchasshape,intensitylevel,motion,number,andrhythm,areexperienceddirectlyasglobal,amo dalperceptualqualities. (Stern,1998:53) Theprimordialperceptioncanbecharacterizedas integrated,amodal,preverbal,andgeneric,andonlyatalaterpointdoesitbecome stratifiedintodifferentmodalities,infactmuchofthelearningprocessinkindergartensandpreschoolsisconcernedwiththeestab lishmentofamodalityspecificperception,inwhichthechildbecomesadeptindistinguishingthefunctioningoftheears,theeyes etc.Ata lateragemanyofuslosetheabilitytoconsciouslyaccessthelevelofamodalperception,butitisbelievedthatmuchartistic expressionoriginatesfromlayersofpreverbalconsciousness,thusbeingpartlydependentonfunctionsofcrossmodality. Consideringthedevelopmentofhumancognitioninthislight,wefindthatcognitivefunctionsareactiveatalllevelsofconscious ness.Atthedeeper,orearlier,levelstheyformthefoundationofamatureandsophisticatedcognition,emergingatalaterage.Itis importanttounderstandthattheearliestlevelsofwhatSterncallssensesofselfarenotsimplyreplaced,thewayasnakeshedshis

hide,butstayswithusasmoreabstractlevelsofthought.And,justlikethenextstageofconsciousnessdoesnotreplacetheearlier form,butdependsonitlikethelearningofarithmeticdependsontheabilitytocount,thedevelopmentofmoresophisticatedcogni tivefunctionsdoesnotmeanthatsimplerformsareexcluded.Inotherwords,eventhoughwemaynotbeawareofit,thisearlystate ofintersubjectivesharingofamodalgestaltsremainswithusasfundamentalforoursocialandcommunicativeskills.Suchisthe originofthemusicalgesture.

MusicalElementandCognitiveResponse
Buthowarewetounderstandthisphenomenonwecallamusicalgesture?Canmusicmakegestures?Orismusicmerely auditory cheesecake,somethingthatpleasesthesenseslikeastreamofhotwaterintheshower?Cognitivescienceteachesusthatneitheris true.Music is sound structuredin ahuman fashion. Thestructure makesit feasible for humans to respond in certain ways when perceivingthesound.Theseresponsesarebiologicallyconstrained,whilebeingsharedbymanypeopleinaculture,andthecreators ofthemusic musiciansandcomposersshapethemusictheyproducebasedontheirknowledgeofourcognitiveresponses. Themusicalgestureisacognitivephenomenon, emerging in the mind inresponsetomusicalpriming.Whenwelistentomusic,what weactuallyhearisanauditorystream,whichissubsequentlybeingprocessedbyauditoryperception.Inordertoeconomicallyand effectuallyprocessthesonicstreamofinformation,ourcognitiveapparatusstandsinneedoforganizinginputinchunksofacer tain size. These chunks are represented amodally in the mind as gestalts, and variously described as moving forms (Hanslick), vitalityaffects(Stern)andenergeticshaping(Hatten).Musicalgesturestemsfromthegenericlevelofperception,whereitistied togestaltperception,motormovementandmentalimagery.Gestures,accordingly,arerichgestalts,thatcombineauditoryinforma tion (hearingthe movement) withimplied visualinformation (imaginingthe movement), somatosensory information (feelingthe movement),andemotionalinformation(interpretingthemovement).Atahigherlevelofcognition,gesturesareorganizedingroups andsequences,leadingtomusicalformandnarrative,butthatwillnotbeconsideredhere.

TheMusicalSign
Thepairingofmusicalelementwithcognitiveresponse,themetaphoricmappingfrommusicdomaintomotordomain,issemioticby nature.Itpertainstothewayhumanbeingsmakesenseoftheworld.Inotherwords,Iamsuggestingthatthemusicalgestureasa cognitiveresponsetomusicalprimingisawayofmakingsenseofmusic,throughthetransformationoftheauditorystreamto interpretablechunks. Thegesturebecomesawayofunderstandingmusicasasemioticsystem, whichis comparabletoothersemiotic systems.Suchaviewcouldhelptobringmusicoutofitsisolatedstatusasahighlyspecialisedphenomenonandintothebroader socialandaestheticfieldofhumanactivity,expressionandcommunication. Inordertounderstandthis,wehavetoperformacarefulanalysisofthesignfunction.Interestingly,theanalysisofthesignasa cognitivefunctionbringsusrightbacktoSaussure.Inhisoriginaldefinitionofthesignfunction,Saussuredeclaresthatthetwo elementsinvolvedinthelinguistic signarebothpsychologicalandareconnectedinthebrainbyanassociativelink (Saussure,1995: 66).Inthesign,Saussureseesthelinkingofasoundpatternwithaconcept,wheretheformeristobeunderstoodasahearerspsy chologicalimpression ofasound (Saussure,1995:66)andnotasthephysicalsounditself, Figure 1: The Saussurean SignFunction

Concept Soundpattern Soundpattern

ForSaussure,thesignisamentalentity,andthesignfunctionispurelyphenomenological.Wecancontrastthiswith thesignaccord ingtoPeirce, withwhomasignstandsforsomethingtosomeone(Peirce, 1992).Hetriestocombinethebestofbothworlds,holding thesigntobesomethingthatlinksthephenophysical(mentalworld)withthegenophysical(realworld),thusleavingthestatusof thesign ontologically adrift. Whethersignsare physical,psychological, both, orneither,is never quite clear with Peirce,andhis developmentofsigntypologiesandlevelsofsignificationhasnotmadethiscrucialquestionanyclearer.Thisisthecruxofthecon troversybetweenAmericansemioticsandEuropeansemiology,withtheAmericanbranchleaningeverstrongertothephysicalside, andtheEuropeanbranchtothephenomenological. Itconcernsthepresentdiscussion,becausemusicalsemioticstodatemainlyhasbeenbasedonPeirceantheories.Thetwomostwell knownandinfluentialtheories,thoseofPhilipTaggandEeroTarasti(Tagg1992Tarasti1994),have,theirmanyqualitiesnotwith standing, developed intricate, musicspecific sign typologies based on Peirce. This means that they analyze musical phenomena throughacodingsystem,inwhichaspecificmusicalelementissaidtobelongtoaspecificclassofsigns.Inordertounderstandthis

code,youhavetounderstandthesignsystem,andthisunderstandingfailsatacertainpoint,becauseofaninsufficientanalysisofthe signfunction.Takingasystemthatisalreadyunclearattheoutset,andapplyingittosomethingasenigmaticasmusic,doesnot bringgreaterclaritytothefield.Atthesametime,suchaprocedureaddstothedivisionbetweenmusicandtherestofthehuman sphere, because it is based on the assumption that musical cognition is separate from general cognition. Nothing could be more wrong!

TheSignFunction
Allhumanactivitiescanbesubjecttospecialization,andhumanscandevelopgreatskillsinanyfieldtheysettheirmindto,includ ingmusic.But,attheoutset,anactivitylikemusicalexpressionandreceptionisubiquitouslyhuman,andthereforemustproceed from general human cognition first, before being specialized. Therefore, an understanding of highly developed musical cognition cannotbevalid,ifitdoesnotproceedfromanaccountofnatural,genericmusicalcognition.And,thisaccountmustbebasedona theoryofgeneralcognitionfirst,beforeamusicalspecializationoftheoriesisattempted.Atheoryofmusicalcognitioncanonlybe usefulandvalid formusicologyaswellasforgeneralcognitivesciencewhenthefollowingisclear:whatisthelimitationsofthe generaltheorythatforcesustolookforaspecializedtheory?andhowdoesthespecializedtheory relatetoandimprovethegeneral theory? Ananalysisofthesignfunctionmustbebasedoncognitivetheoryandneuroscientificdata tobeplausible.Usingthemusicalgesture asaparadigmexample,wehaveseenthattheauditorysystemextractspatterns,likemusicalphrases,fromtheauditorystream.Ina Saussureansignfunctionthesewillfunctionassignifiersorsignifiants.Thenacognitiveresponseisevokedintheformofamusical gesture,whichwillserveasthesignifiedorsignifi.Thesignisaunifiedwholewithanexpressionplane(signifier)andacontent plane(signified).Thismakesthemusicalphrasebeexperiencedsemioticallyasasoundpatternsignifyingagesture.Movementis embeddedinsound: Figure 2:TheMusicalSignFunction

Phrase

Sound pattern pattern

Musical gesture

Thisisthebasicmusicalsignfunction, andit represents thedenotational levelofmusicalsignification.Thecognitivefunction:musi calelementtoevokedresponseisbasicallyasignfunctionwithanexpressionlevelandacontentlevel,likeHjelmslevdemandsofa biplanarortruesemiotics(Hjelmslev,1961).Onecanfindmanyothermusicresponsepairslikethis,but thesoundtogesturesignis genericbecauseitinstantiatestheembodiedlevelofmusicalexperience.Justlikethemusicalgestureisthekeytomusicalmeaning, thephrasetogesturesigninvokesanew, semiotic approachto musical analysis.

TheMeaningofGesture
Whenwecompareasignlikethemusicalphrasewithasignlikeasentence,weseeofcoursethatthemusicalsignhasalowlevelof specification, whilethelinguistic sign has a high level of specification. The musical sign is more vague, more general, while the linguisticsignismorepreciselydefined.Thisdifferencedoesnotmakethemusicalphraseanylessasign.Instead,itshouldbeseen asaqualitativedistinctionbetweentwosemioticsystems,tellingussomethingaboutwhatitmeanstobehuman.Theapparentva guenessof themusicalsigndoesnotmakeitcompletelyempty,anonsignas Umberto Ecowouldhaveit (Eco, 1976). Thespecifica tionisatalowerlevel,indicatingageneraldirectionratherthanaspecificobject.Agesturecommunicates,infactitrepresentsa morebasicorgenericlevelofcommunicationthanwords.Itisanaturalformofcommunication,thefirstonelearned,andthelast resortwhenlanguagefails. Ithasbecomeinternalised throughasocalled Vygotskyanturn, andnowit canbeactivatedbymusicasa mentallyrepresentedgesture,asindicatedbyactivityintheSupplementaryMotorArea(SMA). Thegestureisaphysicalexpressionoffeelingandsensation.Theintersubjectivesharingoffacialexpressionsandcommunicative gesturesmeansthesharingofinnerstatesofbeing.Thesekindsofsomaticsignsaredirectlyconveyinginformationofasubjects emotionalstateandintention.Suchinformationisembeddedinthegestureassign,andevokedbytheimpliedgestureofamusical phrase.Thesignifiedbecomesanewsignwithasignifier(movement)andasignified(emotionalstate),yieldingasocalledsign cascade:

Figure 3:TheGestureasSign
Phrase

Sound pattern

Musical gesture

Implied movement movement

Emotional state

Aspartofourculturalprogramming,thereperformanceofspecificmannersofgesticulationandsignificationasinclassicalcon certs andothermusicalrites representaleveloflearningand reinforcementofculturalvaluesandofstylizedlevelsofcommunica tion.Musiccanbeseenasawholesemiosphereofsignsandsignfunctions,ofsignsyieldingsigns,oflearnedevokedresponsesto certaintypesofinput.Gesturestransmitmeaning,notonlyinthepresent,butthroughhistoryaswell.Considerthegallantstyleof th Mozart,whichtransmitsgesturesfromtheroyalcourtsof18 century centralEuropetothepresentorthevocalarticulationsofjoy th ingospelmusic,thathassaturatedsomuchof20 centurypopularmusic.Asembodiedmeaning,gesturesareanimportantpartof ournationalandculturalidentities. And inmusic,thesharingof gestures isfurtherintensified throughthepulse, whichbringsabouta synchronisationof implied movements. Thisisthepowerofmusic. Wecanshareamusicalexperience,wecanidentifywithspecificmusicalstyles,and,mostimportantly,weseem inallcultures and atalltimestousemusicasanindispensablepartofourmostmeaningfulmoments,asadeviceforsharingandbonding.So,al though musical meaning cannot be pinpointed in any specified manner,likethe meaning of language, there is stillan amount of stablesubstanceinmusicalcommunication,whichcanbedefined.Themostimportant,stableelementinamusicalsemanticsisthe primarysignificationfrommusicalphrasetogestureandfrommusicalgestureto emotionalcontentandsocialbelongingness.

REFERENCES
Eco,Umberto (1976) ATheoryofSemiotics.Chicago: IndianaUniversityPress. Grey,Thomas (2006) 'EduardHanslick',GroveMusicOnline ed. L.Macy(Accessed 30th May 2006), http://www.grovemusic.com Hatten,RobertS. (2004) InterpretingMusicalGestures,TopicsandTropes. Bloomington:IndianaUniversityPress. Hjelmslev,Louis (1961) ProlegomenatoaTheoryofLanguage. Madison:TheUniversityofWisconsinPress. Langer,SusanneK. (1942) PhilosophyinaNewKey.Cambridge,Mass:HarvardUniversityPress. Peirce,CharlesS. (1992) TheEssentialPeirce:SelectedPhilosophical Writings:186793v. 1. Chicago:IndianaUniversityPress. Saussure,Ferdinandde. (1983) CourseinGeneralLinguistics.London:Duckworth. Stern,Daniel. (1998) TheInterpersonalWorldoftheInfant.London:Karnac. Tagg,Philip.(1992) Towardsasigntypologyofmusic, inR.Dalmonteetal. (eds.)Secondoconvegnoeuropeodianalisimusicale. Trento:Universit deglistudidiTrento. Tarasti,Eero(1994) ATheoryofMusicalSemiotics. Bloomington:IndianaUniversityPress. Trevarthen,Colwyn.(1994) InfantSemiosis, inWinfriedNth(ed.): OriginsofSemiosis.Berlin:MoutondeGruyter. Trevarthen,Colwyn(2000) MusicalityandtheIntrinsicMotorPulse. MusicaeScientiae Special Issue: 155 211.