Proceedings of the International Computer Music Conference 2011, University of Huddersfield, UK, 31 July - 5 August 2011

INNOVATION, INTERACTION, EXPERIENCE AND IMAGINATION IN COMPUTER MUSIC EDUCATION

Alessandro Cipriani School of Electronic Music Conservatorio di Musica di Frosinone Edison Studio Rome Via Voghera, 7 00182 Roma a.cipriani@edisonstudio.it www.virtual-sound.com
ABSTRACT This paper presents the Electronic Music and Sound Design project, which explores innovative solutions for teaching computer music, especially as regards the theory and practice of sound synthesis, signal processing and sound design. The project uses integrated and interactive self-learning environments, leading the user on a journey from novice to expert thanks to the interaction between theory and practice, experience and imagination. A number of activities (replacing parts of algorithms, completing unfinished algorithms, correcting algorithms with bugs, interactive sound-building exercises and reverse engineering) help lead the user to what Joel Chadabe terms 'predictive knowledge' the ability to intuit what will happen to a sound before you take a specific action to modify it. The authors approach involves interactions between the perception of sounds and the knowledge and skills deriving from studying the theory and performing the practical activities, as well as fostering the users own individual creativity. 1. INTRODUCTION

Maurizio Giri School of Composition and Electronic Music Conservatorio di Musica di Campobasso - Italy maurizio@giri.it www.virtual-sound.com
ge learners to move away from reliance on theoretical considerations and abstract or rigid rules, while trying to develop their intuition and interaction with their own perceptions. In electronic music learning the learners are often too passive: they maybe understand all the patches and the theory, but they are still unable to correct their errors, adapt, invent, and use the knowledge and skills they have acquired in a personal, creative and active way. At first novices often need to be given context-free rules that they can adhere to, in order to accomplish immediate goals and reinforce their confidence through following fixed procedures. On the other hand we believe it is particularly important to promote context-based experience, develop critical thinking and encourage the use of individual perception and creativity while acquiring new knowledge and skills. There should not be an excessive reliance on technology since software is always subject to development and replacement. As one commentator points out:
Theres always going to be a new technology or a new version of an existing technology to be learned. The technology itself isnt so important; its the constant learning that counts. The body of knowledge is demonstrably not the important part. The model you build in your mind, the questions you ask to build that model, and your experiences and practices built up along the way and that you use daily are far more relevant to your performance. Theyre the things that develop competence and expertise. Mastery of the knowledge alone isnt sufficient.[10].

The Electronic Music and Sound Design project presents a series of innovative solutions for computer music education and training in both the theory and the practice of sound synthesis, signal processing and sound design. This project, based on the Max/MSP software, consists of three books and interactive multimedia environments, which the authors have developed over the course of the last 5 years. The project combines pedagogical contextbased paradigms with integrated and interactive selflearning environments, taking the user on a road of exploration based on the interaction between theory and practice, experience and imagination. 2. FROM CONTEXT-FREE RULES TO INTUITION AND IMAGINATION

The bibliography of electronic music education is very limited and a universal consensus regarding an appropriate teaching method has not been developed. We adopted various methodological criteria, sometimes taking ideas from other disciplines (such as foreign language education), in order to encoura383

Our integrated system gets the novice to imitate the suggested practice while learning the theory, and specific practical exercises of modification, correction and expansion are immediately proposed, according to a problem solving approach. Thus the model of an imitative and passive process is immediately challenged. Instead the activities and tasks are intended to set in motion the knowledge and practical skills of the user in an active, practical and creative way. When learning a foreign language, there is a gap between what one knows and what one is able to use in practice. It is common for a students passive vocabulary (the total number of terms that the student can recognize) to be much larger than the active vocabulary that s/he can actually use while speaking or writing. The same is true when

Proceedings of the International Computer Music Conference 2011, University of Huddersfield, UK, 31 July - 5 August 2011

learning a computer programming language: a student can understand how algorithms work without being able to build them from scratch. We encourage the learner to find different ways to replace parts of algorithms, complete unfinished algorithms, analyze and correct algorithms with bugs, and practice reverse engineering (in which the reader listens to a sound and then tries to invent an algorithm to create a similar sound). Here is an example of activity:

practitioner innovates and offer his/her own original interpretations and thoughts. We believe it is much more effective to combine all these three stages from the very beginning. We intend for our course in electronic music and sound design to lead towards a creative use of technology, as much as possible in conformity with the original, etymological meaning of the word education. In fact it comes from the Latin educare which means to lead out; in the maieutic sense of drawing something forth from learners and giving them the means to find their own voice. Emmersons idea about composition and pedagogy is very similar:
It is my contention that composition has moved to too great a degree towards 'objective' or 'knowledge-based' criteria and has forgotten the role of shared 'subjective' experience and exploration.[9]

3. INNOVATION: COMBINING THEORETICAL KNOWLEDGE AND EXPERIENTIAL LEARNING IN THE SAME SYSTEM
Figure 1. An incomplete patch.
This patch is incomplete. There are four objects that can be found on the right that have no connections. Move these into functional positions based on the following hint: the output of the patch random-minmax is a message, not a signal, while groove~ accepts only signals, used as a stream of multiplication factors, on its inlet. How can we resolve this problem? For what purpose can the *~ object be used? What signal should the number~ object monitor? What does sig~ do? Complete the patch and experiment with various changes to minimum and maximum values, using both positive and negative numbers. (Of course, the minimum value should always be less than the maximum value!) [6]

These activities (featured in the books and in the online patches and interactive material) pose problems to which users are encouraged to find their own solutions. When learning a foreign language, students are given replacement exercises (e.g. replace the underlined verb in the following phrase: I wish I could go), correction exercises (e.g. correct the following phrase: I want to went home), and sentences to be completed (e.g. Id like to ... home). The student must intensively practice similar activities, as well as listening and conversation exercises, in order to avoid an excessively passive and bookish approach to learning. Our approach, likewise, involves interactions between the perception of sounds and the knowledge and skills deriving from reading the book and doing the practical activities, but it also integrates and complements all these factors with the users own creativity. The cycle of training in the martial arts known as Shu Ha Ri dictates that at first the student must copy the techniques exactly as s/he is taught. In the next stage the student assimilates and deepens his/her theoretical and experiential knowledge. In the final stage the

The theory in our books and interactive material is conceived and presented in such a way that the user alternates its study with practical exercises as s/he progresses. The planning of the course takes account of the learning process and the experience that the student progressively acquires in all areas (theoretical, analytical, practical, etc.). The student therefore always learns within a creative and motivating context, always in contact with the production and perception of sound and without ever having to tackle lengthy theoretical discourses or explanations of the mechanisms of the software. The interaction between perception and knowledge, and the interchange between deductive, inductive and creative processes is continuous. There are some similarities between this kind of learning approach and learning to play a musical instrument, especially regarding the sonic feedback the musicians continuously receive in the search for the right sound guided by their technique and intuition, which leads them to gradually improve their performance using perceptual data. Learning to play an instrument and learning sound design are very far from a linear pedagogical process focused on merely providing a reader with objective facts. Our method is intended as a holistic and integrated process of research and discovery. That is one of the reasons why we started to look at foreign language teaching as a source of information about context-based, interactive learning. We found that many activities and ideas in that specific field can be usefully translated into methods for learning a programming language involving sound production and manipulation.
In my view, the manipulation aspect of learning about sound is critically important. It helps lead you to what Joel Chadabe terms predictive knowledge the ability to intuit what will

384

Proceedings of the International Computer Music Conference 2011, University of Huddersfield, UK, 31 July - 5 August 2011

happen to a sound before you take an action to change it. We all have some level of predictive knowledge. For example, most of us know that by turning a volume knob clockwise, the sound coming from our amplifier will get louder. Once we enter the realm of digital sound synthesis, things quickly get more complicated than a volume knob, and we need the first-hand experience of manipulation and perception in order to deepen our predictive knowledge. However, to educate ourselves fully about digitally produced sound, we need more than predictive knowledge. We need to know why our manipulations make the perceptual changes we experience. This theoretical knowledge reinforces our intuitive experiential knowledge, and at the same time, our experience gives perceptual meaning to theoretical explanations. [13]

whose frequencies are close to being integral multiples of some audible fundamental. The partials 110, 220.009, 329.999, 439.991, and 550.007 hertz, for example can be heard unequivocally as an A, even though their period doesnt correspond to the perceived fundamental, but rather to the greatest common divisor of the components, which is 0.001 hertz. To be precise about this, the period of this waveform is 1000 seconds, and a sound with a period as long as this is psycho-acoustically equivalent to a non-periodic sound. [6]

4.

INTERACTION BEFORE PRACTICE

Clarke points out that:

Simply reading a book or attending a lecture can lead to study that is remote from the sound that is the key element in the discipline. Lecturers may play musical examples and written texts may direct students to scores or CDs, but for many students this is not as stimulating as experiencing the music for themselves, especially engaging with it interactively.[7]

But what kind of experience and what type of theory are involved in our method? As regards the difference between teaching to scientific researchers or to musicians/sound designers Giuseppe Di Giugno once pointed out that for the former group the real thing is the formula, or the algorithm and making sounds by the variation of parameters is just an exercise. For musicians and sound designers making sounds is the real thing while learning the algorithm is a means to reach their goals. As most of us are aware this boundary between the two approaches is becoming less and less clear. Our system is aimed primarily at musicians and sound designers and because of that we have tried to create examples which sound good and are stimulating for the practice of composing, so our patches are not just intended as examples to illustrate the theory. Practical experience combined with imagination and perception often challenges the theory and expands its boundaries. In some cases the standard theory is unable to explain some of the reasons why the tips and tricks that are commonly used in sound design work so well. Thus we sometimes had to add some explanations to the theoretical sections, as in the case of multiple harmonics, filters, and the grey areas between the harmonicity and non-harmonicity of sounds. In this latter issue of harmonic/nonharmonic sounds and their connection to the concept of periodicity/aperiodicity we tried to extend the theoretical concepts and combine them with perceptual events.
If the lowest component in a harmonic sound is missing, but the immediately succeeding components are present, the sound will be heard as having a pitch equal to the missing fundamental. If, in addition to the fundamental, we start to remove other lower harmonics, we hear the gradual loss of its harmonicity, because at some point, our brain is no longer able to reconstruct the fundamental. A sound thus obtained is non-harmonic, but at the same time periodic, because its period is still the inverse of the frequency of its virtual fundamental. A non-harmonic sound composed of partials at 100, 205, 290, 425, and 460 hertz, for example, has a fundamental of 5 Hz, and is therefore a periodic sound that repeats 5 times per second (although it is not possible to hear this fundamental). Weve seen that you can have a periodic sound that has no definite pitch. On the other hand, is it possible to have a non-periodic sound that does have a definite pitch? Certainly. It is enough to have components

The interactive examples we provide in our project constitute a bridge between the study of the theory and the interaction with perception. The path laid out in the theoretical sections of our books is accompanied by numerous interactive examples, which are available on the website at www.virtualsound.com/emasd. Thanks to these examples, the user can immediately hear the sounds being discussed, as well as understand their design, without necessarily having to spend time in programming them. In this way, the study of the theory is immediately connected to the concrete experience of sounds. Our objective is to integrate the students understanding and experience of sound design and electronic music. This principle is the basis for all three volumes, as well as for future online materials that will be used to update, broaden, and clarify the existing text. The Interactive Examples Software Application contains a set of examples for each theory chapter. A typical example consists of a custom interface allowing one to choose between a certain number of presets (pre-selected configurations of parameters) illustrating a particular technique of synthesis or sound processing. A spectroscope, a sonogram, and an oscilloscope appear at the bottom of the window, and can be used to analyze the spectral content and the waveform of the sounds produced by the example. Generally the first presets listed for each technique are frozen, i.e. the user cannot manipulate the parameters, but can only hear and analyze the sound with the help of the sonogram, spectroscope and oscilloscope. The last presets for each set of examples are free configurations in which the user can vary and manipulate the parameters and experiment with a particular technique.

385

Proceedings of the International Computer Music Conference 2011, University of Huddersfield, UK, 31 July - 5 August 2011

6.

REFERENCES

[1] Arveiller, J. Comments on University Instruction in Computer Music Composition, in Computer Music Journal, Vol. 6, No. 2 (Summer, 1982), pp. 72-78, 1982. [2] Badii, A and Mothersole, P. Mediating Representations: Domain Knowledge to Pedagogical Content Knowledge in 2nd International Conference on Automated Production of Cross Media Content for Multi-channel Distribution Proceedings, University of Leeds, UK, 2006. [3] Bianchini, R. and Cipriani, A. Three levels of education in Electroacoustic Music: The Virtual Sound Project in ICMC Proceedings 1998, Ann Arbor, MI: Scholarly Publishing Office, University of Michigan Library, 1998. [4] Bianchini, R. and Cipriani, A. Virtual Sound On Line - Computer Music Courses on the Internet in ICMC Proceedings 1999, Ann Arbor, MI: Scholarly Publishing Office, University of Michigan Library, 1999. [5] Bianchini, R. and Cipriani, A. Virtual Sound Contemponet, Roma, Italy, 2003 [6] Cipriani, A. and Giri, M., Electronic Music and Sound Design, Contemponet, Roma, 2010. [7] Clarke, M., Watkins, A., Adkins, M., and Mark Bokowiec Sybil: Synthesis by Interactive Learningin ICMC Proceedings 2004, Ann Arbor, MI: Scholarly Publishing Office, University of Michigan Library, 2004. [8] Clarke, M. From SYnthia to Calma to Sybil: Developing Strategies for Interactive Learning in Music in Technology Supported Learning and Teaching: A Staff Perspective ed. J. O'Donoghue, Information Science Publishing, 2006. [9] Emmerson, S. Composing strategies and pedagogy in Contemporary Music Review, 1989 vol.3, Harwood Academic Publishers, 1989 [10] Hunt, A. Pragmatic Thinking and Learning. The Pragmatic Bookshelf, Raleigh, NC, 2008. [11] Ng, K., Weyde, T., Nesi, P., IMAESTRO: Technology-enhanced Learning for Music in ICMC Proceedings 2008, Ann Arbor, MI: Scholarly Publishing Office, University of Michigan Library, 2008. [12] Vishnick, Martin. Electronic and Electroacoustic Music Composition in Contemporary Education, in Journal of Electroacoustic Music. Vol. 14. London: Sonic Arts Network: 28-35, 2002. [13] Zicarelli, D. Foreword in Cipriani, A., Giri, M. Electronic Music and Sound Design, Contemponet, Roma, 2010.

Figure 2. Interactive Examples, Vector Synthesis on a segment

The picture shows the graphic interface of an interactive example of a Vector Synthesis. Four waveforms can be freely configured by manipulating their spectral components with the mouse. The four waveforms are blended in different proportions by moving a round cursor in the central window. As soon as the cursor is moved, the resulting interpolated spectrum appears, and the relative sound is produced in real-time, thereby providing the user with immediate feedback. In each of the three books, after each theory chapter and the corresponding interactive examples, there is a practice chapter, in which the synthesis and processing techniques are implemented in Max/MSP. In the practice chapters the user finds step-by-step instructions for realizing the previously discussed algorithms, together with several ready-to-use patches, activities of analysis, correction and completion, and a library of Max/MSP abstractions (Virtual Sound Macros library) created for this project. 5. CONCLUSIONS

It is perhaps not so important whether book and project such as ours is physical or virtual. It is the thought that has gone into it that really counts, as well as the effective integration with the software materials (in fact we have supplemented the text with hundreds of patches, interactive examples and useful activities.) The goal of our teaching system is to create a fully integrated pathway of learning. It is not just a set of information on specific kinds of sound synthesis, an informative Max/MSP manual or a collection of heterogeneous materials; its real value is the complete and holistic nature of the whole project. Thus the users learn the theory while also learning to program with Max/MSP and apply their knowledge and skills towards creating the sounds they want. The users of our system can start from scratch and end up dealing with physical modeling, but in the process they will have acquired a method of critical thinking and an experience that will also be useful for approaching and employing different computer music languages (including those of the future) with an open and flexible mind.

386