Eng.

100: Music Signal Processing DSP Lecture 9 Music synthesis

Announcements: Final Exam: Tue. Dec. 18, 4:00-6:00PM. No ofce hours Fri. Nov. 9 (will reschedule closer to nals)
1

Outline
Part 1: Part 2: Part 3: Part 4: M ATLAB loops Music synthesis via Fourier series Other music synthesis techniques Project 3 Q/A

Part 1: M ATLAB loops

M ATLAB anonymous functions

Tedious way to write a 4-note song in M ATLAB:
sound(0.9*cos(2*pi*440*[0:1/8192:0.5])) sound(0.9*cos(2*pi*660*[0:1/8192:0.5])) sound(0.9*cos(2*pi*880*[0:1/8192:0.5])) sound(0.9*cos(2*pi*660*[0:1/8192:1.0]))

Using an anonymous function can simplify and clarify:

S = 8192; playnote = @(f,d) sound(0.9*cos(2*pi*f*[0:1/S:d])); playnote(440, 0.5) playnote(660, 0.5) playnote(880, 0.5) playnote(660, 1.0)

Simpler and easier to see the key elements of the song (notes and duration). Easier to make global changes (such as amplitude 0.9). But still tedious if the song is longer than 4 notes...

M ATLAB loops
Using a for loop is the most concise and elegant:
S = 8192; playnote = @(f,d) sound(0.9*cos(2*pi*f*[0:1/S:d])); fs = [440 660 880 660]; ds = [0.5 0.5 0.5 1.0]; for index=1:numel(fs) playnote(fs(index), ds(index)) end

Here is another loop version that sounds better:

S = 8192; note = @(f,d) 0.9*cos(2*pi*f*[0:1/S:d]); fs = [440 660 880 660]; ds = [0.5 0.5 0.5 1.0]; x = []; for index=1:numel(fs) x = [x note(fs(index), ds(index))]; end sound(x, S)

play

Part 2: Music synthesis via Fourier series

Additive Synthesis: Mathematical formula

Simplied version of Fourier series for monophonic audio: ( ) K k x(t ) = ck cos 2 t T k=1 No DC term for audio: c0 = 0. Phase unimportant for monophonic audio, so k = 0. Example: x(t ) = 0.5 cos(2 400t ) +0.2 cos(2 800t ) +0.1 cos(2 2000t )

Example: Why we might want harmonics

1 x y 0.5 x(t), y(t)

0.5

play
0 0.001 0.002 0.003 0.004 0.005 t 0.006 0.007 0.008 0.009 0.01

play

y(t ) = 0.5 cos(2 400t ) x(t ) = 0.5 cos(2 400t ) +0.2 cos(2 800t ) +0.1 cos(2 2000t )
% fig_why1: example of additive synthesis S = 44100; N = 0.5 * S; % 0.5 sec t = [0:N-1]/S; % time samples: t = n/S y = 0.5 * cos(2*pi * 400 * t); x = y + 0.2 * cos(2*pi * 800 * t) + 0.1 * cos(2*pi * 2000 * t); plot(t, x, -, t, y, --), legend(x, y) xlabel t, ylabel x(t), y(t), axis([0 0.01 -1 1])

Same fundamental period, same pitch, different timbre.

8

M ATLAB implementation: Simple

Example: x(t ) = 0.5 cos(2 400t ) +0.2 cos(2 800t ) +0.1 cos(2 2000t )

A simple M ATLAB version looks a lot like the mathematical formula:

S N t x + + = 44100; = 0.5 * S; % 0.5 sec = [0:N-1]/S; % time samples: t = n/S = 0.5 * cos(2 * pi * 400 * t) ... 0.2 * cos(2 * pi * 800 * t) ... 0.1 * cos(2 * pi * 2000 * t);

There are many hidden for loops above. Where? ?? M ATLAB saves us from the tedium of writing out those loops,
thereby making the syntax look more like the math. In traditional programming languages like C, one would have to code all those loops. This simple implementation is still somewhat tedious, particularly for signals having more harmonics.
9

C99 implementation
#include <math.h> void my_signal(void) { float S = 44100; int N = 0.5 * S; // 0.5 sec float x[N]; // signal samples for (int n=0; n < N; ++n) { float t = n / S; x[n] = 0.5 * cos(2 * M_PI * 400 * t) + 0.2 * cos(2 * M_PI * 800 * t) + 0.1 * cos(2 * M_PI * 2000 * t); } }

10

Example revisited: Square wave

sin(t) 1 0 1 0 5 10 15 1 0 1 0 5 10 15 sin(t) + sin(3t)/3

sin(t) + sin(3t)/3 + sin(5*t)/5 1 0 1 0 5 10 15 1 0 1 0

sin(t) + ... + sin(15*t)/15

10

15

sin(t) + ... + sin(49*t)/49 1 0 1 0 5 10 15 x(t) 1 0 1 0

sin(t) + ... + sin(999*t)/999

5 t

10

15

Many dozens of harmonics to get a good square wave approximation.

11

M ATLAB implementation: Loop over harmonics

Example: x(t ) = 0.5 cos(2 400t ) +0.2 cos(2 800t ) +0.1 cos(2 2000t )
S = N = t = c = f = x = for end 44100; 0.5 * S; % 0.5 sec [0:N-1]/S; % time samples: t = n/S [0.5 0.2 0.1]; % amplitudes [1 2 5] * 400; % frequencies 0; k=1:length(c) x = x + c(k) * cos(2 * pi * f(k) * t);

I think this version is the easiest to read and debug. In fact it is a slight generalization.

) ( k It looks the most like the Fourier series formula: x(t ) = ck cos 2 t . T k=1
K

In Fourier series, the frequencies are multiples: k/T . In this code, the frequencies can be any values we put in the f array.
12

Example: Square wave via loop, with sin

S = N = t = c = f = x = for end
1

44100; 0.5 * S; % 0.5 sec [0:N-1]/S; % time samples: t = n/S 1 ./ [1:2:15]; % amplitudes [1:2:15] * 494; % frequencies 0; k=1:length(c) x = x + c(k) * sin(2 * pi * f(k) * t);

0.5 x(t)

0.5

play

0.001

0.002

0.003

0.004

0.005 t

0.006

0.007

0.008

0.009

0.01

Music example, circa 1978:

13

play

Example: Square wave via loop, with cos

S = N = t = c = f = x = for end
2 1 x(t) 0 1 2

44100; 0.5 * S; % 0.5 sec [0:N-1]/S; % time samples: t = n/S 1 ./ [1:2:15]; % amplitudes [1:2:15] * 494; % frequencies 0; k=1:length(c) x = x + c(k) * cos(2 * pi * f(k) * t);

play

0.001

0.002

0.003

0.004

0.005 t

0.006

0.007

0.008

0.009

0.01

14

M ATLAB implementation: Concise

We can avoid writing any for loops (and reduce typing) by using the following more concise (i.e., tricky) M ATLAB version:
S N t c f z x = = = = = = = 44100; 0.5 * S; % 0.5 sec [0:N-1]/S; % time samples: t = n/S [0.5 0.2 0.1]; % amplitudes [1 2 5] * 400; % frequencies cos(2 * pi * f * t); c * z;

c is 1 3 z is ?? x is ??

f is 3 1

t is 1 N

Where are the (hidden) loops in this version? ?? Use this approach or the previous slide in Project 3.
15

Part 3: Other music synthesis techniques

16

Frequency Modulation (FM) synthesis

In 1973, John Chowning of Stanford invented the use of frequency modulation (FM) as a technique for musical sound synthesis: x(t ) = A sin(2 f t + I sin(2 gt )), where I is the modulation index and f and g are both frequencies. (Yamaha licensed the patent for synthesizers and Stanford made out well.) This is a simple way to generate complicated periodic signals that are rich in harmonics. However, nding the value of I that gives a desired effect requires experimentation.

17

FM example 1: Traditional
S N t I x = = = = = 44100; 1.0 * S; [0:N-1]/S; % time samples: t = n/S 7; % adjustable 0.9 * sin(2*pi*400*t + I * sin(2*pi*400*t));

Very simple implementation, yet harmonically very rich spectrum:

0.7 0.6 0.5 spectrum 0.4 0.3 0.2 0.1 0 0 400 1200 2000 4000 frequency [Hz] 8000

play

18

FM example 2: Time-varying
) Time-varying modulation index: x(t ) = A sin 2 f t + I (t ) sin(2 gt ) . Simple formula / implementation can make remarkably intriguing sounds.
S N t I x = = = = = 44100; 1.0 * S; [0:N-1]/S; % time samples: t = n/S 0 + 9*t/max(t); % slowly increase modulation index 0.9 * sin(2 * pi * 400 * t + I .* sin(2*pi*400*t));

play What is the most informative graphical representation? ??

19

FM example 2: Time-varying
) Time-varying modulation index: x(t ) = A sin 2 f t + I (t ) sin(2 gt ) . Simple formula / implementation can make remarkably intriguing sounds.
S N t I x = = = = = 44100; 1.0 * S; [0:N-1]/S; % time samples: t = n/S 0 + 9*t/max(t); % slowly increase modulation index 0.9 * sin(2 * pi * 400 * t + I .* sin(2*pi*400*t));

play What is the most informative graphical representation? ??

Spectrogram of FM signal with timevarying modulation 4000

frequency [Hz]

2000 1200 400 0 0 0.1 0.2 0.3 0.4 0.5 t [sec] 0.6 0.7 0.8 0.9 1

20

Nonlinearities
Another way to make signals that are rich in harmonics is to use a nonlinear function such as y(t ) = x9(t ).
S N t b x y = = = = = = 44100; 1.0 * S; [0:N-1]/S; % time samples: t = n/S 0.2; % adjustable saturation point cos(2*pi*400*t); x.9;
1 0.5 x(t) 0 0.5 1 0 0.001 0.002 0.003 0.004 0.005 t [sec] 0.006 0.007 0.008 0.009 0.01

1 0.5 y(t) 0 0.5 1 0 0.001 0.002 0.003 0.004 0.005 t [sec] 0.006 0.007 0.008 0.009 0.01

spectrum of y

0.6 0.4 0.2 0 0 400 1200 2000 4000 frequency [Hz] 8000

play

21

Nonlinearities in ampliers
High quality audio ampliers are designed to be very close to linear because any nonlinearity will introduce undesired harmonics. Quality ampliers have a specied maximum total harmonic distortion (THD) that quanties the relative power in the output harmonics for a pure sinusoidal input. A formula for THD is: 2 2 c2 2 + c3 + c4 + THD = 100% 2 c1

What is the best possible value for THD? ?? On the other hand, electric guitarists often deliberately operate their ampliers nonlinearly to induce distortion, thereby introducing more harmonics than produced by a simple vibrating string. pure: play distorted: play

22

Envelope of a musical sound

23

Envelope example: Train whistle

train whistle signal 1

0.5 x(t)

0.5

0.2

0.4

0.6

0.8

1.2

1.4

1.6

train whistle envelope 1 0.8 envelope(t) 0.6 0.4 0.2 0

0.2

0.4

0.6

0.8 t [sec]
24

1.2

1.4

1.6

play

Envelope example: Plucked guitar

guitar signal 1

0.5 x(t)

0.5

0.5

1.5

2.5

3.5

4.5

plucked guitar envelope 1 0.8 envelope(t) 0.6 0.4 0.2 0

0.5

1.5

2 t [sec]

2.5

3.5

4.5

play

25

Envelope implementation
S = N = t = c = f = x = for 44100; 1 * S; [0:N-1]/S; 1 ./ [1:2:15]; % amplitudes [1:2:15] * 494; % frequencies 0; k=1:length(c) x = x + c(k) * sin(2 * pi * f(k) * t);

end env = (1 - exp(-80*t)) .* exp(-3*t); % fast attack; slow decay y = env .* x;

1 1

0.5 x(t) y(t) 0 0.2 0.4 t 0.6 0.8 1

0.5

0.5

0.5

play

0.2

0.4 t

0.6

0.8

play

26

Attack, Decay, Sustain, Release (ADSR)

These refer to the time durations of 4 components of the envelope as controlled in many synthesizers.

27

Summary
There are numerous methods for musical sound synthesis Additive synthesis provides complete control of spectrum Other synthesis methods provide rich spectra with simple operations (FM, nonlinearities) Time-varying spectra can be particularly intriguing Signal envelope (time varying amplitude) also affects sound characteristics Ample room for creativity and originality!

28

Part 4: Project 3 Q/A?

29