Sound Synthesis (Part 1) Graduate School of Culture Technology, - PowerPoint PPT Presentation

2018 Fall CTP431: Music and Audio Computing Sound Synthesis (Part 1) Graduate School of Culture Technology, KAIST Juhan Nam

Outlines • Signal model (analog / digital) – Part 1 - Additive Synthesis - Subtractive Synthesis - Modulation Synthesis - Distortion Synthesis • Sample model (digital) – Part 2 - Sampling Synthesis - Granular Synthesis - Concatenative Synthesis • Physical model (digital) – Part 2 - Digital Waveguide Model

Signal Model • Modeling the patterns of musical tones using elementary waveforms - Time domain: ADSR - Frequency domain: spectrum • Types of signal models - Additive synthesis: a set of sine waveforms - Subtractive synthesis: sawtooth, square waveforms + filters - Frequency modulation synthesis: a pair of sine waveforms - Distortion synthesis: sine waveforms + nonlinear units • These techniques date back to the analog age

Additive Synthesis • Synthesize sounds by adding multiple sine oscillators - Also called Fourier synthesis OSC Amp (Env) OSC Amp (Env) + . . . . . . OSC Amp (Env)

Telharmonium • Additive synthesizer using electro-magnetic “tone wheels” (Cahill, 1897) - Transmitted through telephone lines - Subscription only but the the business failed Tone wheel

Hammond Organ • Drawbars - Control the levels of individual tonewheels

Theremin • A sinusoidal tone generator - Two antennas are remotely controlled to adjust pitch and volume Theremin ( by Léon Theremin, 1928)

Theremin (Clara Rockmore) https://www.youtube.com/watch?v=pSzTPGlNa5U

Sound Examples • Web Audio Demo - http://femurdesign.com/theremin/ - http://www.venlabsla.com/x/additive/additive.html - http://codepen.io/anon/pen/jPGJMK • Examples (instruments) - Kurzweil K150 - https://soundcloud.com/rosst/sets/kurzweil-k150-fs-additive - Kawai K5, K5000

Subtractive Synthesis • Synthesize sounds by filtering wide-band oscillators - Source-Filter model 0.4 0.2 amplitude Oscillators Filter Amp 0 − 0.2 − 0.4 50 52 54 56 58 60 time − milliseconds 20 20 20 10 10 10 0 0 0 Magnitude (dB) Magnitude (dB) Magnitude (dB) − 10 − 10 − 10 − 20 − 20 − 20 − 30 − 30 − 30 − 40 − 40 − 40 − 50 − 50 − 50 − 60 − 60 5 10 15 20 − 60 0 0.5 1 1.5 2 2.5 Frequency (kHz) 5 10 15 20 Frequency (kHz) 4 x 10 Frequency (kHz) Filtered Source Source Filter

Moog Synthesizers MiniMoog (1970)

Moog Synthesizers • Architecture Soft Envelope LFO Control Keyboard Physical Envelope Control Wheels Slides Pedal Parameter Parameter Parameter Audio Path Amp Oscillators Filter (e.g. filter Parameter = offset + depth*control cut-off (dynamic value) (static value) frequency)

“Switched-On-Bach” by Wendy Carlos (1968)

Oscillators • Classic waveforms 2 2 1 0 0 0 − 2 − 1 − 2 0 50 100 150 200 0 50 100 150 200 0 50 100 150 200 20 20 20 Magnitude (dB) Magnitude (dB) Magnitude (dB) − 12dB/oct 0 − 6dB/oct − 6dB/oct 0 0 − 20 − 20 − 20 − 40 − 40 − 40 − 60 − 60 − 60 5 10 15 20 5 10 15 20 5 10 15 20 Frequency (kHz) Frequency (kHz) Frequency (kHz) Triangular Sawtooth Square • Modulation - Pulse width modulation - Hard-sync - More rich harmonics

Amp Envelop Generator • Amplitude envelope generation - ADSR curve: attack, decay, sustain and release - Each state has a pair of time and target level Amplitude Attack Decay Sustain (dB) Release Note On Note Off

Examples • Web Audio Demos - http://www.google.com/doodles/robert-moogs-78th-birthday - http://webaudiodemos.appspot.com/midi-synth/index.html - http://aikelab.net/websynth/ - http://nicroto.github.io/viktor/ • Example Sounds - SuperSaw - Leads - Pad - MoogBass - 8-Bit sounds: https://www.youtube.com/watch?v=tf0-Rrm9dI0 - TR-808: https://www.youtube.com/watch?v=YeZZk2czG1c

Modulation Synthesis • Modulation is originally from communication theory - Carrier: channel signal, e.g., radio or TV channel - Modulator: information signal, e.g., voice, video • Types of modulation synthesis - Amplitude modulation (or ring modulation) - Frequency modulation • Decreasing the frequency of carrier to hearing range can be used to synthesize sound - Generate new sinusoidal components - Modulation is non-linear processing

Ring Modulation / Amplitude Modulation • Change the amplitude of one source with another source - Slow change: tremolo - Fast change: generate a new tone OSC OSC Modulator Modulator OSC OSC x + x Carrier Carrier (1 + a m ( t )) A c cos(2 π f c t ) a m ( t ) A c cos(2 π f c t ) Ring Modulation Amplitude Modulation

Ring Modulation / Amplitude Modulation • Frequency domain - Expressed in terms of its sideband frequencies - The sum and difference of the two frequencies are obtained according to trigonometric identity - If the modulator is a non-sinusoidal tone, a mirrored-spectrum with regard to the carrier frequency is obtained carrier sideband sideband a m ( t ) = A m sin(2 π f m t )) f c -f m f c f c +f m

Examples • Tone generation - SawtoothOsc x SineOsc - https://www.youtube.com/watch?v=yw7_WQmrzuk • Ring modulation is often used as an audio effect - http://webaudio.prototyping.bbc.co.uk/ring-modulator/

Frequency Modulation • Change the frequency of one source with another source - Slow change: vibrato - Fast change: generate a new (and rich) tone - Invented by John Chowning in 1973 à Yamaha DX7 OSC Modulator A c cos(2 π f c t + β sin(2 π f m t )) frequency OSC β = A m Carrier Index of modulation f m

Frequency Modulation • Frequency Domain - Expressed in terms of its sideband frequencies - Their amplitudes are determined by the Bessel function - The sidebands below 0 Hz or above the Nyquist frequency are folded k = −∞ ∑ y ( t ) = A c J k ( β )cos(2 π ( f c + kf m ) t ) k = −∞ carrier sideband1 sideband1 sideband2 sideband2 sideband3 sideband3 f c -3f m f c +f m f c -f m f c f c +3f m f c -2f m f c +2f m

Frequency Modulation • Bessel Function ( − 1) n ( β 2 ) k + 2 n ∞ ∑ J k ( β ) = n !( n + k )! n = 0 1 Carrier Sideband 1 Sideband 2 Sideband 3 Sideband 4 0.5 J_(k) 0 − 0.5 0 50 100 150 200 250 300 350 beta

The Effect of Modulation Index 1 1 Amplitude Amplitude 0 0 − 1 − 1 0 500 1000 1500 2000 0 500 1000 1500 2000 Time (Sample) Time (Sample) 20 20 Magnitude (dB) Magnitude (dB) Beta = 0 Beta = 1 0 0 − 20 − 20 − 40 − 40 − 60 − 60 0.2 0.4 0.6 0.8 1 0.2 0.4 0.6 0.8 1 Frequency (kHz) Frequency (kHz) 1 1 Amplitude Amplitude 0 0 − 1 − 1 0 500 1000 1500 2000 0 500 1000 1500 2000 Time (Sample) Time (Sample) 20 Magnitude (dB) 20 Magnitude (dB) Beta = 10 Beta = 20 0 0 − 20 − 20 − 40 − 40 − 60 − 60 0.2 0.4 0.6 0.8 1 0.2 0.4 0.6 0.8 1 Frequency (kHz) Frequency (kHz) f c = 500, f m = 50

FM Synthesizer Yamaha DX7 (1983)

Examples • Web Audio Demo - http://www.taktech.org/takm/WebFMSynth/ • Sound Examples - Bell - Wood - Brass - Electric Piano - Vibraphone

Non-linear Synthesis (wave-shaping) • Generate a rich sound spectrum by distorting sine waveforms using non-linear transfer functions • Also called “distortion synthesis” 1 1 Amplitude Amplitude 1 0 0 0.5 − 1 − 1 0 50 100 150 200 Amplitude 0 50 100 150 200 Time (Sample) Time (Sample) 0 20 Magnitude (dB) 20 Magnitude (dB) 0 0 − 0.5 − 20 − 20 − 40 − 1 − 40 − 60 − 1 − 0.5 0 0.5 1 5 10 15 20 Time (Sample) − 60 Frequency (kHz) 5 10 15 20 Frequency (kHz) x’=gx: g correspond to the “gain” of the distortion

Distortion Transfer Function • Examples of transfer function: y = f(x) - y = 1.5x’ – 0.5x’ 3 - y = x’/(1+|x’|) - y = sin(x’) - Chebyshev polynomial: T k+1 (x) = 2xT k (x)-T k-1 (x) T 0 (x)=1, T 1 (x)=x, T 2 (x)=2x 2 -1, T 2 (x)=4x 3 -3x