Sound Synthesis (Part 1) Graduate School of Culture Technology, - - PowerPoint PPT Presentation

2018 Fall CTP431: Music and Audio Computing

Sound Synthesis (Part 1)

Graduate School of Culture Technology, KAIST Juhan Nam

Outlines

• Signal model (analog / digital) – Part 1
• Additive Synthesis
• Subtractive Synthesis
• Modulation Synthesis
• Distortion Synthesis
• Sample model (digital) – Part 2
• Sampling Synthesis
• Granular Synthesis
• Concatenative Synthesis
• Physical model (digital) – Part 2
• Digital Waveguide Model

Signal Model

• Modeling the patterns of musical tones using elementary

waveforms

• Time domain: ADSR
• Frequency domain: spectrum
• Types of signal models
• Additive synthesis: a set of sine waveforms
• Subtractive synthesis: sawtooth, square waveforms + filters
• Frequency modulation synthesis: a pair of sine waveforms
• Distortion synthesis: sine waveforms + nonlinear units
• These techniques date back to the analog age

Additive Synthesis

• Synthesize sounds by adding multiple sine oscillators
• Also called Fourier synthesis

OSC OSC OSC . . . Amp (Env) Amp (Env) Amp (Env) . . .

+

Telharmonium

• Additive synthesizer using electro-magnetic “tone wheels”

(Cahill, 1897)

• Transmitted through telephone lines
• Subscription only but the the business failed

Tone wheel

Hammond Organ

• Drawbars
• Control the levels of individual tonewheels

Theremin

• A sinusoidal tone generator
• Two antennas are remotely controlled to adjust pitch and volume

Theremin ( by Léon Theremin, 1928)

Theremin (Clara Rockmore)

https://www.youtube.com/watch?v=pSzTPGlNa5U

Sound Examples

• Web Audio Demo
• http://femurdesign.com/theremin/
• http://www.venlabsla.com/x/additive/additive.html
• http://codepen.io/anon/pen/jPGJMK
• Examples (instruments)
• Kurzweil K150
• https://soundcloud.com/rosst/sets/kurzweil-k150-fs-additive
• Kawai K5, K5000

Subtractive Synthesis

• Synthesize sounds by filtering wide-band oscillators
• Source-Filter model

5 10 15 20 −60 −50 −40 −30 −20 −10 10 20 Frequency (kHz) Magnitude (dB) 5 10 15 20 −60 −50 −40 −30 −20 −10 10 20 Frequency (kHz) Magnitude (dB) 0.5 1 1.5 2 2.5 x 10

4

−60 −50 −40 −30 −20 −10 10 20 Frequency (kHz) Magnitude (dB)

Source Filter Filtered Source

Filter Oscillators Amp

50 52 54 56 58 60 −0.4 −0.2 0.2 0.4 time−milliseconds amplitude

Moog Synthesizers

MiniMoog (1970)

Moog Synthesizers

• Architecture

Envelope

Envelope LFO Wheels Slides Pedal Physical Control

Filter Oscillators Amp

Keyboard Audio Path Soft Control Parameter = offset + depth*control (e.g. filter cut-off frequency) (static value) (dynamic value) Parameter Parameter Parameter

“Switched-On-Bach” by Wendy Carlos (1968)

Oscillators

• Classic waveforms
• Modulation
• Pulse width modulation
• Hard-sync
• More rich harmonics

50 100 150 200 −2 2 5 10 15 20 −60 −40 −20 20 Frequency (kHz) Magnitude (dB)

−6dB/oct

50 100 150 200 −1 1 5 10 15 20 −60 −40 −20 20 Frequency (kHz) Magnitude (dB)

−6dB/oct

50 100 150 200 −2 2 5 10 15 20 −60 −40 −20 20 Frequency (kHz) Magnitude (dB)

−12dB/oct

Sawtooth Triangular Square

Amp Envelop Generator

• Amplitude envelope generation
• ADSR curve: attack, decay, sustain and release
• Each state has a pair of time and target level

Note On Note Off

Attack Decay Sustain Release

Amplitude (dB)

Examples

• Web Audio Demos
• http://www.google.com/doodles/robert-moogs-78th-birthday
• http://webaudiodemos.appspot.com/midi-synth/index.html
• http://aikelab.net/websynth/
• http://nicroto.github.io/viktor/
• Example Sounds
• SuperSaw
• Leads
• Pad
• MoogBass
• 8-Bit sounds: https://www.youtube.com/watch?v=tf0-Rrm9dI0
• TR-808: https://www.youtube.com/watch?v=YeZZk2czG1c

Modulation Synthesis

• Modulation is originally from communication theory
• Carrier: channel signal, e.g., radio or TV channel
• Modulator: information signal, e.g., voice, video
• Types of modulation synthesis
• Amplitude modulation (or ring modulation)
• Frequency modulation
• Decreasing the frequency of carrier to hearing range can be

used to synthesize sound

• Generate new sinusoidal components
• Modulation is non-linear processing

Ring Modulation / Amplitude Modulation

• Change the amplitude of one source with another source
• Slow change: tremolo
• Fast change: generate a new tone

OSC OSC

(1+ am(t))Ac cos(2π fct) am(t)Ac cos(2π fct)

Amplitude Modulation

x

Carrier Modulator

OSC OSC

x

Carrier Modulator

+ Ring Modulation

Ring Modulation / Amplitude Modulation

• Frequency domain
• Expressed in terms of its sideband frequencies
• The sum and difference of the two frequencies are obtained according to

trigonometric identity

• If the modulator is a non-sinusoidal tone, a mirrored-spectrum with regard

to the carrier frequency is obtained fc+fm fc fc-fm

am(t) = Am sin(2π fmt))

carrier sideband sideband

Examples

• Tone generation
• SawtoothOsc x SineOsc
• https://www.youtube.com/watch?v=yw7_WQmrzuk
• Ring modulation is often used as an audio effect
• http://webaudio.prototyping.bbc.co.uk/ring-modulator/

Frequency Modulation

• Change the frequency of one source with another source
• Slow change: vibrato
• Fast change: generate a new (and rich) tone
• Invented by John Chowning in 1973 à Yamaha DX7

Ac cos(2π fct + β sin(2π fmt)) β = Am fm

Index of modulation

OSC OSC

Carrier Modulator frequency

Frequency Modulation

• Frequency Domain
• Expressed in terms of its sideband frequencies
• Their amplitudes are determined by the Bessel function
• The sidebands below 0 Hz or above the Nyquist frequency are folded

y(t) = Ac Jk(

k=−∞ k=−∞

∑

β)cos(2π( fc + kfm)t)

fc+fm fc fc+2fm fc+3fm fc-fm fc-2fm fc-3fm

carrier sideband1 sideband1 sideband2 sideband2 sideband3 sideband3

Frequency Modulation

• Bessel Function

Jk(β) = (−1)n(β 2 )k+2n n!(n + k)! n=0

∞

∑

50 100 150 200 250 300 350 −0.5 0.5 1 beta J_(k) Carrier Sideband 1 Sideband 2 Sideband 3 Sideband 4

The Effect of Modulation Index

500 1000 1500 2000 −1 1 Time (Sample) Amplitude 0.2 0.4 0.6 0.8 1 −60 −40 −20 20 Frequency (kHz) Magnitude (dB) Beta = 0 500 1000 1500 2000 −1 1 Time (Sample) Amplitude 0.2 0.4 0.6 0.8 1 −60 −40 −20 20 Frequency (kHz) Magnitude (dB) Beta = 1 500 1000 1500 2000 −1 1 Time (Sample) Amplitude 0.2 0.4 0.6 0.8 1 −60 −40 −20 20 Frequency (kHz) Magnitude (dB) Beta = 10 500 1000 1500 2000 −1 1 Time (Sample) Amplitude 0.2 0.4 0.6 0.8 1 −60 −40 −20 20 Frequency (kHz) Magnitude (dB) Beta = 20

fc = 500, fm = 50

FM Synthesizer

Yamaha DX7 (1983)

Examples

• Web Audio Demo
• http://www.taktech.org/takm/WebFMSynth/
• Sound Examples
• Bell
• Wood
• Brass
• Electric Piano
• Vibraphone

Non-linear Synthesis (wave-shaping)

• Generate a rich sound spectrum by distorting sine waveforms

using non-linear transfer functions

• Also called “distortion synthesis”

50 100 150 200 −1 1 Time (Sample) Amplitude 5 10 15 20 −60 −40 −20 20 Frequency (kHz) Magnitude (dB)

50 100 150 200 −1 1 Time (Sample) Amplitude 5 10 15 20 −60 −40 −20 20 Frequency (kHz) Magnitude (dB)

−1 −0.5 0.5 1 −1 −0.5 0.5 1 Time (Sample) Amplitude

x’=gx: g correspond to the “gain” of the distortion

Distortion Transfer Function

• Examples of transfer function: y = f(x)
• y = 1.5x’ – 0.5x’3
• y = x’/(1+|x’|)
• y = sin(x’)
• Chebyshev polynomial: Tk+1(x) = 2xTk(x)-Tk-1(x)

T0(x)=1, T1(x)=x, T2(x)=2x2-1, T2(x)=4x3-3x