CPU consumption for AM/FM audio effects Antonio Goulart Marcelo - - PowerPoint PPT Presentation

Introduction Concepts AM/FM DAFx CPU consumption Conclusion

CPU consumption for AM/FM audio effects

Antonio Goulart Marcelo Queiroz Victor Lazzarini Joseph Timoney

University of S˜ ao Paulo Maynooth University ag@ime.usp.br

2019/03/24

1 / 14

Introduction Concepts AM/FM DAFx CPU consumption Conclusion

Audio effects

“I feel that the processing is only there to emphasize things that are already there ... I’m never very interested in the kind of sound processing that leads me way too far away from the original sound, unless it really makes connections.” Hildegard Westerkamp

2 / 14

Introduction Concepts AM/FM DAFx CPU consumption Conclusion

AM/FM framework

y(t) = a(t) cos t f (τ)dτ

• a(t) envelope

AM (ring mod) synth f (t) instantaneous frequency (IF) FM/PM synth

3 / 14

Introduction Concepts AM/FM DAFx CPU consumption Conclusion

Analysis / Processing / Resynthesis

frameworks that offer an alternative domain (to think about the sound) Fourier analysis already familiar Frequency domain correlates well to musical aspects like bass, treble, etc Computed on chunks of the signal Envelope + instantaneous frequency Different perspective to think about effects Computed on a sample-by-sample basis (all-pass filters approach)

4 / 14

Introduction Concepts AM/FM DAFx CPU consumption Conclusion

Analytic signal

for x(t) ∈ R X(ω) = +∞

−∞

x(t)e−iωtdt z(t) = 1 2π +∞ X(ω)eiωtdω

5 / 14

Introduction Concepts AM/FM DAFx CPU consumption Conclusion

Hilbert Transform based decomposition

ˆ x(t) in quadrature with x(t) z(t) = x(t) + iˆ x(t) Envelope radius a(t) =

• x2(t) + ˆ

x2(t) = |z(t)| Phase φ(t) = arctan ˆ x(t) x(t)

• Instantaneous frequency

increment f (t) = ˙ φ(t) Time-varying phasor z(t) = a(t)eiφ(t)

6 / 14

Introduction Concepts AM/FM DAFx CPU consumption Conclusion

AM/FM DAFx

The effects are based on envelope / instantaneous frequency manipulations, followed by the resynthesis step A nice example: the octIFer (our AM/FM octaver) based on multiplying the IF samples by 0.5 in such a way the phasor increments are constrained, resulting in a lower register sound

7 / 14

Introduction Concepts AM/FM DAFx CPU consumption Conclusion

sqENVerb – reverb enhancer

based on extracting the square root of the envelope samples [0, 1] → [0, 1] √x x = 1 √x lower envelope values will result in a higher relative gain

8 / 14

Introduction Concepts AM/FM DAFx CPU consumption Conclusion

sqENVerb – reverb enhancer

9 / 14

Introduction Concepts AM/FM DAFx CPU consumption Conclusion

Motivation

artistic considerations (e.g. maximum tolerated latencies) technical considerations (e.g. hardware sizing) mobile/weak/old computers: RaspberryPi 2B / quad-core ARM Cortex-A7 @ 900 MHz 32 bits / 1 GB SD-RAM @ 400 MHz / Raspbian / Csound 6.08 strong/game computers: ThinkPad x220 / dual-core i5-2520M @ 2.5 GHz 64 bits / 8 GB RAM DDR3 @ 1333 MHz / Debian / Csound 6.09.1 44100 samples per second

10 / 14

Introduction Concepts AM/FM DAFx CPU consumption Conclusion

Methodology

time csound amfmdafx.csd real: total duration of the process under analysis; user: time taken to work directly on the process; sys: time taken to work on system tasks related to the process. CPU(%) = user + sys real

11 / 14

Introduction Concepts AM/FM DAFx CPU consumption Conclusion

CPU consumption for AM/FM DAFx

FFT pair = FFT followed by iFFT AM/FM framework = envelope and IF computation, followed by resynthesis CPU consumption (%) RaspberryPi 2B ThinkPad x220 looped audio 9.49 4.69 clip distortion 10.77 5.06 FFT pair (1024/512) 26.35 8.52 FFT pair (1024/256) 37.38 10.37 FFT pair (1024/128) 45.76 12.31 FFT pair (512/256) 26.05 8.68 FFT pair (512/128) 33.68 10.47 AM/FM framework 25.23 7.53

12 / 14

Introduction Concepts AM/FM DAFx CPU consumption Conclusion

CPU consumption for AM/FM DAFx

CPU consumption (%) RaspberryPi 2B ThinkPad x220 FFT octaver (1024/128) 48.00 12.41 convolution reverb 2500 88.98 14.73 convolution reverb 5000 ): 22.97 simulation reverb 26.13 7.67 AM/FM IF filtering 29.72 7.73 AM/FM IF compression 33.21 7.78 AM/FM IF modulation 29.23 7.92 AM/FM octIFer 29.87 7.81 AM/FM sqENVerb 28.51 7.77

13 / 14

Introduction Concepts AM/FM DAFx CPU consumption Conclusion

Conclusion

envelope / instantaneous frequency: a decomposition that

• ffers an alternative domain for DAFx

subtle modifications might result in interesting effects good reverb and octaver sounds, at a low computational cost decomposition in this study was mono-component; tracking fundamental frequency and adding more oscillators will raise computational cost Thanks a lot! Sorry for not being there ):

14 / 14