Hot Topics in Musical Acoustics Applied to Real-Time Sound Synthesis - - PowerPoint PPT Presentation

▶
hot topics in musical acoustics applied to real time
SMART_READER_LITE
LIVE PREVIEW

Hot Topics in Musical Acoustics Applied to Real-Time Sound Synthesis - - PowerPoint PPT Presentation

Apr 18, 2023 616 likes •829 views

Hot Topics in Musical Acoustics Applied to Real-Time Sound Synthesis Julius Smith CCRMA, Stanford University Acoustical Society of America 166th Meeting Hot Topics Session December 4, 2013 Julius Smith ASA-2013 Hot Topics 3pID3 (MU) 1 /

slide-1
SLIDE 1

Julius Smith ASA-2013 Hot Topics 3pID3 (MU) – 1 / 18

Hot Topics in Musical Acoustics Applied to Real-Time Sound Synthesis

Julius Smith CCRMA, Stanford University Acoustical Society of America 166th Meeting Hot Topics Session

December 4, 2013

slide-2
SLIDE 2

Outline

Enabling Technologies moForte Guitar Faust Intro Research

Julius Smith ASA-2013 Hot Topics 3pID3 (MU) – 2 / 18

slide-3
SLIDE 3

New Enabling Technologies

Enabling Technologies

  • Enabling Technology

moForte Guitar Faust Intro Research

Julius Smith ASA-2013 Hot Topics 3pID3 (MU) – 3 / 18

  • 1. Smart-Phones and Tablets
  • High-quality audio in (mono) and out (stereo)
  • Fast multicore processors (exponentially growing speed)
  • Multitouch display screens (5 for iPhone, 11 for iPad)
  • 2. Domain-Specific Languages
  • Functional AUdio STream (FAUST) — High-Level

Signal-Processing Language (Yann Orlarey, GRAME)

  • Synth-A-Modeler — Physical Model “Diagram” to FAUST

(Edgar Berdahl)

  • 3. New Convex Optimization Techniques and Formulations

(Esteban Maestre)

slide-4
SLIDE 4

Smart-Phone/Tablet Example: moForte Guitar

Enabling Technologies moForte Guitar Faust Intro Research

Julius Smith ASA-2013 Hot Topics 3pID3 (MU) – 4 / 18

slide-5
SLIDE 5

moForte Guitar

Enabling Technologies moForte Guitar

  • moForte Guitar
  • CPU Performance
  • Sound Examples

Faust Intro Research

Julius Smith ASA-2013 Hot Topics 3pID3 (MU) – 5 / 18

Real time on the iPhone 4S (all written in FAUST):

  • Full electric-guitar + effects:

Six vibrating strings — general excitations Distortion Feedback Compression Wah pedal or Autowah Phaser Flanger Five-band parametric equalizer Reverb

  • Responds to

accelerometer, gyros, touches (plucks), swipes (strumming), . . .

  • It is challenging to fully utilize five points of multitouch on the

iPhone and eleven on the iPad!

slide-6
SLIDE 6

moForte Guitar

Enabling Technologies moForte Guitar

  • moForte Guitar
  • CPU Performance
  • Sound Examples

Faust Intro Research

Julius Smith ASA-2013 Hot Topics 3pID3 (MU) – 5 / 18

Real time on the iPhone 4S (all written in FAUST):

  • Full electric-guitar + effects:

Six vibrating strings — general excitations Distortion Feedback Compression Wah pedal or Autowah Phaser Flanger Five-band parametric equalizer Reverb

  • Responds to

accelerometer, gyros, touches (plucks), swipes (strumming), . . .

  • It is challenging to fully utilize five points of multitouch on the

iPhone and eleven on the iPad!

slide-7
SLIDE 7

moForte Guitar

Enabling Technologies moForte Guitar

  • moForte Guitar
  • CPU Performance
  • Sound Examples

Faust Intro Research

Julius Smith ASA-2013 Hot Topics 3pID3 (MU) – 5 / 18

Real time on the iPhone 4S (all written in FAUST):

  • Full electric-guitar + effects:

Six vibrating strings — general excitations Distortion Feedback Compression Wah pedal or Autowah Phaser Flanger Five-band parametric equalizer Reverb

  • Responds to

accelerometer, gyros, touches (plucks), swipes (strumming), . . .

  • It is challenging to fully utilize five points of multitouch on the

iPhone and eleven on the iPad!

slide-8
SLIDE 8

Effects Running in Real Time on the iPhone 4S with All Six Strings Playing

Julius Smith ASA-2013 Hot Topics 3pID3 (MU) – 6 / 18

slide-9
SLIDE 9

iPhone CPU Performance

Enabling Technologies moForte Guitar

  • moForte Guitar
  • CPU Performance
  • Sound Examples

Faust Intro Research

Julius Smith ASA-2013 Hot Topics 3pID3 (MU) – 7 / 18

slide-10
SLIDE 10

Distortion Guitar Sound Examples

Enabling Technologies moForte Guitar

  • moForte Guitar
  • CPU Performance
  • Sound Examples

Faust Intro Research

Julius Smith ASA-2013 Hot Topics 3pID3 (MU) – 8 / 18

  • moForte Guitar Demo Video
  • Distortion Guitar: (WAV) (MP3)
  • Amplifier Feedback 1: (WAV) (MP3)
  • Amplifier Feedback 2: (WAV) (MP3)
slide-11
SLIDE 11

FAUST Introduction

Enabling Technologies moForte Guitar Faust Intro Research

Julius Smith ASA-2013 Hot Topics 3pID3 (MU) – 9 / 18

slide-12
SLIDE 12

FAUST Language

Julius Smith ASA-2013 Hot Topics 3pID3 (MU) – 10 / 18

Short FAUST Program Examples:

process = _ ; process = + ; process = _,_ : + : _; process = pole(0.9) with { pole(p) = + ~ *(p); };

Partial C++ Output for Last Example Above:

virtual void compute (int count, FAUSTFLOAT** input, FAUSTFLOAT** output) { FAUSTFLOAT* input0 = input[0]; FAUSTFLOAT* output0 = output[0]; for (int i=0; i<count; i++) { fRec0[0] = ((float)input0[i] + (0.9f * fRec0[1]));

  • utput0[i] = (FAUSTFLOAT)fRec0[0];

// post processing fRec0[1] = fRec0[0]; } }

slide-13
SLIDE 13

FAUST Example: zita rev1.dsp

Julius Smith ASA-2013 Hot Topics 3pID3 (MU) – 11 / 18

Source Code (FAUSTeffect.lib):

zita_rev_fdn(f1,f2,t60dc,t60m,fsmax) = ((bus(2*N) :> allpass_combs(N) : feedbackmatrix(N)) ~ (delayfilters(N,freqs,durs) : fbdelaylines(N))) ...

Block Diagram (drawn by the FAUST compiler):

in_delay distrib2(8) allpass_combs(8) feedbackmatrix(8) delayfilters(...1, 8, 0.1)))) fbdelaylines(8)

  • utp

zita_rev1_eng...1, 8, 0.1))))(48000)

JACK-Aware Standalone App (generated from FAUST source):

slide-14
SLIDE 14

FAUST Main Programs Supported

Julius Smith ASA-2013 Hot Topics 3pID3 (MU) – 12 / 18

Shell Script

  • Arch. File

Description faust2jack (f2j) jack-gtk.cpp JACK GTK standalone application faust2jaqt jack-qt.cpp JACK QT4 standalone application faust2caqt (f2ca) ca-qt.cpp CoreAudio QT4 standalone application faust2pa pa-gtk.cpp PortAudio GTK standalone application faust2paqt pa-qt.cpp PortAudio QT4 standalone application faust2netjackqt netjack-qt.cpp server with libnetjack support faust2oss

  • ss-gtk.cpp

OSS GTK standalone application faust2alsa alsa-gtk.cpp ALSA GTK standalone application faust2alqt alsa-qt.cpp ALSA QT4 standalone application faust2alsaconsole alsa-console.cpp ALSA terminal program faust2android android.cpp Android phone/tablet application faust2ios ios-coreaudio[-jack].cpp iOS phone/tablet application faust2rpi* [alsa—netjack]-console.cpp Raspberry Pi application sndfile.cpp sound file transformation command bench.cpp speed benchmark faust2octave (f2o)

  • ctave.cpp
  • utput signals to Octave (matlab)
slide-15
SLIDE 15

FAUST Plugin Architectures Supported

Julius Smith ASA-2013 Hot Topics 3pID3 (MU) – 13 / 18

Shell Script

  • Arch. File

Description faust2ladspa ladspa.cpp Linux LADSPA effect plugin faust2dssi dssi.cpp Linux synth plugin, extending LADSPAA faust2lv2 lv2.cpp Linux synth plugin, newer lv2 type faust2lv2-synth lv2synth.cpp Linux synth plugin, newer lv2 type faust2vst vst2p4.cpp VST 2.4 plugin faust2vsti vsti-poly.cpp VSTi instrument faust2au (f2au) au-effect.cpp Apple Audio Unit (AU) effect plugin faust2au au-instrument.cpp Apple AU instrument plugin faust2max max-msp.cpp Max/MSP external faust2puredata puredata.cpp PD external faust2supercollider supercollider.cpp SuperCollider Unit Generator snd-rt-gtk.cpp Snd-RT music programming language faust2q q.cpp Q language plugin faust2csound csound.cpp CSOUND opcode faust2alchemy alchemy-as.cpp Flash/ActionScript plugin (for Web browsers)

slide-16
SLIDE 16

Getting Started with FAUST

Enabling Technologies moForte Guitar Faust Intro

  • FAUST Language
  • Faust ZitaRev1
  • Faust Apps
  • Faust Plugins
  • Faust Pointers

Research

Julius Smith ASA-2013 Hot Topics 3pID3 (MU) – 14 / 18

  • Faust Website: http://faust.grame.fr/

(Click on “Online Examples” and type in some FAUST code!)

  • Faust Intro:

https://ccrma.stanford.edu/~jos/aspf/

(Google Search: “Audio Signal Processing in Faust”)

  • FAUST is Free Open Source Software (FOSS) for

Mac OS, Linux, and Windows

slide-17
SLIDE 17

Ongoing Research

Enabling Technologies moForte Guitar Faust Intro Research

Julius Smith ASA-2013 Hot Topics 3pID3 (MU) – 15 / 18

slide-18
SLIDE 18

Synth-A-Modeler Block-Diagram to FAUST Translator

Enabling Technologies moForte Guitar Faust Intro Research

  • Synth-A-Modeler
  • 2D Bridge Modeling
  • Waveguide Reverb

Julius Smith ASA-2013 Hot Topics 3pID3 (MU) – 16 / 18

  • FAUST is excellent for specifying platform-independent

signal-processing block diagrams

  • For physical models a higher-level front-end is helpful
  • Physical objects need bidirectional connections
  • Linux Audio Conference 2012 (LAC-12) Paper:

http://lac.linuxaudio.org/2012/papers/34.pdf

“An Introduction to the Synth-A-Modeler Compiler: Modular and Open-Source Sound Synthesis using Physical Models,” Edgar Berdahl et al.

  • GPL Free Software:

git clone https://github.com/eberdahl/SaM.git

slide-19
SLIDE 19

2D Bridge Modeling for Bowed Strings

Enabling Technologies moForte Guitar Faust Intro Research

  • Synth-A-Modeler
  • 2D Bridge Modeling
  • Waveguide Reverb

Julius Smith ASA-2013 Hot Topics 3pID3 (MU) – 17 / 18

Reference: E. Maestre, G. P . Scavone, and J. O. Smith III, “Digital modeling of bridge driving-point admittances from measurements on violin-family instruments”, in Proceedings of the Stockholm Musical Acoustics Conference (SMAC-13), http://www.speech.kth.se/smac-smc-2013/. Two-dimensional Bridge Model

slide-20
SLIDE 20

Scattering Delay Networks (SDN)

Enabling Technologies moForte Guitar Faust Intro Research

  • Synth-A-Modeler
  • 2D Bridge Modeling
  • Waveguide Reverb

Julius Smith ASA-2013 Hot Topics 3pID3 (MU) – 18 / 18

“Scattering Delay Networks” by Enzo De Sena, H¨ useyin Hacıhabibo˘ glu, Zoran Cvetkovi´ c, and Julius O. Smith III