Rtosc Realtime Open Sound Control Mark McCurry 2018 Rtosc - - PowerPoint PPT Presentation

▶

Aug 30, 2023 335 likes •705 views

Rtosc Realtime Open Sound Control Rtosc Realtime Open Sound Control Mark McCurry 2018 Rtosc Realtime Open Sound Control Motivation What is OSC? Path + argument types + argument data Types include: i : int32 , s : string , b :

SLIDE 1

Rtosc Realtime Open Sound Control

Mark McCurry 2018

SLIDE 2

Rtosc Realtime Open Sound Control Motivation

What is OSC?

◮ Path + argument types + argument data ◮ Types include: i:int32, s:string, b:binary-blob, f:float32,

h:int64, t:timetag, d:float64, S:symbol, r:rgb, m:4-byte-MIDI, c:int8, T:true, F:false, N:nil, and I:Inf.

SLIDE 3

Rtosc Realtime Open Sound Control Motivation

What is OSC?

◮ Message serialization ◮ Semi-complex inter-process communication

SLIDE 4

Rtosc Realtime Open Sound Control Motivation

What Doesn’t OSC Normally Do?

SLIDE 5

Rtosc Realtime Open Sound Control Motivation

What Does Rtosc Add?

◮ Low level Serialization (C) ◮ High Level Dispatch/Metadata (C++) ◮ Restricted problem domain

SLIDE 6

Rtosc Realtime Open Sound Control C core

OSC Serialization

char buffer[256];

SLIDE 7

Rtosc Realtime Open Sound Control C core

OSC Serialization

char buffer[256]; int ret = rtosc_message(buffer, sizeof(buffer), "/path", "si", "some arguments", 123);

SLIDE 8

Rtosc Realtime Open Sound Control C core

OSC Serialization

char buffer[256]; int ret = rtosc_message(NULL, 0, "/path", "si", "some arguments", 123);

SLIDE 9

Rtosc Realtime Open Sound Control C core

OSC Serialization

char buffer[256]; int ret = rtosc_message(buffer, sizeof(buffer), "/path", "si", "some arguments", 123); const char *args = rtosc_argument_string(buffer);

SLIDE 10

Rtosc Realtime Open Sound Control C core

OSC Serialization

char buffer[256]; int ret = rtosc_message(buffer, sizeof(buffer), "/path", "si", "some arguments", 123); const char args = rtosc_argument_string(buffer); const char first_arg = rtosc_argument(buffer, 0).s; int second_arg = rtosc_argument(buffer, 1).i;

SLIDE 11

Rtosc Realtime Open Sound Control Dispatch API

Working With Messages

◮ Messages need to be dispatched to handle them

SLIDE 12

Rtosc Realtime Open Sound Control Dispatch API

Working With Messages

◮ Messages need to be dispatched to handle them ◮ ZynAddSubFX has a lot of parameters

SLIDE 13

Rtosc Realtime Open Sound Control Dispatch API

Working With Messages

◮ Messages need to be dispatched to handle them ◮ ZynAddSubFX has a lot of parameters ◮ Dispatch needs to be fast

SLIDE 14

Rtosc Realtime Open Sound Control Dispatch API

Dispatch Tree

Message: /foo/bar Tree:-| abc | xxx | yyy | foo() +-|qwerty |path |bar()

SLIDE 15

Rtosc Realtime Open Sound Control Dispatch API

Dispatch Example

struct Envelope { float attack, decay, release; };

SLIDE 16

Rtosc Realtime Open Sound Control Dispatch API

Dispatch Example

struct Envelope { float attack, decay, release; }; rtosc::Ports ports = { {"attack", NULL, NULL, [](const char *msg, rtosc::RtData &rt) { }}, };

SLIDE 17

Rtosc Realtime Open Sound Control Dispatch API

Dispatch Example

struct Envelope { float attack, decay, release; }; rtosc::Ports ports = { {"attack:f", NULL, NULL, [](const char msg, rtosc::RtData &rt) { Envelope &obj = (Envelope*)rt.obj;

bj.attack = rtosc_argument(msg, 0).f;

}}, };

SLIDE 18

Rtosc Realtime Open Sound Control Dispatch API

Dispatch Example

struct Envelope { float attack, decay, release; }; rtosc::Ports ports = { {"attack:f", ":max\0=15\0", NULL, [](const char msg, rtosc::RtData &rt) { Envelope &obj = (Envelope*)rt.obj;

bj.attack = rtosc_argument(msg, 0).f;

}}, };

SLIDE 19

Rtosc Realtime Open Sound Control Dispatch API

Dispatch Example

struct Envelope { float attack, decay, release; }; #define rObject Envelope rtosc::Ports ports = { rParamF(attack, rLinear(0, 15), rMap(unit, sec), "Attack Time"), };

SLIDE 20

Rtosc Realtime Open Sound Control Dispatch API

Dispatch Example

struct Envelope { float attack, decay, release; }; #define rObject Envelope rtosc::Ports ports = { rParamF(attack, rLinear(0, 15), rMap(unit, sec), "Attack Time"), rParamF(decay, rLinear(0, 15), rMap(unit, sec), "Decay Time"), rParamF(release, rLinear(0, 15), rMap(unit, sec), "Release Time"), };

SLIDE 21

Rtosc Realtime Open Sound Control Dispatch API Metadata

Metadata

◮ Minimum/Maximum ◮ Linear/Log scaling ◮ Documentation strings ◮ Units ◮ Option symbol ⇀ value mappings

SLIDE 22

Rtosc Realtime Open Sound Control Dispatch API Metadata

Metadata

◮ Reflection ◮ Avoids repetition ◮ Keeps information near code use

SLIDE 23

Rtosc Realtime Open Sound Control Dispatch API Metadata

Metadata Improvements

◮ osc-doc API reference ◮ Learning MIDI/Plugin-host bindings ◮ Reuse of metadata in generating the GUI

SLIDE 24

Rtosc Realtime Open Sound Control Dispatch API Speed

Rtosc Performance

◮ Rtosc Is fast

SLIDE 25

Rtosc Realtime Open Sound Control Dispatch API Speed

Rtosc Performance

◮ Rtosc Is fast ◮ No really

SLIDE 26

Rtosc Realtime Open Sound Control Dispatch API Speed

Sonic Pi - Integration

Impl per op

ps per second

speedup Encoding an average message fast osc 1.2 us 800,000 9.6x samsosc 3.8 us 260,000 3.1x

sc-ruby

12 us 83,000 – Decoding an average message fast osc 0.6 us 1,700,000 50x samsosc 4.7 us 230,000 7.4x

sc-ruby

29 us 34,000 –

SLIDE 27

Rtosc Realtime Open Sound Control Dispatch API Speed

Liblo point of comparison

Impl. per op

ps per second

speedup Decoding an average message liblo 218 ns 4,600,000

rtosc

53 ns 19,000,000 4.1x Encoding an average message liblo 383 ns 2,600,000

rtosc

125 ns 8,000,000 3.1x Dispatch message on single layer liblo 530 ns 1,900,000

rtosc

54 ns 19,000,000 10x

SLIDE 28

Rtosc Realtime Open Sound Control Dispatch API Speed

Liblo algorithm scaling

ZynAddSubFX has:

◮ 3,805,225 unique OSC paths ◮ e.g. /part1/kit5/adpars/VoicePar7/AmpLfo/Pfreq ◮ An average depth of 6.11 subpaths ◮ With minimal hashing an average of 6.11 matches are needed

for rtosc, and 3,805,225 for liblo

SLIDE 29

Rtosc Realtime Open Sound Control Dispatch API Speed

Liblo algorithm scaling

◮ Liblo match time: 18.3 ms ◮ Rtosc match time: 380 ns

SLIDE 30

Rtosc Realtime Open Sound Control Dispatch API Speed

Liblo algorithm scaling

◮ Liblo match time: 18.3 ms ◮ Rtosc match time: 380 ns ◮ Liblo: ≈55 messages per second ◮ Rtosc: ≈2,600,000 messages per second

SLIDE 31

Rtosc Realtime Open Sound Control Dispatch API Speed

Discussion of Trade offs

◮ Fast, but maintainable

SLIDE 32

Rtosc Realtime Open Sound Control Dispatch API Speed

A comparison

#d e f i n e rObject LFOParams #undef rChangeCb #d e f i n e rChangeCb i f ( obj− >time ) { obj− >last update timestamp = obj− >time− >time ( ) ; } s t a t i c const r t o s c : : Ports p o r t s = { r S e l f (LFOParams ) , rPaste , rOption ( loc , rProp ( i n t e r n a l ) , rOptions ( ad global amp , a d g l o b a l f r e q , a d g l o b a l f i l t e r , ad voice amp , a d v o i c e f r e q , a d v o i c e f i l t e r , u n s p e c i f i e d ) , ” l o c a t i o n

the f i l t e r ”) , rParamF ( Pfreq , rShort (” f r e q ”) , r L i n e a r ( 0 . 0 , 1 . 0 ) , rDefaultDepends ( l o c ) , r P r e s e t ( ad global amp , 0x1 .42850 ap −1) , // 80 r P r e s e t ( a d g l o b a l f r e q , 0x1 .1 a3468p −1) , // 70 r P r e s e t ( a d g l o b a l f i l t e r , 0x1 .42850 ap −1) , r P r e s e t ( ad voice amp , 0x1 .6 ad5acp −1) , // 90 r P r e s e t ( a d v o i c e f r e q , 0x1 .93264 cp −2) , // 50 r P r e s e t ( a d v o i c e f i l t e r , 0x1 .93264 cp −2) , ” frequency

f LFO\n”

” l f o frequency = (2ˆ(10∗ Pfreq )−1)/12 ∗ s t r e t c h \n” ” true frequency i s [ 0 , 8 5 . 3 3 ] Hz”) , rParamZyn ( P i n t e n s i t y , rShort (” depth ”) , rDefaultDepends ( l o c ) , r D e f a u l t (0) , r P r e s e t ( ad voice amp , 32) , r P r e s e t ( a d v o i c e f r e q , 40) , r P r e s e t ( a d v o i c e f i l t e r , 20) , ” I n t e n s i t y

f LFO”) ,

rParamZyn ( Pstartphase , rShort (” s t a r t ”) , r S p e c i a l ( random ) , rDefaultDepends ( l o c ) , r D e f a u l t (64) , r P r e s e t ( a d v o i c e f r e q , 0) , ” S t a r t i n g Phase ”) , rOption ( PLFOtype , rShort (” type ”) , rOptions ( sine , t r i a n g l e , square , up , down , exp1 , exp2 ) , r D e f a u l t ( s i n e ) , ”Shape

f LFO”) ,

rParamZyn ( Prandomness , rShort (” a . r . ” ) , r S p e c i a l ( d i s a b l e ) , r D e f a u l t (0) , ” Amplitude Randomness ( c a l c u l a t e d uniformly at each c y c l e )”) , rParamZyn ( Pfreqrand , rShort (” f . r . ” ) , r S p e c i a l ( d i s a b l e ) , r D e f a u l t (0) , ” Frequency Randomness ( c a l c u l a t e d uniformly at each c y c l e )”) , rParamZyn ( Pdelay , rShort (” delay ”) , r S p e c i a l ( d i s a b l e ) , rDefaultDepends ( l o c ) , r D e f a u l t (0) , r P r e s e t ( ad voice amp , 30) , ” Delay b e fo re LFO s t a r t \n0 . . 4 second delay ”) , rToggle ( Pcontinous , rShort (” c ”) , r D e f a u l t ( f a l s e ) , ” Enable f o r g l o b a l

p e r a ti o n ”) ,

rParamZyn ( Pstretch , rShort (” s t r ”) , rCentered , r D e f a u l t (64) , ”Note frequency s t r e t c h ”) , }; #undef rChangeCb

s t r i n g InterChange : : r e s o l v e E n v e l o p e ( CommandBlock ∗ getData ) { i n t value = l r i n t ( getData− >data . value ) ; bool w r i t e = ( getData− >data . type & 0x40 ) > 0; unsigned char c o n t r o l = getData− >data . c o n t r o l ; unsigned char npart = getData− >data . part ; unsigned char k i t i t e m = getData− >data . k i t ; unsigned char engine = getData− >data . engine ; unsigned char i n s e r t = getData− >data . i n s e r t ; unsigned char insertParam = getData− >data . parameter ; i n t par2 = getData− >data . par2 ; s t r i n g env ; s t r i n g name ; i f ( engine == 0) name = ” AddSynth ”; e l s e i f ( engine == 1) name = ” SubSynth ”; e l s e i f ( engine == 2) name = ” PadSynth ”; e l s e i f ( engine >= 0x80 ) { name = ” Add Voice ”; i n t nvoice = engine & 0 x3f ; name += t o s t r i n g ( nvoice + 1 ) ; i f ( engine >= 0xC0 ) name += ” Modulator ”; } switch ( insertParam ) { case 0: env = ” Amp”; break ; case 1: env = ” Freq ”; break ; case 2: env = ” F i l t ”; break ; case 3: env = ” B. Width ”; break ; } i f ( i n s e r t == 3) { i f ( ! w r i t e ) { r e t u r n (” Freemode add/remove i s w r i t e

nly .

Current p o i n t s ” + t o s t r i n g ( i n t ( par2 ) ) ) ; } i f ( par2 != NO MSG) r e t u r n (” Part ” + t o s t r i n g ( i n t ( npart + 1)) + ” Kit ” + t o s t r i n g ( i n t ( k i t i t e m + 1)) + name + env + ” Env Added Freemode Point ” + t o s t r i n g ( i n t (( c o n t r o l & 0 x3f ) + 1)) + ” X increment ” + t o s t r i n g ( i n t ( par2 )) + ” Y” ) ; e l s e { showValue = f a l s e ; r e t u r n (” Part ” + t o s t r i n g ( i n t ( npart + 1)) + ” Kit ” + t o s t r i n g ( i n t ( k i t i t e m + 1)) + name + env + ” Env Removed Freemode Point ” + t o s t r i n g ( i n t ( c o n t r o l + 1)) + ” Remaining ” + t o s t r i n g ( value ) ) ; } } i f ( i n s e r t == 4) { r e t u r n (” Part ” + t o s t r i n g ( i n t ( npart + 1)) + ” Kit ” + t o s t r i n g ( i n t ( k i t i t e m + 1)) + name + env + ” Env Freemode Point ” + t o s t r i n g ( i n t ( c o n t r o l + 1)) + ” X increment ” + t o s t r i n g ( i n t ( par2 )) + ” Y” ) ; } s t r i n g c o n t s t r ; switch ( c o n t r o l ) { case 0: c o n t s t r = ”A v a l ”; break ; case 1: c o n t s t r = ”A dt ”; break ; case 2: c o n t s t r = ”D v a l ”; break ; case 3: c o n t s t r = ”D dt ”; break ; case 4: c o n t s t r = ”S v a l ”; break ; case 5: c o n t s t r = ”R dt ”; break ; case 6: c o n t s t r = ”R v a l ”; break ; case 7: c o n t s t r = ” Stretch ”; break ; case 16: c o n t s t r = ” frcR ”; break ; case 17: c o n t s t r = ”L ”; break ; case 24: c o n t s t r = ” Edit ”; break ; case 32: c o n t s t r = ”Freemode ”; break ; case 34: c o n t s t r = ” Points ”; c o n t s t r += t o s t r i n g (( i n t ) par2 ) ; break ; case 35: c o n t s t r = ” Sust ”; break ; d e f a u l t : showValue = f a l s e ; c o n t s t r = ” Unrecognised ”; } r e t u r n (” Part ” + t o s t r i n g ( npart + 1) + ” Kit ” + t o s t r i n g ( i n t ( k i t i t e m + 1)) + name + env + ” Env ” + c o n t s t r ) ; } void InterChange : : commandEnvelope ( CommandBlock ∗ getData ) { unsigned char npart = getData− >data . part ; unsigned char k i t i t e m = getData− >data . k i t ; unsigned char engine = getData− >data . engine ; unsigned char insertParam = getData− >data . parameter ; Part ∗ part ; part = synth− >part [ npart ] ; s t r i n g env ; s t r i n g name ; i f ( engine == 0) { switch ( insertParam ) { case 0: envelopeReadWrite ( getData , part − >k i t [ k i t i t e m ] . adpars− >GlobalPar . AmpEnvelope ) ; break ; case 1: envelopeReadWrite ( getData , part − >k i t [ k i t i t e m ] . adpars− >GlobalPar . FreqEnvelope ) ; break ; case 2: envelopeReadWrite ( getData , part − >k i t [ k i t i t e m ] . adpars− >GlobalPar . F i l t e r E n v e l o p e ) ; break ; } } e l s e i f ( engine == 1) { switch ( insertParam ) { case 0: envelopeReadWrite ( getData , part − >k i t [ k i t i t e m ] . subpars − >AmpEnvelope ) ; break ; case 1: envelopeReadWrite ( getData , part − >k i t [ k i t i t e m ] . subpars − >FreqEnvelope ) ; break ; case 2: envelopeReadWrite ( getData , part − >k i t [ k i t i t e m ] . subpars − >G l o b a l F i l t e r E n v e l o p e ) ; break ; case 3: envelopeReadWrite ( getData , part − >k i t [ k i t i t e m ] . subpars − >BandWidthEnvelope ) ; break ; } } e l s e i f ( engine == 2) { switch ( insertParam ) { case 0: envelopeReadWrite ( getData , part − >k i t [ k i t i t e m ] . padpars− >AmpEnvelope ) ; break ; case 1: envelopeReadWrite ( getData , part − >k i t [ k i t i t e m ] . padpars− >FreqEnvelope ) ; break ; case 2: envelopeReadWrite ( getData , part − >k i t [ k i t i t e m ] . padpars− >F i l t e r E n v e l o p e ) ; break ; } } e l s e i f ( engine >= 0x80 ) { i n t nvoice = engine & 0 x3f ; i f ( engine >= 0xC0 ) { switch ( insertParam ) { case 0: envelopeReadWrite ( getData , part − >k i t [ k i t i t e m ] . adpars− >VoicePar [ nvoice ] . FMAmpEnvelope ) ; break ; case 1: envelopeReadWrite ( getData , part − >k i t [ k i t i t e m ] . adpars− >VoicePar [ nvoice ] . FMFreqEnvelope ) ; break ; } } e l s e { switch ( insertParam ) { case 0: envelopeReadWrite ( getData , part − >k i t [ k i t i t e m ] . adpars− >VoicePar [ nvoice ] . AmpEnvelope ) ; break ; case 1: envelopeReadWrite ( getData , part − >k i t [ k i t i t e m ] . adpars− >VoicePar [ nvoice ] . FreqEnvelope ) ; break ; case 2: envelopeReadWrite ( getData , part − >k i t [ k i t i t e m ] . adpars− >VoicePar [ nvoice ] . F i l t e r E n v e l o p e ) ; break ; } } } } void InterChange : : envelopeReadWrite ( CommandBlock ∗getData , EnvelopeParams ∗ pars ) { i n t v a l = l r i n t ( getData− >data . value ) ; // these are a l l i n t e g e r s

bool bool w r i t e = ( getData− >data . type & 0x40 ) > 0; i f ( w r i t e ) s y n c o r a n d f e t c h (&blockRead , 1 ) ; unsigned char point = getData− >data . c o n t r o l ; unsigned char i n s e r t = getData− >data . i n s e r t ; unsigned char Xincrement = getData− >data . par2 ; i n t envpoints = pars− >Penvpoints ; bool isAddpoint = ( Xincrement < 0 x f f ) ; i f ( i n s e r t == 3) // here be dragons : ( { i f ( ! pars− >Pfreemode ) { getData− >data . value = 0 x f f ; getData− >data . par2 = 0 x f f ; r e t u r n ; } i f ( ! w r i t e | | point == 0 | | point >= envpoints ) { getData− >data . value = 0 x f f ; getData− >data . par2 = envpoints ; r e t u r n ; } i f ( isAddpoint ) { i f ( envpoints < MAX ENVELOPE POINTS) { pars− >Penvpoints += 1; f o r ( i n t i = envpoints ; i >= point ; −− i ) { pars− >Penvdt [ i + 1] = pars− >Penvdt [ i ] ; pars− >Penvval [ i + 1] = pars− >Penvval [ i ] ; } i f ( point == 0) pars− >Penvdt [ 1 ] = 64; i f ( point <= pars− >Penvsustain ) ++ pars− >Penvsustain ; pars− >Penvdt [ point ] = Xincrement ; pars− >Penvval [ point ] = v a l ; getData− >data . value = v a l ; getData− >data . par2 = Xincrement ; } e l s e getData− >data . value = 0 x f f ; r e t u r n ; } e l s e i f ( envpoints < 4) { getData− >data . value = 0 x f f ; getData− >data . par2 = 0 x f f ; r e t u r n ; // can ’ t have l e s s than 4 } e l s e { envpoints −= 1; f o r ( i n t i = point ; i < envpoints ; ++ i ) { pars− >Penvdt [ i ] = pars− >Penvdt [ i + 1 ] ; pars− >Penvval [ i ] = pars− >Penvval [ i + 1 ] ; } i f ( point <= pars− >Penvsustain ) −− pars− >Penvsustain ; pars− >Penvpoints = envpoints ; getData− >data . value = envpoints ; } r e t u r n ; } i f ( i n s e r t == 4) { i f ( ! pars− >Pfreemode | | point >= envpoints ) { getData− >data . value = 0 x f f ; getData− >data . par2 = 0 x f f ; r e t u r n ; } i f ( w r i t e ) { pars− >Penvval [ point ] = v a l ; i f ( point == 0) Xincrement = 0; e l s e pars− >Penvdt [ point ] = Xincrement ; } e l s e { v a l = pars− >Penvval [ point ] ; Xincrement = pars− >Penvdt [ point ] ; } getData− >data . value = v a l ; getData− >data . par2 = Xincrement ; r e t u r n ; } switch ( getData− >data . c o n t r o l ) { case 0: i f ( w r i t e ) pars− >PA val = v a l ; e l s e v a l = pars− >PA val ; break ; case 1: i f ( w r i t e ) pars− >PA dt = v a l ; e l s e v a l = pars− >PA dt ; break ; case 2: i f ( w r i t e ) pars− >PD val = v a l ; e l s e v a l = pars− >PD val ; break ; case 3: i f ( w r i t e ) pars− >PD dt = v a l ; e l s e v a l = pars− >PD dt ; break ; case 4: i f ( w r i t e ) pars− >PS val = v a l ; e l s e v a l = pars− >PS val ; break ; case 5: i f ( w r i t e ) pars− >PR dt = v a l ; e l s e v a l = pars− >PR dt ; break ; case 6: i f ( w r i t e ) pars− >PR val = v a l ; e l s e v a l = pars− >PR val ; break ; case 7: i f ( w r i t e ) pars− >Penvstretch = v a l ; e l s e v a l = pars− >Penvstretch ; break ; case 16: i f ( w r i t e ) pars− >P f o r c e d r e l e a s e = ( v a l != 0 ) ; e l s e v a l = pars− >P f o r c e d r e l e a s e ; break ; case 17: i f ( w r i t e ) pars− >P l i n e a r e n v e l o p e = ( v a l != 0 ) ; e l s e v a l = pars− >P l i n e a r e n v e l o p e ; break ; case 24: // t h i s i s l o c a l to the source break ; case 32: i f ( w r i t e ) { i f ( v a l != 0) pars− >Pfreemode = 1; e l s e pars− >Pfreemode = 0; } e l s e v a l = pars− >Pfreemode ; break ; case 34: i f ( ! pars− >Pfreemode ) { v a l = 0 x f f ; Xincrement = 0 x f f ; } e l s e Xincrement = envpoints ; break ; case 35: i f ( w r i t e ) pars− >Penvsustain = v a l ; e l s e v a l = pars− >Penvsustain ; break ; } getData− >data . value = v a l ; getData− >data . par2 = Xincrement ; r e t u r n ; }

SLIDE 33

Rtosc Realtime Open Sound Control Dispatch API Speed

Current/Future Work

◮ Automations ◮ Automated Analysis of Trees ◮ Faster message encode/decode ◮ More standardized metadata

SLIDE 34

Rtosc Realtime Open Sound Control Dispatch API Speed

Conclusions

Rtosc is:

◮ A way to handle OSC Inside RT apps ◮ A library designed to retrofit existing apps ◮ Powerful thanks to low level API and metadata powered high

level

◮ Maintainable ◮ Fast

SLIDE 35

Rtosc Realtime Open Sound Control Dispatch API Speed