Sonificator A Java Framework for writing Applications that use - PowerPoint PPT Presentation

Sonificator A Java Framework for writing Applications that use SuperCollider as Sound - Engine Software written by: Talk by: Christian Mühlethaler, Christian Mühlethaler and Alexander Schuppisser and Alexander Schuppisser Jean-Claude Summermatter

Content of this Talk  SuperCollider  OSC - Protocol  Sonificator  Sample Applications

SuperCollider  State written by James Mc Cartney for Apple  Macintosh  now open source. Available at http://sourceforge.net/projects/supercollider/  ported to Linux by Stefan Kerstens and others - Don’t miss his talk tomorrow!

SuperCollider  Overview  A realtime sound synthesis application  Two parts: SCSynth and SCLang  UnitGenerators the modules that actually make the music  implemented as Plug-Ins (C-language) 

SuperCollider: Server  SCSynth: Server  sound synthesis engine. Produces the sound accessable via the OSC protocoll  (explained later) can run on a different machine than the  client

SuperCollider: Server Architecture Server Architecture  a tree of synth modules that are patched  together through global audio- and control busses  ordered in a tree of nodes that define the order of execution  order of execution: the order in which the server computes the sound for each synth per cycle

SuperCollider: Server Architecture  Node tree  Group  the inner nodes of the node-tree  a collection of other nodes that can be other groups or synths  nodes within a group can be controlled together  at startup of the server there is a top level group as root

SuperCollider: Server Architecture  Node tree (continued)  Synth  The leaves of the node tree  The running modules in the tree: generates or modifies sound (or control values)  reads input and write output to global audio and control busses.

SuperCollider: Server Architecture  a node tree on the server  0 and 1 are groups  2 - 6 are synths  execution order: deep search post order

SuperCollider: Server Architecture  Synths and busses. One possible configuration of the tree

SuperCollider: Synth Definition the templates for the synth nodes  a patch of unit generators (Ugens)  build and compiled in SCLang, loaded  on the server

SuperCollider: Synth Definition ( SynthDef("sine", { //name of the synthdef arg freq=500, out=0; //two control parameter var osc; osc = SinOsc.ar(freq, 0, 0.1);//sine oscillator Out.ar(out, osc);//send output to audio bus “out” }).writeDefFile; //compiled and written to disc )

SuperCollider: Server Commands  communication with the server through server commands: OSC messages  structure: commandName [args]  type • master controls • /quit [] quits the server • /dumpOSC [int: code] displays OSC messages

SuperCollider: Server Commands  type (continued) • node commands • /n-run [int nodeId, int flag] starts or stops node execution • /n_set [int nodeId, String controlName, float controlValue] set a node's control value • synth commands • /s_new [String synthDefName, int nodeId, int addAction, int targetId] creates a new synth from the specified SynthDef • other commands for • Groups, Busses, etc

SuperCollider: Server Commands s = Server.local; s.boot; ( SynthDef("sine", { arg freq=800; var osc;osc = SinOsc.ar(freq, 0, 0.1); Out.ar(0, osc); }).writeDefFile;) s.sendSynthDef("sine"); s.sendMsg("/s_new", "sine", 1000, 1, 0); s.sendMsg("/n_free", 1000); s.quit;

SuperCollider: SCLang  objected oriented language • like smalltalk: “everything is an object” • a lot of features to make control structures for music (patterns, scheduling...) • access to the unit generators: every Ugen has its corresponding SuperCollider class  compiler for SynthDefinitions • compiles also classes and loads them into memory  interpreter • interprets code and controls the server

Sonificator  main ideas • a Java framework that provides the interface to connect SuperCollider (scsynth and SCLang) • creating sound from incoming non-musical information • providing classes to easily create new instruments • sound modules that can be connected together in a tree • the possibility to use SCLang for co-routines • easy creation of sound modules in a 2-dimensional space

OSC - OpenSoundControl Protocol  used to control real-time sound applications in a standard way over the network  little overhead, fast. UDP or TCP/IP  useful: Time-Tags, Bundles, URIs  specification and related links: http://www.cnmat.berkeley.edu/OpenSoundControl/

Soundframeworks supporting OSC  CPS  Picker  Csound  The Slidepipe  Grainwave  SuperCollider  HTM  Reaktor (Native Instruments)  Intakt  RTMix  Max/MSP  Sodaconstructor  Open Sound World  SpinOSC  Pd  Squeak (via Siren)  ... Source: http://www.cnmat.berkeley.edu/OpenSoundControl/

Sonificator: Overview http://www.substring.ch/sound/  two examples that use the Sonificator  Sonificator with three layers  SCLang and SCSynth as receiver

Sonificator: Layer model  OSC Layer • implements the OSC protocol (Open Sound Control) • makes the communication possible with scsynth and SCLang • some inspiration from the old version of the "JavaOSC" library http://www.mat.ucsb.edu/~c.ramakr/illposed/javaosc. html

Sonificator: Layer model  OSC - Layer (continued) • Some classes • OSCPort • sends the OSC Message (or OSC Bundle) to the receiver • OSCMessage • consists of a command and an array of arguments • OSCBundle • consists of a time tag and and an array of OSCMessage • Sends all those messages together • All messages are executed together • OSCType • Superclass for all implemented types: OSCFloat, OSCInt, OSCString, …

public public class class First { public public static static void void main(String[] args) throws throws Exception{ //the port that finally send and receive osc messages, wrapped intoudp OSCPort port = new new OSCPort(); //create the sine node OSCMessage newSine = new new OSCMessage( "/s_new", //command new new OSCType[] { new new OSCString("sine"), //SynthDefName new new OSCInt(1000), //id new new OSCInt(0), //addAction new new OSCInt(0)}); //int - add target ID //send the new sine port.send(newSine); //a glissando int int max = 1000; for for(int int i = 0; i < max; i++){ //create the new message to set the new pitch OSCMessage value = new new OSCMessage( "/n_set", //command new new OSCType[] { new new OSCInt(1000),//id new new OSCString("freq"), //what new new OSCInt(1000+i)//value }); port.send(value); try try { Thread.sleep(5); } catch catch (InterruptedException e2) { e2.printStackTrace(); } } //freeall OSCMessage freeAll = new new OSCMessage("/g_freeAll", new new OSCType[] {new new OSCInt(0)}); port.send(freeAll); System.exit(0); } }

Sonificator: Layer model  SuperCollider Layer • represents a model of the architecture of scsynth, the SuperCollider server • you can work with groups, nodes, and synths • the Java implementation of some SuperCollider classes in the directory /Common/Control/ of the SCClassLibrary • Node, Group, Synth, Server, Bus, Buffer • encapsulates their server commands

Sonificator: Layer model  sound modules are connected together  Generators: generate sound  Processors: modulate sound  GrainRhythm and GrainLine: lives on SCLang

Sonificator: Layer model  Sonificator Layer • abstracts from the SuperCollider layer • provides an API for creating instruments by patching sound modules together • provides a class that simulates the walker in a 2-dimensional sound landscape

Sonificator: Instruments on SCLang  SCLang: great scheduling features  Java -> SCLang -> scsynth (Server)  every part can run on different machines  Communication via OSC  Little traffic to SCLang, lot of traffic to scsynth.  Master commands from the Java part, fine-grain commands from SCLang.

How to make realistic accustic Environment? • time difference between right an left ear • the farer the sound, the lower the volume • reflections • HRTF (not implemented) • diffuse reflections etc.

Sample Application 1: „Sniffer“ using Sonificator Using library and code from sample application of http://jpcap.sourceforge.net/

Sample Application 2: „Game“ using Sonificator Using Java3D from http://www.blackdown.org/

Why two Applications? • Two different programs to make sure the underlying framework is good • „Sniffer“ as a technical Application with the question: How can information be best expressed as sound - that is, to have the most meaning for humans? • „Game“ as a try to approach best a real accustic environment