Sonificator A Java Framework for writing Applications that use - - PowerPoint PPT Presentation

Sonificator

A Java Framework for writing Applications that use SuperCollider as Sound - Engine

Talk by: Christian Mühlethaler and Alexander Schuppisser Software written by: Christian Mühlethaler, Alexander Schuppisser and Jean-Claude Summermatter

Content of this Talk

 SuperCollider  OSC - Protocol  Sonificator  Sample Applications

SuperCollider

• State
• written by James Mc Cartney for Apple

Macintosh

• now open source. Available at

http://sourceforge.net/projects/supercollider/

• ported to Linux by Stefan Kerstens and
• thers - Don’t miss his talk tomorrow!

SuperCollider

• Overview
• A realtime sound synthesis application
• Two parts: SCSynth and SCLang
• UnitGenerators
• the modules that actually make the music
• implemented as Plug-Ins (C-language)

SuperCollider: Server

• SCSynth: Server
• sound synthesis engine. Produces the

sound

• accessable via the OSC protocoll

(explained later)

• can run on a different machine than the

client

SuperCollider: Server Architecture

• Server Architecture
• a tree of synth modules that are patched

together through global audio- and control busses

• rdered in a tree of nodes that define the
• rder of execution
• rder of execution: the order in which the

server computes the sound for each synth per cycle

SuperCollider: Server Architecture

• Node tree
• Group
• the inner nodes of the node-tree
• a collection of other nodes that can be other

groups or synths

• nodes within a group can be controlled together
• at startup of the server there is a top level group

as root

SuperCollider: Server Architecture

• Node tree (continued)
• Synth
• The leaves of the node tree
• The running modules in the tree: generates or

modifies sound (or control values)

• reads input and write output to global audio and

control busses.

SuperCollider: Server Architecture

 a node tree on the

server

 0 and 1 are groups  2 - 6 are synths  execution order: deep

search post order

SuperCollider: Server Architecture

 Synths and busses.

One possible configuration of the tree

SuperCollider: Synth Definition



the templates for the synth nodes



a patch of unit generators (Ugens)



build and compiled in SCLang, loaded

• n the server

SuperCollider: Synth Definition

( SynthDef("sine", { //name of the synthdef arg freq=500, out=0; //two control parameter var osc;

• sc = SinOsc.ar(freq, 0, 0.1);//sine oscillator

Out.ar(out, osc);//send output to audio bus “out” }).writeDefFile; //compiled and written to disc )

SuperCollider: Server Commands

 communication with the server through

server commands: OSC messages

 structure: commandName [args]  type

• master controls
• /quit []

quits the server

• /dumpOSC [int: code]

displays OSC messages

SuperCollider: Server Commands

 type (continued)

• node commands
• /n-run [int nodeId, int flag]

starts or stops node execution

• /n_set [int nodeId, String controlName, float controlValue]

set a node's control value

• synth commands
• /s_new [String synthDefName, int nodeId, int addAction, int targetId]

creates a new synth from the specified SynthDef

• other commands for
• Groups, Busses, etc

SuperCollider: Server Commands

s = Server.local; s.boot; ( SynthDef("sine", { arg freq=800; var osc;osc = SinOsc.ar(freq, 0, 0.1); Out.ar(0, osc); }).writeDefFile;) s.sendSynthDef("sine"); s.sendMsg("/s_new", "sine", 1000, 1, 0); s.sendMsg("/n_free", 1000); s.quit;

SuperCollider: SCLang

 objected oriented language

• like smalltalk: “everything is an object”
• a lot of features to make control structures for

music (patterns, scheduling...)

• access to the unit generators: every Ugen has its

corresponding SuperCollider class

 compiler for SynthDefinitions

• compiles also classes and loads them into memory

 interpreter

• interprets code and controls the server

Sonificator

 main ideas

• a Java framework that provides the interface to

connect SuperCollider (scsynth and SCLang)

• creating sound from incoming non-musical

information

• providing classes to easily create new instruments
• sound modules that can be connected together in a

tree

• the possibility to use SCLang for co-routines
• easy creation of sound modules in a 2-dimensional

space

OSC - OpenSoundControl Protocol

 used to control real-time sound

applications in a standard way over the network

 little overhead, fast. UDP or TCP/IP  useful: Time-Tags, Bundles, URIs  specification and related links: http://www.cnmat.berkeley.edu/OpenSoundControl/

Soundframeworks supporting OSC

 CPS  Csound  Grainwave  HTM  Intakt  Max/MSP  Open Sound World  Pd  ...  Picker  The Slidepipe  SuperCollider  Reaktor (Native

Instruments)

 RTMix  Sodaconstructor  SpinOSC  Squeak (via Siren) Source: http://www.cnmat.berkeley.edu/OpenSoundControl/

Sonificator: Overview

http://www.substring.ch/sound/

 two examples that

use the Sonificator

 Sonificator with three

layers

 SCLang and

SCSynth as receiver

Sonificator: Layer model

 OSC Layer

• implements the OSC protocol (Open Sound

Control)

• makes the communication possible with

scsynth and SCLang

• some inspiration from the old version of the

"JavaOSC" library

http://www.mat.ucsb.edu/~c.ramakr/illposed/javaosc. html

Sonificator: Layer model

 OSC - Layer (continued)

• Some classes
• OSCPort
• sends the OSC Message (or OSC Bundle) to the receiver
• OSCMessage
• consists of a command and an array of arguments
• OSCBundle
• consists of a time tag and and an array of OSCMessage
• Sends all those messages together
• All messages are executed together
• OSCType
• Superclass for all implemented types: OSCFloat, OSCInt,

OSCString, …

public public class class First { public public static static void void main(String[] args) throws throws Exception{ //the port that finally send and receive osc messages, wrapped intoudp OSCPort port = new new OSCPort(); //create the sine node OSCMessage newSine = new new OSCMessage( "/s_new", //command new new OSCType[] { new new OSCString("sine"), //SynthDefName new new OSCInt(1000), //id new new OSCInt(0), //addAction new new OSCInt(0)}); //int - add target ID //send the new sine port.send(newSine); //a glissando int int max = 1000; for for(int int i = 0; i < max; i++){ //create the new message to set the new pitch OSCMessage value = new new OSCMessage( "/n_set", //command new new OSCType[] { new new OSCInt(1000),//id new new OSCString("freq"), //what new new OSCInt(1000+i)//value }); port.send(value); try try { Thread.sleep(5); } catch catch (InterruptedException e2) { e2.printStackTrace(); } } //freeall OSCMessage freeAll = new new OSCMessage("/g_freeAll", new new OSCType[] {new new OSCInt(0)}); port.send(freeAll); System.exit(0); } }

Sonificator: Layer model

 SuperCollider Layer

• represents a model of the architecture of scsynth, the

SuperCollider server

• you can work with groups, nodes, and synths
• the Java implementation of some SuperCollider

classes in the directory /Common/Control/ of the SCClassLibrary

• Node, Group, Synth, Server, Bus, Buffer
• encapsulates their server commands

Sonificator: Layer model

 sound modules are connected

together

 Generators: generate sound  Processors: modulate sound  GrainRhythm and GrainLine:

lives on SCLang

Sonificator: Layer model

 Sonificator Layer

• abstracts from the SuperCollider layer
• provides an API for creating instruments by

patching sound modules together

• provides a class that simulates the walker in

a 2-dimensional sound landscape

Sonificator: Instruments on SCLang

 SCLang: great scheduling features  Java -> SCLang -> scsynth (Server)  every part can run on different machines  Communication via OSC  Little traffic to SCLang, lot of traffic to

scsynth.

 Master commands from the Java part,

fine-grain commands from SCLang.

How to make realistic accustic Environment?

• time difference between

right an left ear

• the farer the sound, the

lower the volume

• reflections
• HRTF (not implemented)
• diffuse reflections etc.

Sample Application 1: „Sniffer“ using Sonificator

Using library and code from sample application of http://jpcap.sourceforge.net/

Sample Application 2: „Game“ using Sonificator

Using Java3D from http://www.blackdown.org/

Why two Applications?

• Two different programs to make sure the

underlying framework is good

• „Sniffer“ as a technical Application with the

question: How can information be best expressed as sound - that is, to have the most meaning for humans?

• „Game“ as a try to approach best a real

accustic environment