An object oriented model for the representation of temporal data in - - PowerPoint PPT Presentation

▶
an object oriented model for the representation of
SMART_READER_LITE
LIVE PREVIEW

An object oriented model for the representation of temporal data in - - PowerPoint PPT Presentation

Mar 25, 2023 97 likes •731 views

An object oriented model for the representation of temporal data in the Integra framework James Bullock 1 Henrik Frisk 2 1 Music Technology Department at Birmingham Conservatoire Birmingham City University 2 Composition Department at Malm

slide-1
SLIDE 1

An object oriented model for the representation of temporal data in the Integra framework

James Bullock1 Henrik Frisk2

1Music Technology Department at Birmingham Conservatoire

Birmingham City University

2Composition Department at Malmö Academy of Music

Lund University

ICMC 2009

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 1 / 18

slide-2
SLIDE 2

What is Integra?

  • “A European Composition and Performance Environment for

Sharing Live Music Technologies”

  • An EC financed project led by Birmingham Conservatoire in the

UK

  • Attempts to address the problems of persistent storage, portability

and standardized intercommunication between systems for electronic music.

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 2 / 18

slide-3
SLIDE 3

What is Integra?

  • “A European Composition and Performance Environment for

Sharing Live Music Technologies”

  • An EC financed project led by Birmingham Conservatoire in the

UK

  • Attempts to address the problems of persistent storage, portability

and standardized intercommunication between systems for electronic music.

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 2 / 18

slide-4
SLIDE 4

What is Integra?

  • “A European Composition and Performance Environment for

Sharing Live Music Technologies”

  • An EC financed project led by Birmingham Conservatoire in the

UK

  • Attempts to address the problems of persistent storage, portability

and standardized intercommunication between systems for electronic music.

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 2 / 18

slide-5
SLIDE 5

Objective

For this research the objectives are:

  • To complement the (synchronous) Integra module with the ability

to store and edit time-based data.

  • Integra module?
  • An abstract definition (and possible implementation) of a DSP

process, a documentation item or a relation between modules.

  • It should be possible to use the same time data regardless of

module implementation.

  • It should be possible to extend and alter existing work.
  • The work here is a proposal, and work is currently at the

development stage.

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 3 / 18

slide-6
SLIDE 6

Objective

For this research the objectives are:

  • To complement the (synchronous) Integra module with the ability

to store and edit time-based data.

  • Integra module?
  • An abstract definition (and possible implementation) of a DSP

process, a documentation item or a relation between modules.

  • It should be possible to use the same time data regardless of

module implementation.

  • It should be possible to extend and alter existing work.
  • The work here is a proposal, and work is currently at the

development stage.

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 3 / 18

slide-7
SLIDE 7

Objective

For this research the objectives are:

  • To complement the (synchronous) Integra module with the ability

to store and edit time-based data.

  • Integra module?
  • An abstract definition (and possible implementation) of a DSP

process, a documentation item or a relation between modules.

  • It should be possible to use the same time data regardless of

module implementation.

  • It should be possible to extend and alter existing work.
  • The work here is a proposal, and work is currently at the

development stage.

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 3 / 18

slide-8
SLIDE 8

Objective

For this research the objectives are:

  • To complement the (synchronous) Integra module with the ability

to store and edit time-based data.

  • Integra module?
  • An abstract definition (and possible implementation) of a DSP

process, a documentation item or a relation between modules.

  • It should be possible to use the same time data regardless of

module implementation.

  • It should be possible to extend and alter existing work.
  • The work here is a proposal, and work is currently at the

development stage.

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 3 / 18

slide-9
SLIDE 9

Objective

For this research the objectives are:

  • To complement the (synchronous) Integra module with the ability

to store and edit time-based data.

  • Integra module?
  • An abstract definition (and possible implementation) of a DSP

process, a documentation item or a relation between modules.

  • It should be possible to use the same time data regardless of

module implementation.

  • It should be possible to extend and alter existing work.
  • The work here is a proposal, and work is currently at the

development stage.

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 3 / 18

slide-10
SLIDE 10

Objective

For this research the objectives are:

  • To complement the (synchronous) Integra module with the ability

to store and edit time-based data.

  • Integra module?
  • An abstract definition (and possible implementation) of a DSP

process, a documentation item or a relation between modules.

  • It should be possible to use the same time data regardless of

module implementation.

  • It should be possible to extend and alter existing work.
  • The work here is a proposal, and work is currently at the

development stage.

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 3 / 18

slide-11
SLIDE 11

Related work (music)

MIDI

The (still?) dominant mechanism for time based information.

SDIF and GDIF

Spectral and gestural data. May be incorporated in IXD.

MetriXML

CLAM’s XML based score file format. Similar to IXD sequences.

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 4 / 18

slide-12
SLIDE 12

Related work (music)

MIDI

The (still?) dominant mechanism for time based information.

SDIF and GDIF

Spectral and gestural data. May be incorporated in IXD.

MetriXML

CLAM’s XML based score file format. Similar to IXD sequences.

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 4 / 18

slide-13
SLIDE 13

Related work (music)

MIDI

The (still?) dominant mechanism for time based information.

SDIF and GDIF

Spectral and gestural data. May be incorporated in IXD.

MetriXML

CLAM’s XML based score file format. Similar to IXD sequences.

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 4 / 18

slide-14
SLIDE 14

Related work (general)

SMIL

A W3C endorsed multimedia format for synchronizing multimedia.

RDF

A language for describing resources (on the web).

OWL

Exhaustive (RDF related) language for descriptive ontologies.

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 5 / 18

slide-15
SLIDE 15

Related work (general)

SMIL

A W3C endorsed multimedia format for synchronizing multimedia.

RDF

A language for describing resources (on the web).

OWL

Exhaustive (RDF related) language for descriptive ontologies.

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 5 / 18

slide-16
SLIDE 16

Related work (general)

SMIL

A W3C endorsed multimedia format for synchronizing multimedia.

RDF

A language for describing resources (on the web).

OWL

Exhaustive (RDF related) language for descriptive ontologies.

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 5 / 18

slide-17
SLIDE 17

The Integra framework

Environment A Environment B

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 6 / 18

slide-18
SLIDE 18

The Integra framework

Environment A Environment B libIntegra

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 6 / 18

slide-19
SLIDE 19

The Integra framework

Environment A Environment B player libIntegra player

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 6 / 18

slide-20
SLIDE 20

The Integra framework

Environment A Environment B player libIntegra player IXD

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 6 / 18

slide-21
SLIDE 21

The player module

Schedule events

  • continuously in Sequences
  • statically as in state changes in Presets

Player features

  • looped and reverse-looped playback of sequenced data
  • random access to sequence data
  • non-linear sequences
  • relative representation of time (non-absolute)
  • non-track based

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 7 / 18

slide-22
SLIDE 22

The player module

Schedule events

  • continuously in Sequences
  • statically as in state changes in Presets

Player features

  • looped and reverse-looped playback of sequenced data
  • random access to sequence data
  • non-linear sequences
  • relative representation of time (non-absolute)
  • non-track based

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 7 / 18

slide-23
SLIDE 23

The player module

Schedule events

  • continuously in Sequences
  • statically as in state changes in Presets

Player features

  • looped and reverse-looped playback of sequenced data
  • random access to sequence data
  • non-linear sequences
  • relative representation of time (non-absolute)
  • non-track based

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 7 / 18

slide-24
SLIDE 24

The player module

Schedule events

  • continuously in Sequences
  • statically as in state changes in Presets

Player features

  • looped and reverse-looped playback of sequenced data
  • random access to sequence data
  • non-linear sequences
  • relative representation of time (non-absolute)
  • non-track based

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 7 / 18

slide-25
SLIDE 25

The player module

Schedule events

  • continuously in Sequences
  • statically as in state changes in Presets

Player features

  • looped and reverse-looped playback of sequenced data
  • random access to sequence data
  • non-linear sequences
  • relative representation of time (non-absolute)
  • non-track based

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 7 / 18

slide-26
SLIDE 26

The player module

Schedule events

  • continuously in Sequences
  • statically as in state changes in Presets

Player features

  • looped and reverse-looped playback of sequenced data
  • random access to sequence data
  • non-linear sequences
  • relative representation of time (non-absolute)
  • non-track based

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 7 / 18

slide-27
SLIDE 27

The Event interface

Events

‘events’ scheduled by the player are instances of the Event interface. Its attributes are:

  • address
  • value
  • interpolation

Presets

Presets inherit from the Event class: An event that contains events.

  • No time information
  • All addresses must be unique

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 8 / 18

slide-28
SLIDE 28

The Event interface

Events

‘events’ scheduled by the player are instances of the Event interface. Its attributes are:

  • address
  • value
  • interpolation

Presets

Presets inherit from the Event class: An event that contains events.

  • No time information
  • All addresses must be unique

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 8 / 18

slide-29
SLIDE 29

The Event interface

Events

‘events’ scheduled by the player are instances of the Event interface. Its attributes are:

  • address
  • value
  • interpolation

Presets

Presets inherit from the Event class: An event that contains events.

  • No time information
  • All addresses must be unique

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 8 / 18

slide-30
SLIDE 30

The Event interface

Events

‘events’ scheduled by the player are instances of the Event interface. Its attributes are:

  • address
  • value
  • interpolation

Presets

Presets inherit from the Event class: An event that contains events.

  • No time information
  • All addresses must be unique

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 8 / 18

slide-31
SLIDE 31

The Event interface

Events

‘events’ scheduled by the player are instances of the Event interface. Its attributes are:

  • address
  • value
  • interpolation

Presets

Presets inherit from the Event class: An event that contains events.

  • No time information
  • All addresses must be unique

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 8 / 18

slide-32
SLIDE 32

The Event interface

Events

‘events’ scheduled by the player are instances of the Event interface. Its attributes are:

  • address
  • value
  • interpolation

Presets

Presets inherit from the Event class: An event that contains events.

  • No time information
  • All addresses must be unique

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 8 / 18

slide-33
SLIDE 33

The Event interface

Events

‘events’ scheduled by the player are instances of the Event interface. Its attributes are:

  • address
  • value
  • interpolation

Presets

Presets inherit from the Event class: An event that contains events.

  • No time information
  • All addresses must be unique

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 8 / 18

slide-34
SLIDE 34

Player module example

Player.1 Player.3 Player.2 Harmonizer.1 Delay.1

Asynchronous input

Sequence data read from file /Harmonizer.1/transpose 6 /Player.3/record 1 /Player.2/synchro 11502 /Delay.1/delay-time 100 OFF ON /Player.1/play 1 /Delay.1/delay-time 100

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 9 / 18

slide-35
SLIDE 35

Storage

The IXD file format

  • Already used for module definitions and collections of modules

(patches).

  • Time based data forms an extension to the existing formats.

Sequence

Sequences of events in time.

Preset

A set of events describing a state.

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 10 / 18

slide-36
SLIDE 36

Storage

The IXD file format

  • Already used for module definitions and collections of modules

(patches).

  • Time based data forms an extension to the existing formats.

Sequence

Sequences of events in time.

Preset

A set of events describing a state.

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 10 / 18

slide-37
SLIDE 37

Storage

The IXD file format

  • Already used for module definitions and collections of modules

(patches).

  • Time based data forms an extension to the existing formats.

Sequence

Sequences of events in time.

Preset

A set of events describing a state.

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 10 / 18

slide-38
SLIDE 38

Storage

The IXD file format

  • Already used for module definitions and collections of modules

(patches).

  • Time based data forms an extension to the existing formats.

Sequence

Sequences of events in time.

Preset

A set of events describing a state.

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 10 / 18

slide-39
SLIDE 39

Storage

The IXD file format

  • Already used for module definitions and collections of modules

(patches).

  • Time based data forms an extension to the existing formats.

Sequence

Sequences of events in time.

Preset

A set of events describing a state.

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 10 / 18

slide-40
SLIDE 40

Storing Sequences

Re-usability

A Sequence can link in other Sequences, or parts of other Sequences:

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 11 / 18

slide-41
SLIDE 41

Storing Sequences

Re-usability

A Sequence can link in other Sequences, or parts of other Sequences:

<state> <value title="mysequence" xlinktype="simple" href="mysequence.ixd" show="embed" selector="2" /> </state>

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 11 / 18

slide-42
SLIDE 42

Storing Sequences

Re-usability

A Sequence can link in other Sequences, or parts of other Sequences:

<state> <value title="mysequence" xlinktype="simple" href="mysequence.ixd" show="embed" selector="2" /> </state>

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 11 / 18

slide-43
SLIDE 43

Storing Sequences

Sequences

List of timed events. Sequences may trigger other Sequences.

<sequence id=0> <event tick="0" id="1" marker="Foo Bar"> <address class="delay" attribute="time"/> <value>400</value> </event> <!-- we deleted event id="2" at some point --> <event tick="0" id="3" marker=""> <!-- this goes to 'another' player --> <address class="player" attribute="play"/> <value>1</value> </event> <event tick="100" id="4" marker="Baz Bam"> <address class="delay" attribute="time"/> <value>800</value> </event> </sequence>

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 12 / 18

slide-44
SLIDE 44

Storing Sequences

Sequences

List of timed events. Sequences may trigger other Sequences.

<sequence id=0> <event tick="0" id="1" marker="Foo Bar"> <address class="delay" attribute="time"/> <value>400</value> </event> <!-- we deleted event id="2" at some point --> <event tick="0" id="3" marker=""> <!-- this goes to 'another' player --> <address class="player" attribute="play"/> <value>1</value> </event> <event tick="100" id="4" marker="Baz Bam"> <address class="delay" attribute="time"/> <value>800</value> </event> </sequence>

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 12 / 18

slide-45
SLIDE 45

Storing Presets

Preset

Presets define events: address/value pairs with no time information

<preset class="delay" name="delay preset 1"> <event> <address attribute="frequency"/> <value>800</value> </event> <event> <address attribute="phase"/> <value>0.5</value> </event> </preset>

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 13 / 18

slide-46
SLIDE 46

Storing Presets

Preset

Presets define events: address/value pairs with no time information

<event tick="100" type="preset" href="mypreset.ixd" id="5" marker="Load delay preset 1" selector="2"> <preset class="delay" name="delay preset 1"> <event> <address attribute="frequency"/> <value>800</value> </event> <event> <address attribute="phase"/> <value>0.5</value> </event> </preset>

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 13 / 18

slide-47
SLIDE 47

Storing Presets

Preset

Presets define events: address/value pairs with no time information

<event tick="100" type="preset" href="mypreset.ixd" id="5" marker="Load delay preset 1" selector="2"> <preset class="delay" name="delay preset 1"> <event> <address attribute="frequency"/> <value>800</value> </event> <event> <address attribute="phase"/> <value>0.5</value> </event> </preset>

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 13 / 18

slide-48
SLIDE 48

Inheritance

Overriding properties

Existing data may be extended, dynamically or statically.

<sequence> <name>sawtooth_mod</name> <description>Simple linear ramping to modulate the frequency of a sawtooth

  • scillator</description>

<tag>ramp</tag> <event tick="0" id="1" marker="Section 1"> <address class="SawTooth" attribute="frequency"/> <interpolation>1</interpolation> <value>550</value> </event> <event tick="100" id="2" marker=""> <address class="SawTooth" attribute="frequency"/> <interpolation>1</interpolation> <value>800</value> </event> <event tick="100" id="3" marker="Section 2"> <address class="SawTooth" attribute="frequency"/> <interpolation>0</interpolation> <value>100</value> </event> </sequence>

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 14 / 18

slide-49
SLIDE 49

Inheritance

Overriding properties

Existing data may be extended, dynamically or statically.

<sequence> <name>sawtooth_mod</name> <description>Simple linear ramping to modulate the frequency of a sawtooth

  • scillator</description>

<tag>ramp</tag> <event tick="0" id="1" marker="Section 1"> <address class="SawTooth" attribute="frequency"/> <interpolation>1</interpolation> <value>550</value> </event> <event tick="100" id="2" marker=""> <address class="SawTooth" attribute="frequency"/> <interpolation>1</interpolation> <value>800</value> </event> <event tick="100" id="3" marker="Section 2"> <address class="SawTooth" attribute="frequency"/> <interpolation>0</interpolation> <value>100</value> </event> </sequence> <parent title="sawtooth_mod" xlinktype="simple" href="sawtooth_mod.ixd" role="InstanceData" show="embed" /> <event tick="*" id="*" marker="*"> <address attribute="frequency"/> <value operation="+">10</value> <value operation="*">tick</value> <value operation="*">0.01</value> </event>

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 14 / 18

slide-50
SLIDE 50

Meta-Data

Tags

  • Sequences and Presets may be tagged with semantic information.
  • Relations between entities may be created.

Documentation

  • Documentation resources may be linked in with Sequence and

Preset files.

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 15 / 18

slide-51
SLIDE 51

Meta-Data

Tags

  • Sequences and Presets may be tagged with semantic information.
  • Relations between entities may be created.

Documentation

  • Documentation resources may be linked in with Sequence and

Preset files.

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 15 / 18

slide-52
SLIDE 52

Meta-Data

Tags

  • Sequences and Presets may be tagged with semantic information.
  • Relations between entities may be created.

Documentation

  • Documentation resources may be linked in with Sequence and

Preset files.

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 15 / 18

slide-53
SLIDE 53

Meta-Data

Tags

  • Sequences and Presets may be tagged with semantic information.
  • Relations between entities may be created.

Documentation

  • Documentation resources may be linked in with Sequence and

Preset files.

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 15 / 18

slide-54
SLIDE 54

Meta-Data

Tags

  • Sequences and Presets may be tagged with semantic information.
  • Relations between entities may be created.

Documentation

  • Documentation resources may be linked in with Sequence and

Preset files.

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 15 / 18

slide-55
SLIDE 55

Implementation

Integra Environment (beta)

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 16 / 18

slide-56
SLIDE 56

Implementation

Integra Environment (beta)

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 16 / 18

slide-57
SLIDE 57

Implementation

Integra Environment (beta)

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 16 / 18

slide-58
SLIDE 58

Summary

A proposed format for storing and sharing time based data.

  • XML-based drawing on MIDI, RDF and SMIL with the ability to

include SDIF and GDIF .

  • Extending the IXD Schemas for Module definitions, Collections

and Integra documentation.

  • Employ semantic richness and sustainability.

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 17 / 18

slide-59
SLIDE 59

Summary

A proposed format for storing and sharing time based data.

  • XML-based drawing on MIDI, RDF and SMIL with the ability to

include SDIF and GDIF .

  • Extending the IXD Schemas for Module definitions, Collections

and Integra documentation.

  • Employ semantic richness and sustainability.

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 17 / 18

slide-60
SLIDE 60

Summary

A proposed format for storing and sharing time based data.

  • XML-based drawing on MIDI, RDF and SMIL with the ability to

include SDIF and GDIF .

  • Extending the IXD Schemas for Module definitions, Collections

and Integra documentation.

  • Employ semantic richness and sustainability.

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 17 / 18

slide-61
SLIDE 61

Thank you!

Funding

The Integra project is funded by the European Commission and is a collaboration between Universities, research centers and New Music Ensembles in Europe.

Questions?

. . .

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 18 / 18

slide-62
SLIDE 62

Thank you!

Funding

The Integra project is funded by the European Commission and is a collaboration between Universities, research centers and New Music Ensembles in Europe.

Questions?

. . .

Bullock & Frisk (Birmingham, Lund) Temporal data representation ICMC 2009 18 / 18