application developed by Guerino Mazzola . RUBATO is a universal - - PowerPoint PPT Presentation

 Origins in PRESTO, and early computer

application developed by Guerino Mazzola.

 RUBATO is a universal music software

environment developed since 1992 under the direction of Guerino Mazzola.

 RUBATO COMPOSER system developed in

Gérard Milmeister’s doctoral dissertation (2006) where he implemented the Functorial Concept Architecture, based on the data format of Forms and Denotators. http://www.rubato.org/

 This software works with components called

rubettes ( perform basic tasks for musical representation and whose interface with other rubettes is based on the universal data format

• f denotators).

 The data format of denotators uses set-valued

presheaves over the category of modules and diaffine morphisms http://www.rubato.org/

 In addition to what RUBATO COMPOSER

is designed to be for the composer and music theorist, it is also an excellent tool for learning sophisticated mathematical concepts.

 The mathematics involved are sophisticated,

and could be accessible in a formal way to the average mathematics student in their senior year, after having some experience with courses such as Linear Algebra, Modern Algebra, Analysis or Topology, but would usually be taught at the graduate level.

 Possibilities of knowledge expansion and new

applications

 Danger of superficiality, contamination and

surrender to fashion.

 Possibilities that Mathematical Music Theory,

and its applications, have to offer to the knowledge base of mathematics, music, and computer science students, without excluding those from other fields.

 It has been generally acknowledged that there is a

gap between the formality of modern mathematics as conceived and taught by trained mathematicians, and the mathematics that are seen by non mathematicians as relevant.

 When the mathematics are embedded in different

practical contexts, it is often easier to get students to think mathematically in a natural manner

 Even mathematics students themselves often have

difficulty in making meaning out of the formal presentation of their subject (MacLane, 2005).

 The creation of interdisciplinary

curriculum materials and courses, using RUBATO COMPOSER as a common ground, opens a realm of possibilities for Mathematical Music Theory, and for the development of research and researchers in the field. It also can be justified, in and of itself, where RUBATO COMPOSER is conceived of as a learning tool.

 The International Journal of Computers for

Mathematical Learning “… publishes contributions that explore the unique potential

• f new technologies for deepening our

understanding of the field of mathematics learning and teaching.”

 A revision of articles from 2006-8, illustrates

that the notion of using specific software to enhance the learning of mathematics has a respectable recent history, and has been analyzed using well documented research paradigms.

 Using formalism to construct

meaning is a very difficult method for students to learn, but this is the

• nly route to learning large

portions of mathematics

 The writing of computer programs

to express mathematical concepts can be an effective way of achieving this goal of advanced mathematical learning.(Dubinsky, 2000)

 The RUBATO COMPOSER software opens up

the possibility of creating meaning behind the formalisms of advanced mathematical areas, and accelerating processes of learning and understanding.

 These mathematical areas (Abstract Algebra

beyond Group Theory, Category and Topos theory) are not really addressed in the literature on computer-based learning, or on collegiate mathematics education in general.

 Computer-based learning in music is usually

related to training in aural skills, sight reading, and other subjects essential to the music student.

 Musical representation languages such as

Common Music, OpenMusic, and Humdrum, for composing and analyzing music, that do require programming skills.

 However, RUBATO COMPOSER offers the

• pportunity of introducing the music student to

the higher mathematics involved in modern Mathematical Music Theory.

 This can be done in a relatively (not completely)

“painless” manner, as compared to what it would require to learn this material in the traditional way.

 RUBATO COMPOSER is based on the data

format of Forms and Denotators.

 Forms are mathematical spaces with a precise

structure, and Denotators are objects in the Form spaces.

 Category theory is the mathematical

foundation on which this particular conceptual basis of Mathematical Music Theory is built.

 In the RUBATO COMPOSER architecture,

modules are a basic element, much like primitive types in programming languages.

 The recursive structure of a Form, if not circular,

will eventually “stop” at a Simple form which, for all practical purposes, is a module.

 Morphisms between modules (changes of

address), are built into the software.

 In the development of the data base management

systems, the objects must be named and defined in a recursive way and they must admit types that, in this context, are such as limit, colimit, and power.

 It is necessary to work with the algebraic structure

• f modules, yet form constructions whose

prototypes are found in the category of sets.

 This is the reason why, in the context of RUBATO

COMPOSER, the approach is to work in the functor category of presheaves over modules (whose objects are the functors ).

 Through the creation of denotators, and the

recursive structure of types when working with forms, the mathematics student accustomed to the formalism of abstract mathematics has the opportunity to participate in a concrete implementation of these concepts.

 The mathematics student who still struggles to

find meaning in the abstract formalism, may find a vehicle through which this process can be accelerated.

 The majority of rubettes available at this time are

• f low level nature.

 One of the objectives of the developers of

RUBATO COMPOSER is to create more high level rubettes that present ‘friendlier’ interfaces and language for the non-mathematical user, in particular the composer or musicologist.

 However, musicians interested in using

technology in an innovative manner, cannot isolate themselves from the mathematics used to create their tools.

 The musical analysis itself, and much of the

musical ontology, is intricately related to the mathematical framework.

 The music student does not have to deal with

the mathematical objects in the same way as the mathematics student (nor does the computer science student). However, if he wants to follow the developments of research in Mathematical Music Theory, he needs an understanding of the language and concepts behind the tools.

 This is especially true in the case of RUBATO

COMPOSER, which has been designed as the result of a precisely defined, and perhaps revolutionary, approach to musical analysis.

 Even with the high level rubettes that are, and

will be, available, it is possible to retrace the steps and uncover the mathematics behind their construction.

 When the terminology changes from

‘transposition’ to ‘translation’ and, in general, from the musical ‘inversion’, ‘retrograde’, ‘augmentation’, to the language of mathematical transformations, or morphisms, the music student is presented with an

• pportunity to develop an understanding of

the meaning behind the formalism.

 In RUBATO COMPOSER not only translations,

but general affine morphisms as well, can be used to generate musical ornamentation.

 The Wallpaper rubette, developed by Florian

Thalmann, also opens the possibility of generating morphisms in any n-dimensional space (for example, using the five simple forms

• f the Note denotator - Onset, Pitch, Duration,

Loudness and Voice- an affine transformation in 5D can be defined).

 When working with affine transformations in 2D

space, the command can be given by just ‘dragging’, instead of defining the morphisms.

 A unit introducing the basic concepts of linear

algebra, group theory, and geometry needed to understand mathematical music theory, as it has been developed over the last 40 years, can be created.

 Most ‘extreme’ example, up until now, the

BigBang rubette, developed by Florian Thalmann, in the context of Mathematical Gesture Theory and Computer Semiotics.

 Based on a general framework for geometric

composition techniques.

 Given a set of notes, their image is calculated

through affine invertible maps in n- dimensional space.

 There is a theorem that states that the affine

invertible map in n dimensions can be written as a composition of transformations, each one acting on only one or two of the n dimensions.

 The component functions (act on only one or

two of the n dimensions) represented geometrically as five standard mathematical

• perations that have their musical

representation:

 Translation (transposition in music)  Reflection (inversion, retrograde in music)  Rotation (retrograde inversion in music)  Dilations (augmentations in music)  Shearings (arpeggios).

 Sample of a Unit and its Focus on

different Majors

 A sample unit has been created to show how the

analysis and creation of a musical object can give students from different disciplines, in particular mathematics, computer science and music, a deeper understanding of abstract mathematics while satisfying aesthetic interests as well.

 Description of the Module: The development of a

melodic phrase, recursively transformed by transformations in the plane as ornamentation, using the Wallpaper rubette in RUBATO COMPOSER

 Objectives and Activities:

All students will be able to:

 Identify rigid transformations in the plane and

give them musical meaning. For example:

 mathematical translations – musical transpositions;  mathematical reflection – musical inversion, retrograde;  mathematical rotation – musical inversion-retrograde;  mathematical dilatation – musical augmentation in time;  mathematical shearing – musical arpeggios in time.

 All students will be able to:  Use the software RUBATO COMPOSER and, in

particular, the Wallpaper rubette, to generate musical ornamentation by means of diagrams of morphisms (functions).

 Create and interpret transformations, and

compositions of transformations, like the following, in which is a rotation of 180 followed by a translation, and is a translation.

1

f

2

f

 Select any of the coordinates of a Note denotator

(which is 5-dimensional) and combine them.

 When two coordinates are chosen, say Onset and

Pitch, students will relate them to the rigid transformations in the Euclidean plane.

 Mathematics students (and those from computer

science and other related areas) will formally construct the morphisms, while Music students can use the succession of primitive transformations by dragging with the mouse

Mathematics students will be able to:



Construct the composition of module morphisms from the Form Note to the Form Note. For example, using the coordinates Onset and Pitch, they can construct the following composition of embeddings, projections and affine transformations.



The creation of a melodic phrase where and are the injections and is the embedding . The transformation (a musical embellishment, as seen in the previous slide) is then applied, and to return the coordinates to the module morphisms and the projections and are applied where c represents quantized to

1

i

2

i

2



2

e



n

• n

p

1

p

2

p

 = ○ f ○ (( ○ o) + ( ○ ○ p)): A →  = c ○ ○ f ○ ((i1 ○ o) + ( ○ ○ p)): A →

n

• 1

p

1

i

2

i

2

e

2

p

2

i

2

e

n

p

General Topic Mathematics Highlights Music Highlights Transformational Theory Group Theory, Set Theory, Function Theory, Geometry, Topology Analysis of Musical Works, from all genres. In the case of Classical music, analysis of modern atonal music that is not approachable with traditional tools from music theory. Subdivision of the Octave; Maximally Even Sets Group Theory, Number Theory, Differential Equations, Continuous Fractions Exploration of different and exotic scales; Composition using unusual scales. Forms and Denotators; The Software RUBATO COMPOSER Category and Functor Theory, Topos Theory, Sets and Modules, Linear, Affine and Diaffine Transformations. Linear Algebra, Geometry, Mathematical Gesture Theory Music Composition; Embellishment

• f

existing music; Algorithms for Composition; Counterpoint Theory



Mazzola, G, Milmeister, G, Morsy, K., Thalmann, F. Functors for Music: The Rubato Composer System. In Adams, R., Gibson, S., Müller Arisona,

• S. (eds.), Transdisciplinary Digital Art. Sound, Vision and the New Screen,

Springer (2008).



Milmeister, Gérard. The Rubato Composer Music Software Component- Based Implmentation of a Functorial Concept Architecture. Springer-Verlag (2009).



Thalmann, Florian and Mazzola, Guerino. The BigBang Rubette: Gestural Music Composition with Rubato Composer ICMC 2008 http://classes.berklee.edu/mbierylo/ICMC08/defevent/papers/cr1316.p df



Thalmann, Florian. Musical Composition with Grid Diagrams of Transformations, Masters Thesis, Bern (2007)



International Journal of Computers for Mathematical Learning,

http://www.springer.com/education/mathematics+educatio n/journal/10758



Dubinsky, E.: Meaning and Formalism in Mathematics, Int J Comput Math Learning, 5, 211-240 (2000).



MacLane, Saunders. Despite Physicists, Proof is Essential in

• Mathematics. Synthese 111, 2, 147-154 (May, 1997).

Creating and Implementing a Form Space and Denotator for Bass Using the Category-Theoretic Concept Framework

The Dilemma

• The dilemma resides in how to

maintain the algebraic structure

• f the category of modules(over

any ring, with diaffine morphisms) and, at the same time, construct such objects as limits, colimits, and power, and classify truth.

The Dilemma

• The functorial approach leads to the

resolution of this dilemma by working in the category of presheaves over modules (whose

• bjects are the functors F: Mod → Sets, and

whose morphisms are natural transformations

• f functors) and which will be denoted as

Mod@.

• This category is a topos, which means that it

allows all limits, colimits, and subobject classifiers Ω, while retaining the algebraic structure that is needed from the category Mod.

My Research

• My research consists of creating a form

broad enough for the majority of simple electric bass scores, and a denotator which represents the jazz song “All of Me”. The recursivity of the mathematical definitions of form and denotator are made evident in this application

Bass Score Form

Name Form Bass Score

GeneralNotes

SimpleNote

Denotator “SimpleNote”

• For an example of how to

build a denotator I will take a small part of my denotator named “SimpleNote”

Creating the Denotator of “All of Me”

• A single denotator N1 of the form:

SimpleNote is created from the coordinates

• f the denotator which themselves are forms
• f type simple: Onset, Pitch, Duration,

Loudness, and Voice.

Creating the Denotator of “All of Me”

• As we don't have time to see

how all of the module morphisms are constructed we will build the pitch module morphism “mp”.

• All the others are built

analogously.

Bassline for “All of Me”

Creating the Denotator in Rubato Composer

• To create the denotator for the Jazz song “All
• f Me” we must first create the Module

Morphisms

Creating the Denotator in Rubato Composer

• Once we've opened our module

morphism builder in Rubato Composer we will start creating a module morphism for pitch.

• First we must create “mp2” and “mp1”

and then they will be used to make “mp”, which is a composition of the two.

Creating the Denotator in Rubato Composer

• For mp2 the domain is determined from

the number of musical notes in the bass line.

• For instance, the bass line I wrote for “All
• f Me” contains 64 notes.
• We establish the first note as the anchor

note, so for our domain we use Z63.

Creating the Denotator in Rubato Composer

• mp2 is an embedding of the canonical vectors plus the

zero vector, that goes from Z63 → Q63.

mp1

• mp1 will take us from Q63→ Q.
• We will set up the module

morphism in the same way as mp2, except we will select affine instead of canonical.

mp1

mp

• To create mp, we bring up the

module morphism builder, and create a module morphism with the domain

• f mp2 and a codomain of

mp1.

• Which results in mp1○mp2 =

mp.

mp

mp

• mp: Z63 → Q63 → Q: x → (4, 7, 9, 7, 4,

0, 2, 4, 8, 11, 8, 4, 2, -1, -4, -3, -1, 1, 4, 7, 6, 5, 4, 2, 5, 9, 12, 11, 9, 5, 2, 4, -8, 3, 4, 8, 4, 0, -1, -3, -1, 0, 4, 9, 7, 5, 4, 2, 6, 12, 11, 9, 6, 0, 2, 5, 9, 2, 6, 7, -5, -1, 2)●x + 48

• Example, when x=(0,...,0)

(4,..., 2)●(0,...0) + 48 → 0+48 → 48 Which is the first note in our bassline.

Module Morphisms

• All of the Module Morphisms

are made in the same way.

• Once all of the Module

Morphisms are built we can arrange them using the denotator builder.

Denotator Builder

Creating Denotator

Using Rubettes in Rubato Composer

• To play our bass line in Rubato

Composer, we must create a network using rubettes.

• We will need to set up three rubettes;

the Source rubette, the @AddressEval rubette, and the Scoreplay rubette.

Source

Source

Source

@AddressEval Rubette

• Next open the @AddressEval

Rubette.

• This rubette will be directly

connected to the Source rubette.

Scoreplay Rubette

• To play our bassline we need

to open the scoreplay rubette and connect it to the @AddressEval rubette.

• This is done the same way as

the Source and @AddressEval rubettes.

Finished Network

Pianola

Rubato Composer and its Functorial Approach:

From Morphisms to Gestures through Rubettes

Jonathan Cantrell Junior Mathematics Georgia State University

Perspectives on Music

Where did we start?

 Melodies and harmonies applied across time  Often written in sequential fashion

Where are we now?

 Asynchronous editing tools allows a composer to work in non-linear fashion

Digital Audio Workstations e.g Pro Tools MIDI Sequencers e.g Logic, Cubase

 Still music is written in distinct phrases and compiled together

Architecture of Rubato Composer

Defined recursively Utilizes the Form and Denotator concept Allows for heterogeneous types

 Simple, Limit, Colimit, or Power

Implemented as free modules over rings Working from the category of presheaves over modules

Diaffine Transformations

Working in the category Mod of modules over any ring whose morphisms are diaffine transformations which gives us the ability to perform operations from one module to another Diaffine transformations are module morphisms plus a translation

 A dilinear homomorphism from an R-module M to an S-module N plus a translation in N

Why Topoi?

Rigidly defined categories which are a generalization of the category of Sets Allows the composer to perform set-valued

• perations where the elements in the sets are

module morphisms over any ring Sets are required within the framework of the Form and Denotator concept as the evaluation

• f the colimit of Score:Note

Geometric Representation

Module morphisms in n-dimensional space embedded into a Form of type Simple Any diaffine transformation h in n-dimensional space can be written as a composition of transformations hi which involve only one or two dimensions of the n dimensions and leave the

• thers unchanged

In our particular example, we are in ℝ5 we want to operate exclusively on pitch and onset, therefore we apply this construct to work in ℝ2

Why Mod?

Why does the algebraic structure need to be retained? It allows us to map individual Simple Forms to the plane and perform affine transformations in a different space, as exemplified in the Wallpaper rubette We apply the mathematical tools of translation as musical transpositions, and reflections as retrograde

My Research

I have developed an example of a 12-note melodic phrase recursively transformed using the Wallpaper rubette. I further generalize this series of transformations using the high-level tool of the BigBang rubette available in Rubato Composer

Compound Transformations

An example of two translations applied recursively. For a specific Note denotator, we operate exclusively on the module morphisms Onset and Pitch

Compositions

As stated, the module morphisms contained in the Simple forms Onset, o: A → ℝ and Pitch, p: A → ℚ are extracted from the Note denotator We now need to compose these module morphisms as follows

Compostitions

This is described by the following compositions:

 i₁ ○ o : A → ℝ2,  i₂ ○ e₂ ○ p : A → ℝ2,

Where i₁ and i₂ are the injections ℝ → ℝ2, and e₂ is the embedding ℚ → ℝ.

In order to combine these two morphisms into a single instance of ℝ2 we must sum them such that onset and pitch become respective axes in ℝ2

Compositions

The transformation f is then applied, and finally, to return the coordinates to the module morphisms on and pn, we apply the projections p1 and p2 as follows where c represents ℝ quantized to ℚ

 on = p1 ○ f ○ ((i1 ○ o) + (i2 ○ e2 ○ p)): A → ℝ  pn = c ○ p2 ○ f ○ ((i1 ○ o) + (i2 ○ e2 ○ p)): A → ℚ

Tracing the modules on which these compositions take place we have

 onset: ℤ11 → ℝ11 → ℝ → ℝ2 → ℝ2 → ℝ  pitch: ℤ11 → ℚ11 → ℚ → ℝ → ℝ2 → ℝ2 → ℝ → ℚ

Morphisms to Gestures

The Wallpaper rubette is an example of a low- level process

 We are working in a very mathematical context

For the composer, this will not always be appropriate, as mathematics may be a means rather than an end For this reason Gesture Theory is being developed by Dr. Guerino Mazzola and Florian Thalmann as implemented in the BigBang rubette

 Gestures as curves in topological space

Future Applications

The Rubato Framework gives the composer an alternate view of composition, working from a functorial perspective Also the musician can gain insight into a branch

• f mathematics using intuition as a guide
• pening up exciting educational avenues

The highly characterizable nature of the category theoretic framework opens up the

• pportunity for any system to modeled

effectively

Bibliography

– Milmeister, Gérard. The Rubato Composer Music Software: Component-Based Implementation of a Functorial Concpet Architecture Zürich: 2006 – Thalmann, Florian and Mazzola, Guerino. The BigBang Rubette: Gestural Music Composition with Rubato Composer – Thalmann, Florian. Musical Composition with Grid Diagrams of Transformations Bern: 2007 – “Pro Tools.” http://www.digidesign.com/ – “Logic.” http://www.apple.com/logicstudio/ – “Cubase.” http://www.steinberg.net