What is Music? The Super-Stimulus Theory Philip Dorrell philip - - PowerPoint PPT Presentation

What is Music?

The Super-Stimulus Theory

Philip Dorrell

philip (funny email character) 1729 (little dot) com

14 Chetwode Grove, Newlands, Wellington, New Zealand.

What is Music? – p. 1/84

Introduction

What is Music? – p. 2/84

Who is Philip Dorrell?

Mostly work as a software developer. Not a professional scientist, mathematician or anything else relevant to music science. Enthusiastic amateur. Interests include: Mathematics Science Reckless and imprudent application of previous two items to other fields of endeavour: Human society, morality, politics The human mind: consciousness, dreams Music

http://www.1729.com/

What is Music? – p. 3/84

My Book

“What is Music? Solving a Scientific Mystery” Self-published Web site: http://whatismusic.info/ Currently available via POD at lulu.com for US$30 + postage

What is Music? – p. 4/84

What is Music? (Part 1)

Music is a subjective phenomenon. We experience: Pleasure Emotion We observe that others respond to music. We can describe features of music that we observe: Melody: scales Harmony: chords, bass, home chords Rhythm: bars, metre, note lengths But, the description is incomplete.

What is Music? – p. 5/84

What is Music? (Part 2)

A complete theory should ... explain what happens inside our brains when we listen to music, relate this to evolution by natural selection: What is the purpose of music? give us a complete description of music, correspond to our subjective experience of music.

What is Music? – p. 6/84

The Luxury Yacht Test

• 1. Discover complete theory of music.
• 2. From theory, determine a generative algorithm for

music.

• 3. Use algorithm to generate new original strong music.
• 4. Profit!
• 5. Purchase luxury yacht.

What is Music? – p. 7/84

Existing Music Science: Assumptions

Evolutionary Assumption: Good, but many implausible hypotheses Music Assumption: Music has a purpose. Communication Hypothesis: Music is communication. Social Assumption: Music has a social purpose. “In the Past” Assumption: Music used to serve a purpose. Music-Language Assumption: Yes, but how? Musical “syntax” theories. Cultural Assumption: It’s all culturally defined.

What is Music? – p. 8/84

Neuroscientific Assumptions

Cortical Plasticity Assumption: The brain adapts to general features of music (not pre-adapted). Temporal Coding: timings of neural firing in phase with sound vibrations

What is Music? – p. 9/84

Musical Aspect Assumptions

Simultaneous Pitch Assumption: Brain mechanisms for detecting relationships between simultaneous pitch

• values. (But why should it care?)

Scale Assumption: Plasticity, Categorisation Hierarchical Segmentation:

bars beats shortest beat period time

What is Music? – p. 10/84

Music & Musicality

What is Music? – p. 11/84

Purpose of Music

Which aspect of music has the purpose? Composing music? Performing music? Listening to (and responding to) music? Are we assuming too much? The Music Assumption: Music itself has a purpose. Alternative: Music is a side-effect (of something else).

What is Music? – p. 12/84

Information Processing

The brain is an information processing system. Music is information. But what does it mean?

What is Music? – p. 13/84

Music = input or output?

Hypothesis: Music is primarily an input. Almost everyone appreciates good music. Creating music good enough to listen to is non-trivial: Performance requires skill and practice. Composition is very difficult.

What is Music? – p. 14/84

input = Music, output = ?

What is Music? – p. 15/84

input = Music, output = Musicality

Hypothesis: The output is “musicality”: how good the music is. But what is the meaning of musicality? Apparently 1-dimensional

What is Music? – p. 16/84

Does music have a purpose?

The goal of composition and performance is to create good music. Music exists because it is musical. Musicality is a perceived property of music. Too circular: no plausible selective pressure.

What is Music? – p. 17/84

Perception of Musicality

Hypothesis: Musicality is a property of something else. What? What other thing is like music? Answer: Speech. Common features of speech and music: “Melody” “Rhythm” Vocal Timbre

What is Music? – p. 18/84

The Musicality of Speech

Hypothesis: Musicality is a perceived aspect of speech. Music is (obviously) much more musical than speech. Music is a super-stimulus.

What is Music? – p. 19/84

What does Musicality Mean?

If musicality is a perceived aspect of speech – What does it mean? For the moment, leave this unanswered. Proceed on assumption that it is some property that matters.

What is Music? – p. 20/84

Compare Matching Aspects

Melody: Speech: (mostly) continuous Music: discontinuous, notes come from scales (but

• ften as close to continuous as is permitted by scale

constraint) Rhythm: Speech: irregular Music: regular and hierarchical Timbre (exact match): The human voice is a preferred musical instrument. Other instruments make voice-like sounds.

What is Music? – p. 21/84

Compare Non-Matching Aspects (part 1)

For speech aspects not in music, assume they are not relevant to the determination of musicality. Phonemes Grammar Vocabulary Semantics These features do occur in song lyrics, but mostly satisfy consistency requirements: Words match rhythm of music. Words match melody (in tone languages). Semantics match emotion of music.

What is Music? – p. 22/84

Compare Non-Matching Aspects (part 2)

Certain aspects of music appear to have no equivalent in speech: Harmony Bass Scales Home notes and home chords Repetition

What is Music? – p. 23/84

Super-stimulus Aspects

Each speech aspect has a cortical map perceiving it. Each cortical map has a super-stimulus. Each such super-stimulus corresponds to an aspect of music. The super-stimulus may have properties not in the normal stimulus. This explanation applies to both matching and (apparently) non-matching aspects.

What is Music? – p. 24/84

Individual Aspects

What is Music? – p. 25/84

Harmony and Bass

What is Music? – p. 26/84

Harmony (part 1)

A “harmonic” cortical map responds to simultaneous pitch values, related by consonant intervals. No simultaneous pitch values in speech. But, harmonic map also responds to sequential pitch values, related by consonant intervals: Sequential and simultaneous playing of notes in a chord both give us the “feel” of the chord. Conclusion: The purpose of the harmonic cortical is to respond to consonant relationships between pitch values

• ccurring at different times within one speech

melody.

What is Music? – p. 27/84

Bass

A cortical map with greater response to low pitch values. The super-stimulus consists of very low pitch values.

What is Music? – p. 28/84

Harmony (part 2)

Observed properties of chords: Chords persist for one or more bars. A chord is a perceived property of that portion of the tune. Chords change on the main beat. Notes of a chord tend to be consonantly related to each

• ther.

Bass note = root note of chord. Chords are perceived modulo octaves.

What is Music? – p. 29/84

Harmony (part 3)

We can deduce the properties of the cortical map that responds to chords: Neurons have persistent response to pitch values that

• ccur.

Neural activity tends to be reset on a strong beat. Mutual reinforcement of activity between neurons representing pitch values separated by consonant intervals. Reinforcement of neural activity by activity in corresponding bass cortical map neurons. Neurons respond to pitch values modulo octaves (i.e. “chroma”).

What is Music? – p. 30/84

Harmonic Map Response

f 5/4f 3/2f 2f 5/2f t0 t1 t2 t3 t4 t5 t6

What is Music? – p. 31/84

Rhythm

What is Music? – p. 32/84

Rhythm (part 1)

Response to regular beats Cortical map with 1 dimension = beat period. Multiple peaks in spectrum, for example for 4/4 time: 1 bar 1/2 bar = 2 notes 1/4 bar = 1 note 1/2 note 1/4 note

What is Music? – p. 33/84

Regular Beat Cortical Map Response

4/4 time:

Position in cortical map

4/4

2000ms 1000ms 500ms 250ms 125ms

Irregular speech rhythm:

Position in cortical map

What is Music? – p. 34/84

Regular Beat Cortical Map: 6/8 time

Position in cortical map

6/8

1800ms 900ms 300ms 150ms

Position in cortical map

6/8

960ms 480ms 160ms 80ms

What is Music? – p. 35/84

Rhythm (part 2)

Note length Like regular beat, but without requirement for repetition Possible 2nd dimension to cortical map:

• ne extreme = note length

second extreme = regular beat in-between points could account for some syncopation effects

What is Music? – p. 36/84

Scales

What is Music? – p. 37/84

The Scale

As a property of a tune Which pitch values have occurred recently? Persistent response Response to pitch value modulo octaves

What is Music? – p. 38/84

Scale Map Response: Speech

Pitch Time Intensity

What is Music? – p. 39/84

Scale Map Response: Music

Pitch Time Intensity

What is Music? – p. 40/84

Scale Map Response: Music (Saturated)

Pitch Time Intensity

What is Music? – p. 41/84

A Pattern Emerges

Scale Map: fixed peaks of activity, only for music Regular Beat Map: fixed peaks of activity, only for music Harmonic Map: fixed peaks of activity (for duration of chord), only for music A common underlying pattern: Constant Activity Patterns Measured within each cortical map Summed across all relevant cortical maps Maybe this is what musicality is.

What is Music? – p. 42/84

Symmetry

What is Music? – p. 43/84

Symmetry

A set of transformations applied to a structure Features of the structure invariant under the transformation Geometrical symmetries, with geometrical transformations: Rotational Reflective Translational Abstract symmetries (i.e. not geometrical)

What is Music? – p. 44/84

Symmetries of Music: An Example

Pitch Translation Invariance Transpose music into a different key. It’s still perceived as being the “same” music. How? Why?

What is Music? – p. 45/84

Symmetries of Music: A List

A full (?) list: Pitch Translation Invariance Time Scaling Invariance Amplitude Scaling Invariance Octave Translation Invariance Time Translation Invariance Pitch Reflection Invariance (?)

What is Music? – p. 46/84

Symmetries of Music: Variations

Different symmetries apply to different aspects Some are more exact and some are less exact Local/Global Different purposes: Functional Implementation (Mixture of both)

What is Music? – p. 47/84

Symmetry: Pitch Translation Invariance

Transposing music into a different key. (Really a frequency scaling, but can be considered translation if we regard pitch as representing log frequency.) Global: applies to all of a tune Close to exact over small ranges. So exact, we almost take it for granted. Absolute pitch is not pitch translation invariant. Absolute pitch perception must be irrelevant to calculation of musicality. Purpose: people speak in different pitch ranges. Calibration: against perception of consonant intervals

What is Music? – p. 48/84

Symmetry: Time Scaling Invariance

Playing music faster or slower. Global: applies to all of a tune. (Although very slow change in tempo is not perceptible.) Much less exact than pitch translation invariance: generally a clear preference for a given tempo. but, we can reliably perceive the equivalence of the same rhythm at different tempos. Purpose: people speak at different speeds. Calibration: against pairs of durations related by ratio of 2 (mainly) and 3 (somewhat).

What is Music? – p. 49/84

Symmetry: Amplitude Scaling Invariance

Playing music louder or softer. Global. Exact, except that we prefer music we like to be played louder. May have a non-trivial implementation: Louder sound at given pitch will activate a larger set

• f neurons.

Purpose: people talk softer and louder, and also closer and farther away.

What is Music? – p. 50/84

Symmetry: Octave Translation Invariance

Transposing music up or down an octave. Scales are octave translation invariant. Symmetry applies to chords, bass notes and notes within chords. Local, in that individual notes or chords can be translated up or down an octave without (to a first approximation) affecting the quality of the harmony. Purpose? Reduces size of “subtraction” table Tolerate octave ambiguity to achieve more accurate perception modulo octaves. Calibration: perceived ratios of 1:2 between harmonics.

What is Music? – p. 51/84

Subtraction Table

1 2 3 1 2 3 3 2 1

• 3
• 2
• 1

Y X

What is Music? – p. 52/84

Symmetry: Time Translation Invariance

Playing music earlier or later. Extremely exact. Seemingly trivial. Not necessarily trivial to implement: requires self-defined “frame of reference” for time coordinates of notes and events in music. Also, achieves perception of similarity of non-freely repeated components of music.

What is Music? – p. 53/84

Pitch Reflection Invariance

Home chords appear to be determined by process of mutual reinforcement between neurons representing different pitch values, in a way which is invariant under pitch reflection. D is point of symmetry of “white notes” scale. C major and A minor are reflections of each other. They are preferred home chords for the “white notes” scale (at least in modern popular music). If perception of consonant intervals is calibrated by perception of simultaneously occurring pairs of harmonics, then this relationship is intrinsically symmetric.

What is Music? – p. 54/84

Home Chords and Harmonic Heptagon

D F A C E G B A C E C E G

What is Music? – p. 55/84

Pitch Translation/Time Scaling Analogy

Both transformations result in translations of activity patterns along cortical maps. Corresponding intensities of neural activity are invariant. There are calibratable relationships invariant under the transformations: Pitch: consonant intervals Beat period: 2×, 3× multiples Both serve functional purpose: Talkers with different pitch ranges speaking the “same” melodies. Talkers at different speeds speaking the “same” rhythms.

What is Music? – p. 56/84

Calibration

What is Music? – p. 57/84

Calibration of Pitch Translation Invariance

4-way relationship: X − Y = Z − W (log frequencies) Define 3-way relationship between pitch values and intervals: X − Y = I = Z − W, I = interval size. Development/growth processes are too inexact. Cannot “know” in advance that two intervals are the “same”. Calibration required

What is Music? – p. 58/84

Schwartz et. al

Schwartz, Howe and Purves: The Statistical Structure of Human Speech Sounds Predicts Musical Universals (Journal of Neuroscience, August 2003). Empirically determine consonance from speech data. Confirms relationship between speech and music. Assumes frequency division: f1/f2 If we leave out division, then instead of recording ratios we record pairs of pitch values: (f1, f2) Peaks on 1-dimensional distribution become “stripes” on 2-dimensional distribution. Each “stripe” represents a perceived interval. Calibrated “subtraction table”

What is Music? – p. 59/84

CAP & Musicality

What is Music? – p. 60/84

What is Musicality?

Aspect of activity in multiple cortical maps Constant activity patterns in cortical maps But the cortical maps have other purposes. Musicality is a secondary perceived aspect. Special “musicality” neurons: Found in different cortical maps Detect “constancy” of activity patterns Detect multiple active/inactive zones Responses of different musicality neurons summed together.

What is Music? – p. 61/84

Constant Activity Patterns (1D)

t=0s t=5s t=10s

Music

Min Max Activation Position in Map

Speech

Min Max Activation Position in Map

What is Music? – p. 62/84

Constant Activity Patterns (2D)

Speech ... Music ...

t = 0s t = 5s t = 10s t = 0s t = 5s t = 10s

What is Music? – p. 63/84

Constant Activity Pattern Detection

... etc. Map 1 Map 2 Map 3 Map 4 CAP detection CAP detection CAP detection CAP detection Other aspects of speech perception Summation Perceived musicality

What is Music? – p. 64/84

CAP-Detecting Neuron

“Edge” detector:

Constant activity pattern detector Inhibitory inputs Excitatory inputs

What is Music? – p. 65/84

Emotional Effect

Musicality causes an emotional effect. What does this effect mean? Are we perceiving the emotion of the speaker? Musicality is observed to reinforce the listener’s emotional reaction to speech (i.e. to the song lyrics). Yet musicality must be a perceived property of speech (and therefore of the speaker).

What is Music? – p. 66/84

Hypothesis

Perception of musicality is the perception of an aspect

• f the mental state of the speaker.

Perceived musicality determines the appropriate reaction of the listener to the speech coming from the speaker.

What is Music? – p. 67/84

More Hypotheses

Mental state of speaker determines activity patterns in multiple cortical maps. Cortical maps in the listener “echo” activity patterns in cortical maps in speaker. The “echoing” might be rather faint, but ... Combine measured values of musicality across multiple cortical maps to get a more reliable measurement.

What is Music? – p. 68/84

And Another Hypothesis

Musicality is 1-dimensional. It reflects 1-dimensional “conscious arousal” (of speaker). “Conscious arousal” causes release of non-specific modulatory neurotransmitters which affect neural activity across multiple cortical maps. The listener’s brain knows that if the speaker is consciously aroused, then the listener should take the speech content more seriously. (This “knowledge” is not conscious knowledge.) Thus the reinforcement of the listener’s emotional reaction.

What is Music? – p. 69/84

Flow of Information

Information in speaker’s brain Speech content Conscious arousal in speaker’s brain Constant activity patterns in speaker’s brain Generated speech Perceived speech Echoed constant activity patterns in listener’s brain Listener’s perception

• f speaker’s

conscious arousal Listener’s emotional response to speech content Perceived speech content

Speaker’s Brain Listener’s Brain

(Speech sound travels through the air)

What is Music? – p. 70/84

Constraints & Rules

What is Music? – p. 71/84

Constraints (part 1)

Music satisfies multiple constraints: Known features and “rules” Unknown features: “rules” are incomplete, not sufficient to compose “really good” music. The constraint that activity patterns be constant (or as constant as possible) applies across multiple cortical maps. This corresponds to the multiple constraints defined by the “rules” of music.

What is Music? – p. 72/84

Constraints (part 2)

The constraint of constant activity patterns still allows many different patterns in each relevant cortical map. So there is still a very large set of possible musical items.

What is Music? – p. 73/84

Constraints (part 3)

Activity patterns of different cortical maps are not all independent – overlap in the information they represent. Creates conflicts between CAP constraints on different cortical maps. Different genres give preference to different cortical maps (or different parts of cortical maps). “Rule” breaking: optimise global sum of musicality values, without necessarily optimising individual musicality values Accidentals (break “rule” that notes are from diatonic scale) Irregular bar lengths (break “rule” about regular beat)

What is Music? – p. 74/84

Other Aspects

What is Music? – p. 75/84

Home Chords (1)

Similar in some ways to chords, but: A property of the whole tune (for simple tunes) Stronger mutual reinforcement of consonantly related pitch values and inhibition of dissonantly related pitch values: only 3-note major or minor chords are possible. Almost a property of the scale (independently of the melody) “Frame of reference”

What is Music? – p. 76/84

Home Chords (2)

D F A C E G B A C E C E G

Pitch reflection symmetry (?) Caused by symmetry of mutual interactions.

What is Music? – p. 77/84

Repetition

Another feature of music Free repetition Non-free repetition (In-between?) Except for reduplication, no exact repetition in normal speech Exact musical repetition could be a super-stimulus for less exact repetition in speech...

What is Music? – p. 78/84

Musical Taste

Super-stimulus may vary more than perception of normal stimulus. Critical period: adolescence? Critical period for something: a form of imprinting? Dependence on language? Dependence on exposure to types of music? Musicality neurons: wired up when found to be sometimes activated and sometimes not activated.

What is Music? – p. 79/84

Determination of Emotion

If musicality is 1-dimensional, what determines different emotions? Other factors determine direction of emotion: happy, sad, angry etc. Major & minor chords? Theory of 3-consonance? Other factors: tempo, loudness, timbre. Musicality determines intensity of emotion.

What is Music? – p. 80/84

Dance

Music is a super-stimulus for the perception of constant activity patterns in cortical maps that perceive speech. Speech perception includes visual aspects: The speaker’s facial expressions The speaker’s gestures Other body language So there could be corresponding visual aspects of music which are super-stimuli for these visual aspects

• f speech perception.

Most likely candidate: Dance.

What is Music? – p. 81/84

Summary

What is Music? – p. 82/84

Summary

Music is a super-stimulus for the perception of musicality. Musicality is a perceived aspect of speech. Musicality is perceived in the constancy of activity patterns across multiple speech-related cortical maps. Constant activity patterns in the listener’s brain are “echoes” of constant activity patterns in the speaker’s brain. Constant activity patterns in the speaker’s brain are caused by conscious arousal. Perception of the speaker’s conscious arousal tells the listener’s brain: “Take this speech seriously!”.

What is Music? – p. 83/84

The End

What is Music? – p. 84/84