[PPT] - E X P L A I N I N G A N D P R E D I C T I N G T H E P E R C E P T PowerPoint Presentation

SLIDE 1

E X P L A I N I N G A N D P R E D I C T I N G T H E P E R C E P T I O N O F M U S I C A L S T R U C T U R E

J O R D A N B . L . S M I T H M A T H E M U S I C A L C O N V E R S A T I O N S S T U D Y D A Y, 1 2 F E B R U A RY 2 0 1 5 R A F F L E S I N S T I T U T I O N

SLIDE 2

O U T L I N E

What is musical structure?
How do people perceive structure?
Gestalt-based theories
Implication-Realization theory
Listener considerations
Conclusion

SLIDE 3

“Shake It Off” by Taylor Swift

https://www.youtube.com/watch?v=nfWlot6h_JM

SLIDE 4

W H AT I S S T R U C T U R E ?

How did you hear this piece of music?
How did you perceive this piece of music?

SLIDE 5

W H AT I S S T R U C T U R E ?

How did you perceive this piece of music?

drums voice horns

SLIDE 6

W H AT I S S T R U C T U R E ?

How did you perceive this piece of music?

drums voice horns pre- chorus

verse chorus

SLIDE 7

W H AT I S S T R U C T U R E ?

How did you perceive this piece of music?

drums voice horns

verse

pre- chorus

chorus measures/ phrases

beats

sections/ structure

SLIDE 8

H O W D O P E O P L E P E R C E I V E S T R U C T U R E ?

Gestalt-based theories
Implication-Realization theory
Listener considerations

SLIDE 9

G E S TA LT- B A S E D T H E O R I E S

Proximity Similarity

SLIDE 10

G E N E R AT I V E T H E O RY O F T O N A L M U S I C

Principle of Proximity

Lerdahl and Jackendoof 1983 image credit: http://noike.info/~kenzi/roughly/paper/GTTM/12/12_Psychological_and_Linguistic_Connections.html

SLIDE 11

G E N E R AT I V E T H E O RY O F T O N A L M U S I C

Principle of Similarity

Lerdahl and Jackendoof 1983 image credit: http://noike.info/~kenzi/roughly/paper/GTTM/12/12_Psychological_and_Linguistic_Connections.html

SLIDE 12

G E N E R AT I V E T H E O RY O F T O N A L M U S I C

Cooperation Conflict Two rules:

SLIDE 13

G E N E R AT I V E T H E O RY O F T O N A L M U S I C

Conflict of Rules

Lerdahl and Jackendoof 1983 image credit: http://noike.info/~kenzi/roughly/paper/GTTM/12/12_Psychological_and_Linguistic_Connections.html

SLIDE 14

G E N E R AT I V E T H E O RY O F T O N A L M U S I C

Goal of GTTM: to devise a set of rules from which a complete hierarchical grouping structure can be inferred

Lerdahl and Jackendoof 1983 image credit: http://noike.info/~kenzi/roughly/paper/GTTM/12/12_Psychological_and_Linguistic_Connections.html

SLIDE 15

G E S TA LT- B A S E D T H E O R I E S

Proximity Similarity

SLIDE 16

G E N E R AT I V E T H E O RY O F T O N A L M U S I C

Fred Lerdahl and Ray Jackendoff, 1983
Goal of GTTM: to devise a set of rules from which a

complete hierarchical grouping structure can be inferred

Inspired by Gestalt theory and by ideas of “universal

grammar” in language

Assumes an ideal listener familiar with Western tonal

music

SLIDE 17

?

Good Continuation

SLIDE 18

M E L O D I C E X P E C TAT I O N

SLIDE 19

M E L O D I C E X P E C TAT I O N

SLIDE 20

M E L O D I C E X P E C TAT I O N

SLIDE 21

I M P L I C AT I O N - R E A L I Z AT I O N T H E O RY

Eugene Narmour, 1990
Goal of I-R Theory: to devise a set of rules from which

a complete hierarchical grouping structure can be inferred

…using explicit reference to human cognitive

processes

…while carefully separating what is universal from

what is culturally learned

SLIDE 22

Two expectations are universal:
A + A → A
A + B → C

I M P L I C AT I O N - R E A L I Z AT I O N T H E O RY

𝄟 ♩ ♩ ♩ 𝄟 ♩ ♩ ♩

SLIDE 23

Refinements of these expectations, based on interval

size and direction, are culturally learned. For example:

Large intervals usually followed by smaller

intervals

Large intervals usually followed by a change in

direction

I M P L I C AT I O N - R E A L I Z AT I O N T H E O RY

SLIDE 24

How does expectation lead to structure?
Surprise leads to boundaries
Closure leads to boundaries

I M P L I C AT I O N - R E A L I Z AT I O N T H E O RY

SLIDE 25

W H AT I S S T R U C T U R E ?

How did you perceive this piece of music?

drums voice horns

SLIDE 26

I M P L I C AT I O N - R E A L I Z AT I O N T H E O RY

I-R model focuses almost exclusively on melody as a

sequence of intervals.

What about harmony, rhythm, timbre?
Narmour hinted at theory in 1977…
The Analysis and Cognition of Basic Melodic

Structures: The Implication-Realization Model published in 1990…

Parts 2–4 forthcoming…

SLIDE 27

L I S T E N E R C O N S I D E R AT I O N S

Previous theories all posit ideal listeners
i.e., for a given melody, there is a “best” analysis.
But, listeners differ in many ways!
Cultural knowledge
Level of musical training
Listening context
Familiarity with the music

SLIDE 28

E X P E R I M E N T S A B O U T L I S T E N E R D I F F E R E N C E S

Elizabeth Margulis:

What is the effect of repeated listenings?

Listeners heard the same piece four times in a row
Each time, they indicated every single literal

repetition they identified

Margulis tallied the correct indications and their

lengths

SLIDE 29

image credit: http://imslp.org/wiki/Pi%C3%A8ces_de_clavecin_%28Rameau,_Jean-Philippe%29

SLIDE 30

E X P E R I M E N T S A B O U T L I S T E N E R D I F F E R E N C E S

Elizabeth Margulis:

What is the effect of repeated listenings?

Margulis 2012

SLIDE 31

L I S T E N E R C O N S I D E R AT I O N S

Some listener disagreements seem less predictable…

SLIDE 32

Bruderer, McKinney and Kohlrausch 2009

SLIDE 33

W H AT C A U S E S A L I S T E N E R T O H E A R A B O U N D A RY ?

Clarke and Krumhansl 1990:

pause (silence)
return of material (chordal)
change of dynamic
new material
change of rhythm
change of pitch content
change of articulation
start of development
change of register (expansion)
change of dynamic contour
change of texture

Bruderer et al. 2009:

change in harmonic

progression

change in melody
change in tempo
change in rhythm
change in timbre
change in loudness / dynamics
breaks
global structure
repetitions

SLIDE 34

W H AT C A U S E S A L I S T E N E R T O H E A R A B O U N D A RY ?

1. Were listeners paying attention to these features, or

were these features attention-grabbing?

2. Can we trust the listeners to

self-report the correct features?

SLIDE 35

Aviezer, Trope and Todorov 2012

W H AT C A U S E S A L I S T E N E R T O H E A R A B O U N D A RY ?

image credits: various from Google Images

SLIDE 36

What is the viewer paying attention to? What is the listener paying attention to?

SLIDE 37

E X P E R I M E N T S A B O U T L I S T E N E R D I F F E R E N C E S

I ran an experiment last year on…

SLIDE 38

38

E X P E R I M E N T 1 : AT T E N D T O T H E PAT T E R N

SLIDE 39

39

E X P E R I M E N T 1 : AT T E N D T O T H E PAT T E R N

SLIDE 40

40

E X P E R I M E N T 2 : B O U N D A RY S A L I E N C E

SLIDE 41

E X P E R I M E N T 2 : B O U N D A RY S A L I E N C E

Hypothesis: focusing on a feature makes changes in that

feature more salient.

Participants focused on a single feature while listening to

an AB-pattern clip, then rated salience of the change they heard

Independent variable: Match between focal and changing

feature varies: match, convolved, or wrong

SLIDE 42

E X P E R I M E N T 2 : B O U N D A RY S A L I E N C E

Result: Yes,

attention did affect the salience of the changes!

SLIDE 43

E X P E R I M E N T 1 : AT T E N D T O T H E PAT T E R N

Hypothesis: focusing on a feature makes one more

likely to perceive groups according to that feature

Participants secretly primed to focus on a feature with

a distractor task: detect whether a pattern occurs

Then they indicated their preferred grouping.
Independent variables: relevance of probe; presence
f probe.

SLIDE 44

E X P E R I M E N T 1 : AT T E N D T O T H E PAT T E R N

50/50 65/35

SLIDE 45

E X P E R I M E N T 1 : AT T E N D T O T H E PAT T E R N

Result: Yes,

attention did influence the perceived groupings!

Effect varied

with feature

SLIDE 46

E X P E R I M E N T 1 : D E P E N D E N C E O F G R O U P I N G S T R U C T U R E O N AT T E N T I O N

−2 −1 1 2 20 30 40 50

Musical training score Mean confidence in grouping preference

0.00 0.25 0.50 0.75 1.00 20 30 40

Musical training score Mean pattern identification accuracy

SLIDE 47

E X P E R I M E N T I M PA C T

Attention impacts the perception of groupings for

listeners

Disagreements between listeners could be caused by

differences in attention

Add it to the (growing) list: familiarity, training,

SLIDE 48

C O N C L U S I O N

Generative Theory of Tonal Music
Explicit set of rules for generating hierarchical analyses
f tonal music
Implication-Realization Theory
Expectation has a central role in music perception
Founded in cognitive science; makes testable claims
Listener differences challenge both theories
Consider the non-ideal listener

SLIDE 49

T H A N K Y O U !

Hillel Aviezer, Yaacov Trope, and Alexander Todorov. Body cues, not facial expressions,

discriminate between intense positive and negative emotions. Science, 338:1225–1229, 2012.

Michael Bruderer, Martin McKinney, and Armin Kohlrausch. The percep- tion of structural

boundaries in melody lines of Western popular music. MusicæScientæ, 13(2):273–313, 2009.

Eric F. Clarke and Carol L. Krumhansl. Perceiving musical time. Music Perception, 7(3):213–51,

1990.

Fred Lerdahl and Ray S. Jackendoff. A Generative Theory of Tonal Music. MIT Press, 1983.
Elizabeth Margulis. Musical repetition detection across multiple exposures. Music Perception,

29(4):377–385, 2012.

Eugene Narmour. The Analysis and Cognition of Basic Melodic Structures: The Implication-

Realization Model. University of Chicago Press, Chicago, IL, USA, 1990.

Jordan B. L. Smith. Explaining listener differences in the perception of musical structure. PhD

thesis, Queen Mary University of London. 2014.

SLIDE 50

T H A N K Y O U !