Hearing & Music (Chapter 10-11a) Lecture 18 Jonathan Pillow - - PowerPoint PPT Presentation

(Chapter 10-11a) Lecture 18

Jonathan Pillow Sensation & Perception (PSY 345 / NEU 325) 
 Spring 2019

Hearing & Music

1

Example: guitar string

Fundamental F1 (1st harmonic) 2nd harmonic F2 (2 x F1) 3rd harmonic F3 (3 x F1)

Harmonics

• Objects tend to vibrate at multiple “resonant frequencies”


(integer multiples of some fundamental frequency)

• most vibrations die down, but some persist because their

wavelength is reinforced by the object’s physical properties

• Auditory system acutely sensitive to harmonics

2

Many sounds, including voices, are harmonic

3

If the fundamental of a harmonic sound is removed, listeners will still hear its pitch

demo:

https://oup-arc.com/access/content/sensation-and-perception-5e-student-resources/ sensation-and-perception-5e-activity-10-2?previousFilter=tag_chapter-10

4

Missing Fundamental

• only 3

harmonics are needed

5

Only three harmonics are needed to hear a missing fundamental

• all harmonics are

aligned at the fundamental freq.

• fundamental could

therefore be conveyed by temporal code (“phase locking”)

2 3 4 2+3+4

6

Only three harmonics are needed to hear a missing fundamental

• Could also be

conveyed by “pattern matching”

• f the place code
• n the cochlea

2 3 4 2+3+4

7

Complex Sounds

Timbre: Psychological sensation by which a listener can judge that two sounds with the same fundamental loudness and pitch are dissimilar

• conveyed by harmonics and other frequencies
• Perception of timbre depends on context in which

sound is heard Timbre demo:

https://oup-arc.com/access/content/sensation-and-perception-5e-student-resources/ sensation-and-perception-5e-activity-10-3?previousFilter=tag_chapter-10

8

Figure 10.20 Timbre

9

Auditory Scene Analysis

What happens in natural situations?

• Acoustic environment can be a busy place with multiple

sound sources

• How does the auditory system sort out these sources?

Source segregation - processing an auditory scene consisting of multiple sound sources into its separate sources

10

Waveforms from all sounds are summed into a single waveform arriving at the ears

11

Cocktail party effect

• We can “select out” and attend to one conversation even

when many are present simultaneously

• first documented by Colin Cherry, 1953

12

Cocktail party effect

Cherry’s findings:

• Same voice speaking, Presented to Both ears ⇒ Very Difficult
• Same voice speaking, Separate ears ⇒ Easy

13

• Did notice: change from male to female speaker

Cocktail party effect

However, subjects:

• couldn’t identify a single phrase from non-attended ear
• couldn’t say for sure if it was English
• didn’t notice a change to German
• didn’t notice speech being played backward

14

Cocktail party effect

• Suggests we can easily use spatial, timing, and spectral cues

to separate sound streams, but cannot attend to multiple sound streams at the same time!

15

Continuity and Restoration Effects

How do we know that listeners hear sounds as continuous?

• Principle of good continuation: in spite of interruptions, one

can still “hear” a sound

• Experiments (e.g., Kluender and Jenison, 1992) suggest that

missing sounds are restored and encoded in the brain as if they were actually present!

16

Continuity Effects

17

Also true for speech: Adding noise can improve comprehension

• riginal

speech speech w/ gaps gaps filled by noise

18

speech with a gap gap filled by noise (cough) Q: Can you tell which phoneme is missing?

Brain automatically fills in sound that is missing due to noise

19

speech with a gap gap filled by noise (cough) Q: Can you tell which phoneme is missing?

Brain automatically fills in sound that is missing due to noise

20

Continuity and Restoration Effects in Music

http://www.youtube.com/watch?v=8D7hCqGm0X0 Beat-box tutorial:

21

Yanny vs. Laurel

https://www.youtube.com/watch?time_continue=11&v=7X_WvGAhMlQ

22

Summary

• Interaural timing differences (ITD)
• Interaural level differences (ILD)
• MSO, LSO
• cone of confusion
• head-related transfer function (HRTF)
• harmonics
• missing fundamental
• timbre
• auditory scene analysis
• cocktail party effect
• continuity and restoration effects

23

Music and Speech
 Perception

11

24

Music

• Universal: found across cultures
• Also distinctive of cultures
• Can have strong emotional effects
• Has strong physiological effects
• Pythagoras: Numbers and music intervals
• Strong link between music and mathematics

25

The sounds of music (i.e. “pitch”) extend across a frequency range from about 25 to 4200 Hz

• 7 octaves
• hard to identify
• ctaves above

this limit

26

Musical Intervals

Octave: The interval between two sound frequencies having a ratio of 2:1

• Example: Middle C (C4) has a fundamental

frequency of 261.6 Hz;

• notes that are one octave from middle C:

130.8 Hz (C3) (below) 523.2 Hz (C5) (above)

27

Helix showing two characteristics of musical pitch:

1) tone height - increasing with frequency 2) tone chroma - which

note within a scale (determines “feel” of note)

28

Helix showing two characteristics of musical pitch:

Octave determines pitch more strongly than frequency

• C3 (130.8 Hz) sounds

more similar to C4 (261.6 Hz) than to E3 (164.8 Hz)

• There is more to musical

pitch than just frequency!

29

Chords

Certain combinations are considered pleasing, or “consonant” G Major Chord, several heights G - B - D Varies across cultures, but tends to focus on even ratios of frequencies (F1): “fifth” - 2:3 “major third” - 4:5

30

Dissonant: Less elegant ratios of note frequencies e.g., diminished fifth or “tritone”: 7:5 (six semitones)

The name diabolus in musica ("the Devil in music") has been applied to the interval from at least the early 18th century. Johann Joseph Fux cites the phrase in his seminal 1725 work Gradus ad Parnassum. Georg Telemann in 1733 notes, "'mi against fa', which the ancients called 'Satan in music'.” Because of that original symbolic association with the devil and its avoidance, this interval came to be heard in Western cultural convention as suggesting an "evil" connotative meaning in music. Today the interval continues to suggest an "oppressive", "scary", or "evil" sound.

Disapproved of since the middle ages:

31

• Melody: An arrangement of notes or chords in

succession

• Examples: “Twinkle, Twinkle, Little Star” or

“Baa Baa Black Sheep”

• defined by relationship between notes (i.e.,

relative pitch, not absolute pitch)

• Key: refers to the reference or starting note

(Same melody can be played starting from a different note)

32

Absolute vs. Relative pitch

Most people have relative pitch - they can identify the interval from one note to another (eg, an octave or a fifth). absolute pitch - very rare ability (1 / 10,000 in US) to identify or create a musical note without any external reference

• more prevalent among people who grew up in east Asia

(influence of tonal languages?)

• may also have genetic component (currently unknown)
• musical training during early critical period seems to have

influence

• more common in people with autism / Aspberger’s
• may make it difficult to play music transposed to different key

33

Emotional content

Emotional content of music is:

• 1. psychological, not physical:
• same set of notes can sound “happy” or “sad” depending
• n where you start (C major vs. A minor scale)
• 2. culture-dependent:
• emotional coloration of sounds (even consonant and

dissonant intervals) are not the same across cultures

34

Cultural differences

• Some relationships between notes, such as octaves

(2:1), are universal;

• fifth (3:2) is next most common interval 


(harmonic series is a common inspiration for scales) However, cultures divide up the octave quite differently:

• Western music: 12 semi-tone (equal-tempered) scale


(but “blue note” in jazz/blues falls in between ) 7-note (eg major) scales for composition: do-re-mi-fa-so-la-ti

• Arabic music: 24 quarter-tones
• Japan, China, India many others: 5-note (pentatonic) scale
• Indonesian pentatonic scale: pelog

https://en.wikipedia.org/wiki/Pelog

35