Timing relationships between actions and sound in music performance - PowerPoint PPT Presentation

Timing relationships between actions and sound in music performance Peter Q. Pfordresher & Brian Benitez Department of Psychology and Institute for Music Research The University of Texas at San Antonio Supported in part by NSF Grant BCS-0344892

Background • Altered auditory feedback paradigm – Disruption from Delayed Auditory Feedback (DAF, Black, 1951; Lee, 1950 ) • What does DAF disruption mean about temporal coordination between timing of actions and feedback? – Absolute time hypothesis (e.g., MacKay, 1987) – Relative time hypothesis (e.g., Finney & Warren, 2003; Howell et al., 1983; Pfordresher & Palmer, 2002) • Limitation: Delay lengths fixed

Fixed delays and phase Phase relationship 42% 36% 31% 28% 50% 250ms 250ms 250ms 250ms 250ms Delays Onsets IOIs 500ms 600ms 700ms 800ms 900ms IOI #1 (IOI #1) (IOI #1) (IOI #1) (IOI #1) 500ms

Adjustable delays and absolute time Absolute time 300ms 350ms 400ms 450ms 250ms 50% 50% 50% 50% 50% Delays Onsets IOIs 500ms 600ms 700ms 800ms 900ms IOI #1 (IOI #1) (IOI #1) (IOI #1) (IOI #1) 500ms

Predictions • Relative time hypothesis – Advantage for delays that crate onset synchrony (100% , 200%), possibly also 50% (alternation) – Tempo x delay interaction for fixed delays, but not for adjustable delays • Absolute time hypothesis – Disruption maximal for ~ 200ms delay (270 ms according to Gates et al., 1974). • Presume range from 200 – 300ms • Disruption may asymptote (e.g., Howell et al., 1983) or decrease (e.g., Fairbanks & Guttman, 1954) after critical interval – Tempo x delay interaction for adjustable, not fixed

Experiment 1 Method • Participants = 12 non-pianists • Synchronization/Continuation paradigm • Movement type / task complexity – Tap: Isochronous tapping – Sequence: Perform melody on keyboard (simplified) 1 2 3 4 5 1 2 3 5 4 3 2 3

Experiment 1 Method (continued) • Delay type (in addition to normal control) – Fixed: 330ms, 500ms, 660ms – Adjustable: 66%, 100%, 132% • Tempo (IOI): 330ms, 500ms, 660ms tempo adjustable delay values tempo fixed delay values 330 500 660 0.66 1 1.2 330 1 1.52 2 330 217.8 330 396 500 0.66 1 1.32 500 330 500 600 660 0.5 0.76 1 660 435.6 660 792

Data Analysis • Disruption = Mean IOI (Continuation) – Mean IOI (synchronization) – Removed errors (< 5%) and events following errors – Removed outliers (+/- 3 SD) and any < 100 or > 1000 • Synchronization performance (+/- 1 SD): Error bars Sequencing Trials Tapping Trials 750 700 = +/- 1 SD 700 650 650 600 600 Produced IOI Produced IOI 550 550 500 500 450 450 produced produced 400 400 ideal ideal 350 350 300 300 300 400 500 600 700 300 400 500 600 700 Metronm e IOI Metronm e IOI

Results: Sequencing Trials Fixed Delays Delay (ms) 150 0 Diff MN IOI 100 330 Arrows = Error bars 500 50 Integer phase = +/- 1 SE (predict low) 660 0 330 500 660 -50 Proscribed IOI (ms) Adjustable Delays Delay (% IOI) 150 0 Arrows = Diff MN IOI 100 200-300 ms 66% (predict high) 100% 50 132% 0 330 500 660 -50 Proscribed IOI (ms)

Results: Sequencing Trials Tempo (IOI) Delays on different continua 660 ms 500 ms 330 ms Fixed Delays Fixed Delays 100 100 Diff MN IOI Diff MN IOI 50 50 0 0 0 200 400 600 800 0% 50% 100% 150% 200% 250% -50 -50 Plotted as length Plotted as phase Adjustable Delays Adjustable Delays 60 60 40 40 Diff MN IOI 20 Diff MN IOI 20 0 0 -20 0 200 400 600 800 1000 -20 0% 50% 100% 150% -40 -40 -60 -60 Plotted as length Plotted as phase

Results: Tapping Trials Fixed Delays Delay (ms) 150 0 Diff MN IOI 100 330 Arrows = 500 50 Integer phase 660 (predict low) 0 330 500 660 -50 Proscribed IOI (ms) Adjustable Delays Delay (% IOI) 150 0 Diff MN IOI Arrows = 100 66% 200-300 ms 100% 50 (predict high) 132% 0 330 500 660 -50 Proscribed IOI (ms)

Results: Tapping Trials Tempo (IOI) Delays on different continua 660 ms 500 ms 330 ms Fixed Delays Fixed Delays 80 80 60 60 Diff MN IOI Diff MN IOI 40 40 20 20 0 0 -20 0 200 400 600 800 -20 0% 50% 100% 150% 200% 250% Plotted as length Plotted as phase Adjustable Delays Adjustable Delays 60 60 40 Diff MN IOI 40 Diff MN IOI 20 20 0 0 0 200 400 600 800 1000 -20 0% 50% 100% 150% -20 -40 -40 Plotted as length Plotted as phase

Experiment 2: Short delays • Experiment 1 not ideally suited to test for “peak” around delays of 200ms. • Phase shifts at 50% may show facilitation (Pfordresher & Palmer, 2002) • Changed delay amounts – Fixed: 165ms, 250ms, 330ms – Adjustable: 33%, 50%, 66% adjustable delay tempo fixed delay values tempo values 165 250 330 0.33 0.5 0.66 330 0.5 0.76 1 330 108.9 165 217.8 500 0.33 0.5 0.66 500 165 250 330 660 0.25 0.38 0.5 660 217.8 330 435.6

Method (continued) • 12 Additional non-pianists • Same data analysis – Performance on synchronization (+/- 1SD): Sequencing Trials Tapping Trials 700 700 650 650 Error bars 600 600 = +/- 1 SD Produced IOI 550 Produced IOI 550 500 500 450 450 400 produced produced 400 ideal ideal 350 350 300 300 300 400 500 600 700 300 400 500 600 700 Metronm e IOI Metronm e IOI

Results: Sequence Fixed Delays Delay (ms) 150 50% 100% 50% 50% 0 Diff MN IOI 100 165 Arrows = Error bars Integer phase 250 = +/- 1 SE 50 (predict low) 330 0 330 500 660 -50 Proscribed IOI (ms) Adjustable Delays Delay (% IOI) 150 0 Diff MN IOI Arrows = 100 33% 200-300 ms 50% 50 (predict high) 66% 0 330 500 660 -50 Proscribed IOI (ms)

Results: Sequence Tempo (IOI) Delays on different continua 660 ms 500 ms 330 ms Fixed Delays Fixed Delays 100 100 80 80 Diff MN IOI Diff Mn IOI 60 60 40 40 20 20 0 0 -20 0 100 200 300 400 -20 0% 50% 100% 150% Plotted as length Plotted as phase Adjustable Delays Adjustable Delays 80 80 60 60 Mn IOI Mn IOI 40 40 20 20 0 0 0% 20% 40% 60% 80% 0 100 200 300 400 500 Plotted as phase Plotted as length

Results: Tapping Trials Fixed Delays 150 50% 100% 50% 50% Delay (ms) Diff MN IOI 100 0 Arrows = 165 Integer phase 50 250 (predict low) 330 0 m330 m500 m660 -50 Proscribed IOI (ms) Adjustable Delays Delay (% IOI) 150 0 Diff MN IOI 100 33% Arrows = 50% 50 200-300 ms 66% (predict high) 0 m330 m500 m660 -50 Proscribed IOI (ms)

Results: Tapping Trials Tempo (IOI) Delays on different continua 660 ms 500 ms 330 ms Fixed Delays Fixed Delays 60 60 40 40 Diff MN IOI Diff MN IOI 20 20 0 0 0% 50% 100% 150% -20 0 100 200 300 400 -20 Plotted as phase Plotted as length Adjustable Delays Adjustable Delays 80 80 60 Diff MN IOI Diff MN IOI 60 40 40 20 20 0 0 0 100 200 300 400 500 0% 20% 40% 60% 80% Plotted as length Plotted as phase

Both Experiments: Phase 60 50 MN Diff IOI 40 30 20 10 0 0% 50% 100% 150% 200% 250% Plotted as phase 100 60 Fixed Sequence Adjustable 50 80 Tap 40 60 Mn Diff IOI 30 Mn Diff IOI 40 20 20 10 0 0 0% 50% 100% 150% 200% 250% 0% 50% 100% 150% 200% 250% -10 -20 Plotted as phase -20 Plotted as phase

Both Experiments: Length 50 40 MN Diff IOI 30 20 10 0 0 200 400 600 800 1000 Plotted as length Sequence 80 Fixed 60 Tap 70 Adjustable 50 60 50 40 Mn Diff IOI 40 Mn Diff IOI 30 30 20 20 10 10 0 0 0 200 400 600 800 1000 -10 0 200 400 600 800 1000 -20 -10 Plotted as length Plotted as length

Conclusions • Evidence for an advantage of simple phase ratios (but not .5), regardless of delay type or movement type – Maximal disruption around θ = .50 - .75, depending on movement type • Weaker effects of absolute time may also contribute – Global influence across tempo conditions, not evident within each tempo condition – More apparent for adjustable delays, sequencing

And now, a shameless plug… Auditory Perception, Cognition, and Action Meeting (APCAM) Keynote Speaker: Professor Dylan Jones, Cardiff University Thursday November 10, 2005, Toronto (Before Psychonomics) Submission Deadline: August 26, 2005 More information at: www.apcam.us mcauley@bgnet.bgsu.edu