[PPT] - Identification of perceptual qualities in textural sounds using the PowerPoint Presentation

SLIDE 1

Austrian Research Institute for Artificial Intelligence (OFAI)

Audio Mostly 2011, Coimbra/Portugal

Identification of perceptual qualities in textural sounds using the repertory grid method

Thomas Grill, Arthur Flexer (OFAI), Stuart Cunningham (CAIR)

SLIDE 2

SLIDE 3

Grill, Flexer, Cunningham: Identification of perceptual qualities in textural sounds using the repertory grid method Audio Mostly 2011, Coimbra/Portugal

Interfacing with recorded sounds

Sound designers and electronic musicians maintain huge

collections of sampled digital sounds

Usual categorization scheme: Semantic tagging

(sound origin, recording context, etc.)

Sonic qualities are equally important/interesting for musical

use of samples as sound material

Samples are often non-instrumental sounds (atmospheres etc.)

3

SLIDE 4

Grill, Flexer, Cunningham: Identification of perceptual qualities in textural sounds using the repertory grid method Audio Mostly 2011, Coimbra/Portugal

cataRT: Interfacing with sounds using MIR-style audio descriptors

4

SpectralCentroid ⟶ Periodicity ⟶

SLIDE 5

Grill, Flexer, Cunningham: Identification of perceptual qualities in textural sounds using the repertory grid method Audio Mostly 2011, Coimbra/Portugal

Larger scope: Sample-based electronic instruments

5

SLIDE 6

Grill, Flexer, Cunningham: Identification of perceptual qualities in textural sounds using the repertory grid method Audio Mostly 2011, Coimbra/Portugal

Larger scope: Sample-based electronic instruments

For live-performance in improvised or composed music
No pre-listening
Be agile!
Be expressive!
Be present!

6

➡Look into strategies of embodied interaction ➡Have agents assist you (in clever ways)

SLIDE 7

Grill, Flexer, Cunningham: Identification of perceptual qualities in textural sounds using the repertory grid method Audio Mostly 2011, Coimbra/Portugal [*] N. Saint-Arnaud and K. Popat. “Analysis and Synthesis of Sound Textures”, Auditory Scene Analysis,

D. F. Rosenthal, Horoshi G. Okuno, editors. Lawrence Erlbaum Association, New Jersey, 1998

Textural sounds

A sound texture should exhibit similar characteristics over time.

It can have local structure and randomness but the characteristics of the fine structure must remain constant on the large scale.

A sound texture is characterized by its sustain. [...]

High-level characteristics must be exposed within the attention span of a few seconds.

7

SLIDE 8

Grill, Flexer, Cunningham: Identification of perceptual qualities in textural sounds using the repertory grid method Audio Mostly 2011, Coimbra/Portugal

Textural sounds (example)

8

SLIDE 9

Grill, Flexer, Cunningham: Identification of perceptual qualities in textural sounds using the repertory grid method Audio Mostly 2011, Coimbra/Portugal [1] Pierre Schaeffer, Traité des objets musicaux, 1966 [2] George Kelly, The psychology of personal constructs, 1955

Sounds and perception

Non-expert vs. expert listening
Connotation of sound vs. Reduced Listening [1]
Attribution of Personal Constructs (PCs) [2]

9

SLIDE 10

Grill, Flexer, Cunningham: Identification of perceptual qualities in textural sounds using the repertory grid method Audio Mostly 2011, Coimbra/Portugal [*] George Kelly, The psychology of personal constructs, 1955

Repertory Grid interview method

Systematic approach to uncover PCs
PCs represent world-views
Presentation of examples to represent a specific part of the

world

Choice of subjects helps to discover group norms
Minimal intervention by the interviewer

10

SLIDE 11

Grill, Flexer, Cunningham: Identification of perceptual qualities in textural sounds using the repertory grid method Audio Mostly 2011, Coimbra/Portugal

Repertory Grid for sounds / 1

Focusing on textural sounds, both natural and abstract
Hours of sounds boiled down to 100 examples (5–10 secs)
Again condensed to 20 most varied textures
Interviewees (subjects) are asked to name differences

between two randomly chosen examples

11

SLIDE 12

Grill, Flexer, Cunningham: Identification of perceptual qualities in textural sounds using the repertory grid method Audio Mostly 2011, Coimbra/Portugal

Example

12

Straight differentiation:

In which ways do two sounds differ?

Triads:

Group three objects to form two groups, then name differences between groups

SLIDE 13

Grill, Flexer, Cunningham: Identification of perceptual qualities in textural sounds using the repertory grid method Audio Mostly 2011, Coimbra/Portugal

Test setup

Sounds mixed-down to mono, loudness normalized,

about 5 seconds long

One close high-quality speaker (single source)
Subjects from the authors' research and artistic contexts
Subjects were triggered to focus on the perceived quality

rather than on potential origins or sources

Sounds randomly chosen, no visible file names etc.

13

SLIDE 14

Grill, Flexer, Cunningham: Identification of perceptual qualities in textural sounds using the repertory grid method Audio Mostly 2011, Coimbra/Portugal

Repertory Grid for sounds / 2

Elicitation of 10–15 bipolar constructs per subject
Subjects rate all 20 sounds (grades 1 to 5)

using own personal constructs

14

motion textural impulse high excentric evolutionary well-defined regular narrative pitched smooth static coherent continuous low contained repetitive diffused irregular static non-pitched porous

A

4 4 4 1 2 4 4 2 4 3 3

B

5 3 5 5 5 1 3 1 5 2 1

C

4 5 2 2 4

5

5 3 5 5 4

D

4 2 5 4 3 4 4 3 4 2 3

E

2 4 1 1 2 4 1 5 5 3 5

F

1 1 2 2 2

3

2 5 5 4 5

G

5 5 5 5 5 2 1 2 5 1 1

H

4 3 3 1 2 5 1 1 5 2 4

I

4 2 2 2 2 5 2 2 4 1 4

J

2 1 5 3 1

2

5 5 3 5 3

K

5 2 4 4 4 4 3 1 5 4 2

L

1 1 1 3 1

2

1 5 5 5 5

M

4 5 5 1 2 2 3 2 5 3 2

N

3 1 4 4 1 4 4 5 5 4 2

O

4 2 4 3 3

3

5 4 3 5 3

P

2 2 3 3 3 4 5 3 5 5 4

Q

5 5 5 3 5

5

1 1 5 1 1

R

3 3 4 2 3 2 2 3 4 2 3

S

2 2 5 2 3 4 4 4 2 3 2

T

1 1 4 4 1 4 3 2 3 5 2

1 … 5

SLIDE 15

Grill, Flexer, Cunningham: Identification of perceptual qualities in textural sounds using the repertory grid method Audio Mostly 2011, Coimbra/Portugal

Repertory Grid for sounds / 3

Comparison of PCs possible using (Euclidean) distances

between 20-dimensional vectors characterizing each PC

Self-similarity matrix of PCs (in total 202×202 elements)
Low distance indicates synonymy of PCs (among all subjects)
Hierarchical clustering for grouping

15

SLIDE 16

Grill, Flexer, Cunningham: Identification of perceptual qualities in textural sounds using the repertory grid method Audio Mostly 2011, Coimbra/Portugal

16 subjects
expert listeners
202 constructs
mostly German

high/low

rdered/chaotic

16

SLIDE 17

Grill, Flexer, Cunningham: Identification of perceptual qualities in textural sounds using the repertory grid method Audio Mostly 2011, Coimbra/Portugal

Distilled constructs

PCs in German original PCs translated to English

hoch – tief hell – dunkel high – low bright – dull regelmäßig – unregelmäßig geordnet – chaotisch

rdered – chaotic

coherent – erratic glatt – rau weich – rau smooth – coarse soft – raspy natürlich – künstlich analog – digital natural – artificial analog – digital statisch – dynamisch starr – bewegt static – dynamic rigid – eventful nahe – fern klar – verschwommen near – far clear – blurred kantig – rund zerrissen – kompakt edgy – flowing disjointed – continuous dicht – spärlich flächig – punktuell dense – sparse expansive – selective homogen – heterogen gleichförmig – differenziert homogeneous – heterogeneous uniform – differentiated tonhaltig – geräuschhaft tonal – noisy

17

SLIDE 18

Grill, Flexer, Cunningham: Identification of perceptual qualities in textural sounds using the repertory grid method Audio Mostly 2011, Coimbra/Portugal

Evaluate constructs on full set of sounds

Web-based form connected to a database
Rating of random examples from the full set of 100 sounds in

respect to the distilled constructs

Posted to various forums (music-IR, auditory, etc.)
German and English version

18

SLIDE 19

Grill, Flexer, Cunningham: Identification of perceptual qualities in textural sounds using the repertory grid method Audio Mostly 2011, Coimbra/Portugal 19

http://grrrr.org/test/classify

SLIDE 20

Grill, Flexer, Cunningham: Identification of perceptual qualities in textural sounds using the repertory grid method Audio Mostly 2011, Coimbra/Portugal

Evaluation

104 subjects in total
59 (35 german, 24 english) with more than 10 ratings
>16800 grades on 10 qualities / 100 sounds
on average 9.4 constructs rated per sound
each item has been graded at least 10 times,
n average more than 16 times
Variance normalization per user (to account for different

grading habits)

20

SLIDE 21

Grill, Flexer, Cunningham: Identification of perceptual qualities in textural sounds using the repertory grid method Audio Mostly 2011, Coimbra/Portugal

Sounds along axis high–low

21

⟵ high low ⟶

SLIDE 22

Grill, Flexer, Cunningham: Identification of perceptual qualities in textural sounds using the repertory grid method Audio Mostly 2011, Coimbra/Portugal

Sounds along axis ordered–chaotic

22

⟵ ordered chaotic ⟶

SLIDE 23

Grill, Flexer, Cunningham: Identification of perceptual qualities in textural sounds using the repertory grid method Audio Mostly 2011, Coimbra/Portugal

Sounds along axis tonal–noisy

23

⟵ tonal noisy ⟶

SLIDE 24

Grill, Flexer, Cunningham: Identification of perceptual qualities in textural sounds using the repertory grid method Audio Mostly 2011, Coimbra/Portugal

Inter-rater agreement

24

*nine subjects who took part in the elicitation process Construct Agreement α (core group)* Agreement α (all n ≥ 10) high – low 0.588 0.519

rdered – chaotic

0.556 0.447 natural – artificial 0.551 0.492 smooth – coarse 0.527 0.420 tonal – noisy 0.523 0.435 homogeneous – heterogeneous 0.519 0.416 dense – sparse 0.492 0.342 edgy – flowing 0.465 0.376 static – dynamic 0.403 0.383 near – far 0.252 0.249

SLIDE 25

Grill, Flexer, Cunningham: Identification of perceptual qualities in textural sounds using the repertory grid method Audio Mostly 2011, Coimbra/Portugal

Agreement of average ratings between raters using German resp. English

25

Construct Agreement α (all n ≥ 10) natural – artificial 0.871 high – low 0.859 smooth – coarse 0.852 tonal – noisy 0.843

rdered – chaotic

0.831 homogeneous – heterogeneous 0.831 near – far 0.735 dense – sparse 0.706 static – dynamic 0.608 edgy – flowing 0.593

SLIDE 26

Grill, Flexer, Cunningham: Identification of perceptual qualities in textural sounds using the repertory grid method Audio Mostly 2011, Coimbra/Portugal

Pearson correlation between constructs

26

SLIDE 27

Grill, Flexer, Cunningham: Identification of perceptual qualities in textural sounds using the repertory grid method Audio Mostly 2011, Coimbra/Portugal

Pearson correlation between constructs

27

SLIDE 28

Grill, Flexer, Cunningham: Identification of perceptual qualities in textural sounds using the repertory grid method Audio Mostly 2011, Coimbra/Portugal

Correlation of constructs to some standard audio descriptors

28

SLIDE 29

Grill, Flexer, Cunningham: Identification of perceptual qualities in textural sounds using the repertory grid method Audio Mostly 2011, Coimbra/Portugal

Summary

By use of systematic listening tests with the Repertory Grid

method we have identified 10 bi-polar constructs to describe textural sounds

Further evaluation on a larger set of sounds yielded inter-

rater agreement and correlation measures

6 constructs show a reasonable agreement among the

subjects of the study

Huge data-set available for further studies

29

SLIDE 30

Grill, Flexer, Cunningham: Identification of perceptual qualities in textural sounds using the repertory grid method Audio Mostly 2011, Coimbra/Portugal

Future work

Construction of computable audio descriptors representing

the perceptual qualities

Use the Repertory Grid method also for gestural sounds
Using the identified qualities as a semantic starting-point for
ther modalities, e.g. visual representation

30

SLIDE 31

Grill, Flexer, Cunningham: Identification of perceptual qualities in textural sounds using the repertory grid method Audio Mostly 2011, Coimbra/Portugal

Visualization of textural sounds

31

Continuous map

f texture set

Iconic representation

f individual textures

SLIDE 32

Grill, Flexer, Cunningham: Identification of perceptual qualities in textural sounds using the repertory grid method Audio Mostly 2011, Coimbra/Portugal

This is the end

Your comments please!
Thanks to Vienna Science and Technology Fund (WWTF)
Get in contact:

Thomas Grill – gr@grrrr.org Arthur Flexer – arthur.flexer@ofai.at Stuart Cunningham – s.cunningham@glyndwr.ac.uk

32