Visualization of Perceptual Qualities in Textural Sounds DAF-x - - PowerPoint PPT Presentation

Austrian Research Institute for Artificial Intelligence (OFAI)

DAF-x 2011, IRCAM/Paris/France

Visualization of Perceptual Qualities in Textural Sounds

Thomas Grill (OFAI), Ulla Rauter

Grill, Rauter: Visualization of Perceptual Qualities in Textural Sounds DAF-x 2011, IRCAM/Paris/France [*] Wolfgang Köhler, Gestalt psychology,1929

Multi-modal perception

2

Grill, Rauter: Visualization of Perceptual Qualities in Textural Sounds DAF-x 2011, IRCAM/Paris/France

Multi-modal perception

• Ecological perception [Gibson]
• Embodied sound and music cognition

[Leman, Noë, Godøy, Jensenius]

3

Grill, Rauter: Visualization of Perceptual Qualities in Textural Sounds DAF-x 2011, IRCAM/Paris/France Grill, Flexer, Cunningham, Identification of perceptual qualities in textural sounds using the repertory grid method, Audio Mostly 2011

so far: Elicitation of perceptual qualities of sound textures

• Repertory grid method:

Systematic approach to uncover PCs, with minimal intervention by the interviewer

• Selection of 20 varied textures presented to 16 interviewees
• Elicitation of 10 "independent" qualities
• Follow-up online survey using 100 textures:

104 subjects in total, >16800 rates

• Evaluation: Interrater agreement, correlation measures

4

Grill, Rauter: Visualization of Perceptual Qualities in Textural Sounds DAF-x 2011, IRCAM/Paris/France

so far: Relevant perceptual constructs for the description of textural sounds

5

*nine subjects who took part in the elicitation process Construct Agreement α (core group)* Agreement α (all n ≥ 10) high – low 0.588 0.519

• rdered – chaotic

0.556 0.447 natural – artificial 0.551 0.492 smooth – coarse 0.527 0.420 tonal – noisy 0.523 0.435 homogeneous – heterogeneous 0.519 0.416 dense – sparse 0.492 0.342 edgy – flowing 0.465 0.376 static – dynamic 0.403 0.383 near – far 0.252 0.249

Grill, Flexer, Cunningham, Identification of perceptual qualities in textural sounds using the repertory grid method, Audio Mostly 2011

Grill, Rauter: Visualization of Perceptual Qualities in Textural Sounds DAF-x 2011, IRCAM/Paris/France

Transferring auditory qualities to the visual domain

• Various different sketches for graphic representations
• Should allow iconic and continuous map representations
• Intuitive first approach, then further iteratively refined

6

Grill, Rauter: Visualization of Perceptual Qualities in Textural Sounds DAF-x 2011, IRCAM/Paris/France

Transferring auditory qualities to the visual domain

• high–low ⇨ brightness and color hue (red–yellow)
• ordered–chaotic ⇨ arrangement of elements
• smooth–coarse ⇨ smoothness of element outline
• tonal–noisy ⇨ color saturation (grey–colorful)
• homogeneous–heterogeneous ⇨

variation in color-related parameters, especially hue

7

Grill, Rauter: Visualization of Perceptual Qualities in Textural Sounds DAF-x 2011, IRCAM/Paris/France

Online survey: Visualization of textural sounds

8

http://grrrr.org/test/texvis

Grill, Rauter: Visualization of Perceptual Qualities in Textural Sounds DAF-x 2011, IRCAM/Paris/France

Online survey: Visualization of textural sounds

9

http://grrrr.org/test/texvis

Grill, Rauter: Visualization of Perceptual Qualities in Textural Sounds DAF-x 2011, IRCAM/Paris/France

Evaluation: Average RMS error

10

Grill, Rauter: Visualization of Perceptual Qualities in Textural Sounds DAF-x 2011, IRCAM/Paris/France

Evaluation: Pearson correlation selection to reference

11

Grill, Rauter: Visualization of Perceptual Qualities in Textural Sounds DAF-x 2011, IRCAM/Paris/France

Evaluation: Pearson correlation selection to reference

11

Grill, Rauter: Visualization of Perceptual Qualities in Textural Sounds DAF-x 2011, IRCAM/Paris/France

Evaluation: Pearson correlation selection to reference

11

Grill, Rauter: Visualization of Perceptual Qualities in Textural Sounds DAF-x 2011, IRCAM/Paris/France

Musical interface: Continuous map of textural sounds

12

http://grrrr.org/test/texvis/map.html

Grill, Rauter: Visualization of Perceptual Qualities in Textural Sounds DAF-x 2011, IRCAM/Paris/France

Thanks for your attention!

• Your comments please!
• Funded by Vienna Science and Technology Fund

(WWTF project Audiominer MA09-024)

• Get in contact:

Thomas Grill – http://grrrr.org

13