Auralization Technology Maarten Hornikx April 4, 2019 Introduction - - PowerPoint PPT Presentation

Auralization Technology

Maarten Hornikx April 4, 2019

Introduction Auralization technology Discussion/Next steps

Group expertise

Co Computational and Ex Experimental Ac Acoustics Methods

Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps

Group expertise

www.acoustics-engineering.com

To Tools for researchers an and designers

Ac Acoustic prediction software Ac Acoustic measurement software

Fund Fundament ental and nd ap applied resear arch

Pr Propagation effects Au Auralization

Co Computational and Ex Experimental Ac Acoustics Methods

Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps

Group overview

PhD students Post-docs Staff

Maarten Hornikx Constant Hak Qin Yi Baltazar Briere Raúl Pagán Muñoz Fotis Georgiou Jikke Reinten Chang Liu Ella Braat-Eggen Indra Sihar Sai Trikootam Huiqing Wang Wouter Wittebol

Project researchers

Jeroen Donners Jieun Yang JinJack Tan

• pen

position Marc van Baelen

www.tue.nl/buildingacoustics

Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps

Auralization example

Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps

Auralization definition/origin

Auralization is the process of rendering audible, by physical or mathematical modeling, the sound field of a source in a space. [1]

Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps

Role of auralization in the urban sound community

Auralization as communication tool Auralization as (perceptual) research tool

Empa Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps

Contents of presentation

Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps

Contents of presentation

◮ State-of-the-art in auralization technology, specifically for urban sound

Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps

Contents of presentation

◮ State-of-the-art in auralization technology, specifically for urban sound ◮ Challenges in auralization technology

Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps

Contents of presentation

◮ State-of-the-art in auralization technology, specifically for urban sound ◮ Challenges in auralization technology ◮ Discussion and next steps

Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps Overview

Contents

Introduction Auralization technology Overview Auralization components Reproduction Discussion/Next steps

Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps Overview

Auralization chain

Source Path Receiver Reproduction

Georgiou [2] Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps Auralization components

Contents

Introduction Auralization technology Overview Auralization components Reproduction Discussion/Next steps

Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps Auralization components

Source

◮ Measurements

• Near the source [3]
• At a receiver position [4,5]

Meng et al. [4] Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps Auralization components

Source

◮ Predictions

• Full physical modelling (as in musical acoustics [6,7]
• Spectral Modeling Synthesis (SMS): shaped noise plus modal

components ([8] and refs. therein)

◮ Mixed approach

• Granular synthesis [9]

Pieren [8] Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps Auralization components

Path: Measurements

◮ Impulse response measurements

Georgiou [2] Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps Auralization components

Path: Predictions

◮ Full physical modelling (wave-based model) ([10],refs in [11]) ◮ Geometrical acoustics (typically noise-mapping software approaches) (refs

in [11])

Hornikx [12] https://sagetechnologies.com/whats-new/122-olive-tree-lab- terrain Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps Auralization components

Receiver: Measurements

Closely linked to the reproduction system

◮ Capturing spatial sound field by array sensor [13] ◮ Individual HRTFs or HRTF from dummy head [14,22]

Sakamoto et al. [13] Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps Auralization components

Receiver: Predictions

Closely linked to the reproduction system

◮ Computing spatial sound field by receiver array [14] ◮ Modelled HRTFs [15], anthropometry based [16]

Sheaffer et al. [17] Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps Auralization components

Combinations of approaches

For example

Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps Auralization components

Combinations of approaches

For example

◮ Measured source - predicted path - predicted HRTF

Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps Auralization components

Combinations of approaches

For example

◮ Measured source - predicted path - predicted HRTF ◮ Predicted source - predicted path - measured individual HRTF

Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps Auralization components

Combinations of approaches

For example

◮ Measured source - predicted path - predicted HRTF ◮ Predicted source - predicted path - measured individual HRTF ◮ Predicted source - predicted path + HRTF

Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps Auralization components

Combinations of approaches

For example

◮ Measured source - predicted path - predicted HRTF ◮ Predicted source - predicted path - measured individual HRTF ◮ Predicted source - predicted path + HRTF ◮ Measured source + path + HRTF

Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps Reproduction

Contents

Introduction Auralization technology Overview Auralization components Reproduction Discussion/Next steps

Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps Reproduction

Reproduction system

Headphone reproduction

• Reproducing at ears directly
• HRTF predictions/measurements

Multispeaker reproduction

• Reproducing sound field at ’sweet

spot’ in room

• Input from spherical array micro-

phones

Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps Reproduction

Challenges

Moving source, moving receiver, rotating receiver

Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps Reproduction

Challenges

Moving source, moving receiver, rotating receiver

◮ Uncontrolled: Incorporated in measurements

Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps Reproduction

Challenges

Moving source, moving receiver, rotating receiver

◮ Uncontrolled: Incorporated in measurements ◮ Controlled: Incorporation in predictions

Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps Reproduction

Challenges

Moving source, moving receiver, rotating receiver

◮ Uncontrolled: Incorporated in measurements ◮ Controlled: Incorporation in predictions

• Time-variant filtering [18] or convolving with discrete point-to-point

path[10]

Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps Reproduction

Challenges

Moving source, moving receiver, rotating receiver

◮ Uncontrolled: Incorporated in measurements ◮ Controlled: Incorporation in predictions

• Time-variant filtering [18] or convolving with discrete point-to-point

path[10]

• Doppler effect by resampling time signal [18]

Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps Reproduction

Challenges

Moving source, moving receiver, rotating receiver

◮ Uncontrolled: Incorporated in measurements ◮ Controlled: Incorporation in predictions

• Time-variant filtering [18] or convolving with discrete point-to-point

path[10]

• Doppler effect by resampling time signal [18]
• Advanced convolution techniques [19]

Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps Reproduction

Challenges

Moving source, moving receiver, rotating receiver

◮ Uncontrolled: Incorporated in measurements ◮ Controlled: Incorporation in predictions

• Time-variant filtering [18] or convolving with discrete point-to-point

path[10]

• Doppler effect by resampling time signal [18]
• Advanced convolution techniques [19]
• Usage of modern CPU/GPU implementation for calculation of path and

convolution operations [20]

Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps

Discussion

All those components/choices?

Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps

Discussion

All those components/choices? What is good (enough) for reproduction?

Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps

Discussion

All those components/choices? What is good (enough) for reproduction? Quality of Auralization methods needs to be assessed (no standards for this as for audio systems [21])

Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps

Discussion

All those components/choices? What is good (enough) for reproduction? Quality of Auralization methods needs to be assessed (no standards for this as for audio systems [21]) Attributes that can be rated:

Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps

Discussion

All those components/choices? What is good (enough) for reproduction? Quality of Auralization methods needs to be assessed (no standards for this as for audio systems [21]) Attributes that can be rated:

◮ Related to representation of the experienced auralization: Authenticity,

plausibility, realism

Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps

Discussion

All those components/choices? What is good (enough) for reproduction? Quality of Auralization methods needs to be assessed (no standards for this as for audio systems [21]) Attributes that can be rated:

◮ Related to representation of the experienced auralization: Authenticity,

plausibility, realism

◮ Related to auralization system: smoothness, responsiveness,

immersiveness, externalization

Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps

Discussion

All those components/choices? What is good (enough) for reproduction? Quality of Auralization methods needs to be assessed (no standards for this as for audio systems [21]) Attributes that can be rated:

◮ Related to representation of the experienced auralization: Authenticity,

plausibility, realism

◮ Related to auralization system: smoothness, responsiveness,

immersiveness, externalization

◮ Related to sound environment: loudness, pitch, ...

Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps

Discussion

All those components/choices? What is good (enough) for reproduction? Quality of Auralization methods needs to be assessed (no standards for this as for audio systems [21]) Attributes that can be rated:

◮ Related to representation of the experienced auralization: Authenticity,

plausibility, realism

◮ Related to auralization system: smoothness, responsiveness,

immersiveness, externalization

◮ Related to sound environment: loudness, pitch, ... ◮ Direct comparison: Difference between auralization based on

measurements/predictions

Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps

Discussion

Major choice: Loudspeakers or Headphones

Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps

Discussion

Major choice: Loudspeakers or Headphones Choice depends on purpose of the usage:

Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps

Discussion

Major choice: Loudspeakers or Headphones Choice depends on purpose of the usage:

◮ Experience of an existing situation versus perceptual research

Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps

Discussion

Major choice: Loudspeakers or Headphones Choice depends on purpose of the usage:

◮ Experience of an existing situation versus perceptual research ◮ Level of interactivity: static, rotations, movement, own produced sounds?

Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps

Discussion

Major choice: Loudspeakers or Headphones Choice depends on purpose of the usage:

◮ Experience of an existing situation versus perceptual research ◮ Level of interactivity: static, rotations, movement, own produced sounds? ◮ Budget

Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps

Discussion

Major choice: Loudspeakers or Headphones Choice depends on purpose of the usage:

◮ Experience of an existing situation versus perceptual research ◮ Level of interactivity: static, rotations, movement, own produced sounds? ◮ Budget ◮ ...

Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps

Next steps

Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps

Next steps

◮ Quality rating of auralization methods/classes of auralization methods

Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps

Next steps

◮ Quality rating of auralization methods/classes of auralization methods ◮ Multisensory applications

Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps

Next steps

◮ Quality rating of auralization methods/classes of auralization methods ◮ Multisensory applications ◮ High order microphone arrays for capturing spatial and temporal sound

fields

Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps

Next steps

◮ Quality rating of auralization methods/classes of auralization methods ◮ Multisensory applications ◮ High order microphone arrays for capturing spatial and temporal sound

fields

◮ Treadmill simulators for higher level of immersive experience

Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps

Next steps

◮ Quality rating of auralization methods/classes of auralization methods ◮ Multisensory applications ◮ High order microphone arrays for capturing spatial and temporal sound

fields

◮ Treadmill simulators for higher level of immersive experience ◮ Higher level of interactivity: own vocalizations/footsteps

Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps

Next steps

◮ Quality rating of auralization methods/classes of auralization methods ◮ Multisensory applications ◮ High order microphone arrays for capturing spatial and temporal sound

fields

◮ Treadmill simulators for higher level of immersive experience ◮ Higher level of interactivity: own vocalizations/footsteps ◮ Improved real-time capabilities

Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps

Next steps

◮ Quality rating of auralization methods/classes of auralization methods ◮ Multisensory applications ◮ High order microphone arrays for capturing spatial and temporal sound

fields

◮ Treadmill simulators for higher level of immersive experience ◮ Higher level of interactivity: own vocalizations/footsteps ◮ Improved real-time capabilities ◮ Hybrid acoustic modelling

Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps

Next steps

◮ Quality rating of auralization methods/classes of auralization methods ◮ Multisensory applications ◮ High order microphone arrays for capturing spatial and temporal sound

fields

◮ Treadmill simulators for higher level of immersive experience ◮ Higher level of interactivity: own vocalizations/footsteps ◮ Improved real-time capabilities ◮ Hybrid acoustic modelling ◮ Spherical HRTF functions (for nearby sources)

Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps

References

1 Kleiner, M., Dalenb¨ ack, B.I. and Svensson, P., 1993. Auralization-an overview. Journal of the Audio Engineering Society, 41(11), pp.861-875. 2 Georgiou, F., Modeling for auralization of urban environments. PhD thesis, Eindhoven University of Technology, 2018. 3 Johansson, ¨ O., Sch¨

• nfeld, S. and Lindforss, D., 2012. Sound sketch procedure for auralization of the interior sound of a high

speed train. In International Congress and Exposition on Noise Control Engineering: 19/08/2012-22/08/2012. International institute of noise control engineering. 4 Meng, F., Behler, G. and Vorl¨ ander, M., 2018. A synthesis model for a moving sound source based on beamforming. Acta Acustica united with Acustica, 104(2), pp.351-362. 5 Meng, F., Li, Y., Masiero, B. and Vorl¨ ander, M., 2019. Signal reconstruction of fast moving sound sources using compressive

• beamforming. Applied Acoustics, 150, pp. 236-245.

6 Chabassier, J., Chaigne, A. and Joly, P., 2013. Modeling and simulation of a grand piano. The Journal of the Acoustical Society

• f America, 134(1), pp.648-665.

7 Bilbao, S., 2009. Numerical sound synthesis: finite difference schemes and simulation in musical acoustics. John Wiley & Sons. 8 Pieren, R.D., 2018. Auralization of Environmental Acoustical Sceneries: Synthesis of Road Traffic, Railway and Wind Turbine Noise. 9 Maillard, J. and Jagla, J., 2012, August. Auralization of non-stationary traffic noise using sample based synthesis-Comparison with pass-by recordings. In INTER-NOISE and NOISE-CON Congress and Conference Proceedings (Vol. 2012, No. 5, pp. 5978-5989). Institute of Noise Control Engineering. 10 Georgiou, F., Hornikx, M. and Kohlrausch, A., 2019. Auralization of a Car Pass-By Using Impulse Responses Computed with a Wave-Based Method. Acta Acustica united with Acustica, 105(2), pp.381-391. 11 Hornikx, M., 2016. Ten questions concerning computational urban acoustics. Building and Environment, 106, pp.409-421. 12 Hornikx, M., 2009. Numerical modelling of sound propagation to closed urban courtyards (pp. 133-137). Gothenburg, Sweden: Chalmers University of Technology. 13 Sakamoto, S., Hongo, S., Okamoto, T., Iwaya, Y. and Suzuki, Y., 2015. Sound-space recording and binaural presentation system based on a 252-channel microphone array. Acoustical Science and Technology, 36(6), pp.516-526. 14 De Vos, R. and Hornikx, M., 2017. Acoustic properties of tongue clicks used for human echolocation. Acta Acustica United With Acustica, 103(6), pp.1106-1115. Maarten Hornikx TU/e Building Acoustics research group Auralization Technology

Introduction Auralization technology Discussion/Next steps

References

15 Meshram, A., Mehra, R. and Manocha, D., 2014, August. Efficient HRTF computation using adaptive rectangular decomposition. In Audio Engineering Society Conference: 55th International Conference: Spatial Audio. Audio Engineering Society. 16 Bomhardt, R., Lins, M. and Fels, J., 2016. Analytical ellipsoidal model of interaural time differences for the individualization of head-related impulse responses. Journal of the Audio Engineering Society, 64(11), pp.882-894. 17 Sheaffer, J., Webb, C. and Fazenda, B.M., 2013. Modelling binaural receivers in finite difference simulation of room acoustics. In Proceedings of Meetings on Acoustics ICA2013 (Vol. 19, No. 1, p. 015098). ASA. 18 Pieren, R., B¨ utler, T. and Heutschi, K., 2016. Auralization of accelerating passenger cars using spectral modeling synthesis. Applied Sciences, 6(1), p.5. 19 Wefers, F., 2015. Partitioned convolution algorithms for real-time auralization (Vol. 20). Logos Verlag Berlin GmbH 20 Mehra, R., Raghuvanshi, N., Savioja, L., Lin, M.C. and Manocha, D., 2012. An efficient GPU-based time domain solver for the acoustic wave equation. Applied Acoustics, 73(2), pp.83-94 21 ITU-Recomendation BS.1534-3: Method for the subjective assessment of intermediate quality level of audio systems. Tech. rep., International Telecommunication Union, Geneva, Switzerland, 2015. 22 Guezenoc, C. and Seguier, R., 2018, October. HRTF Individualization: A Survey. In Audio Engineering Society Convention 145. Audio Engineering Society. Maarten Hornikx TU/e Building Acoustics research group Auralization Technology