iJDSP Interactive Illustrations of Speech/Audio Processing Concepts - - PowerPoint PPT Presentation

▶
ijdsp interactive illustrations of
SMART_READER_LITE
LIVE PREVIEW

iJDSP Interactive Illustrations of Speech/Audio Processing Concepts - - PowerPoint PPT Presentation

Mar 13, 2023 268 likes •582 views

J-DSP Editor iJDSP Interactive Illustrations of Speech/Audio Processing Concepts NSF Phase 3 J-DSP Workshop, UCy Presentation of an Independent Study By Girish Kalyanasundaram, MS by Thesis in EE Advisor: Dr. Andreas Spanias, Professor,

slide-1
SLIDE 1

http://jdsp.asu.edu

J-DSP Editor

iJDSP – Interactive Illustrations of Speech/Audio Processing Concepts

NSF Phase 3 J-DSP Workshop, UCy Presentation of an Independent Study By Girish Kalyanasundaram, MS by Thesis in EE Advisor: Dr. Andreas Spanias, Professor, Electrical Engineering, ASU

1

slide-2
SLIDE 2

http://jdsp.asu.edu

J-DSP Editor

Block Diagram Based Learning in iJDSP

  • Requirements:

– Provision of speech/audio signals – Microphone Recording and Playback facility – Frame-by-Frame Processing Capability – Effective visualization tools

  • Effective for

constructing basic systems for visualizing speech/audio DSP concepts.

2

slide-3
SLIDE 3

http://jdsp.asu.edu

J-DSP Editor

Extension of iJDSP Architecture

  • Added framework to create blocks capable of

processing long signals.

  • These blocks can also interface with

conventional blocks processing short signals.

  • Frame by frame processing functionality.
  • Paradigm designed for creating planned

functions in iJDSP – instant setup of complex diagrams.

3

slide-4
SLIDE 4

http://jdsp.asu.edu

J-DSP Editor

The Long Signal Generator

  • Configurable Parameters:

– Frame Size – Inter-frame overlap – Choice of signal – Gain applied to signal

  • Hosts pre-

defined speech, music and noise signals.

  • Frame-wise

traversal and visualization facilitated.

4

slide-5
SLIDE 5

http://jdsp.asu.edu

J-DSP Editor

Sound Recording & Rendering

  • Records sound

at specified sampling rate (8, 16 or 44.1 kHz).

  • Frame-wise

traversal of signal through playback buttons.

  • Sound player

aggregates parsed data for rendering.

5

slide-6
SLIDE 6

http://jdsp.asu.edu

J-DSP Editor

Frame-By-Frame Visualizations

  • Playback buttons

used to traverse through frames

  • f incoming

signal.

Plot Frequency Response

6

slide-7
SLIDE 7

http://jdsp.asu.edu

J-DSP Editor

Speech/Audio DSP Functions

  • Spectrogram
  • Linear Predictive Coding (LPC)
  • Quantization
  • Line Spectral Pairs
  • MPEG I Layer 3 Psychoacoustic Model
  • Loudness Estimation

7

slide-8
SLIDE 8

http://jdsp.asu.edu

J-DSP Editor

Spectrogram

8

slide-9
SLIDE 9

http://jdsp.asu.edu

J-DSP Editor

Spectrogram (Contd.)

  • Enables visualization of spectrograms of long

signals.

  • Can study the properties of spectrograms.
  • Effect of window length and overlap in time

and frequency resolution.

  • Window types and effect of taper on

resolution.

9

slide-10
SLIDE 10

http://jdsp.asu.edu

J-DSP Editor

Linear Predictive Coding

  • LPC dashboard to

view LPC coefficients, formants, residuals, and PZ Plot.

10

slide-11
SLIDE 11

http://jdsp.asu.edu

J-DSP Editor

LPC & Quantization

  • Observation of quantization effects on LPC

coefficients and residual.

  • Observation of overall SNR of signal.

11

slide-12
SLIDE 12

http://jdsp.asu.edu

J-DSP Editor

Quantizer Block

  • Quantizer block can be use in quantizing LPC

coefficients.

12

slide-13
SLIDE 13

http://jdsp.asu.edu

J-DSP Editor

LPC & Quantization – SNR

  • The SNR block has been equipped with

capability to aggregate SNR for entire signals.

13

slide-14
SLIDE 14

http://jdsp.asu.edu

J-DSP Editor

Line Spectral Pairs (LSPs)

  • Block to display LSPs and Line spectral

frequencies of input LPCs.

  • LPC – LSP Placement Demo.
  • Block demonstrating quantization effects of

LPC.

14

slide-15
SLIDE 15

http://jdsp.asu.edu

J-DSP Editor

LSP Block

  • Displays the LSP Filters Pole Zeros Plots and

the line spectral frequency values.

15

slide-16
SLIDE 16

http://jdsp.asu.edu

J-DSP Editor

LPC-LSP Demo

  • LPC poles can be placed and the LSPs will be

shown dynamically.

  • Relation between the LPC pole locations and

the LSP poles properties can be studied.

16

slide-17
SLIDE 17

http://jdsp.asu.edu

J-DSP Editor

LPC-LSP Quantizer

  • This block can demonstrate that LSF

quantization prevents unstable poles better than LPC quantization.

17

slide-18
SLIDE 18

http://jdsp.asu.edu

J-DSP Editor

LPC-LSP Quantizer (Contd.)

  • The block can be used in combination with the

LPC-LSP placement demo block to achieve this.

18

slide-19
SLIDE 19

http://jdsp.asu.edu

J-DSP Editor

LPC-LSP Quantizer (Contd.)

  • Displays to view LPC poles and LSF Zeros

19

slide-20
SLIDE 20

http://jdsp.asu.edu

J-DSP Editor

Psychoacoustic Model

  • The psychoacoustic model block implements

the MPEG I Layer 3 psychoacoustic model .

  • A modified form of the ISO standard C source

code, with improved efficiency for the psychoacoustic model is adopted.

  • Visualizations shown:

– Global Masking Threshold (JND curve) – Psychoacoustically peak-picked spectrum – Comparison between original and psychoacoustically peak-picked signal

20

slide-21
SLIDE 21

http://jdsp.asu.edu

J-DSP Editor

Psychoacoustic Model Interface

  • JND curve.

21

slide-22
SLIDE 22

http://jdsp.asu.edu

J-DSP Editor

Psychoacoustic Model Interface

  • Psychoacoustically masked spectrum.

22

slide-23
SLIDE 23

http://jdsp.asu.edu

J-DSP Editor

Psychoacoustic Model Interface

  • Original vs. peak-picked signal.

23

slide-24
SLIDE 24

http://jdsp.asu.edu

J-DSP Editor

Loudness

  • A measure of perceived

intensity.

  • A ‘psychophysical’

phenomenon.

  • A non-linear quantity.
  • Reflects properties of the

auditory system.

Human Auditory System:

  • The ear acts as a bank of band-pass filters

detecting frequencies in the sound.

  • Perceptual models capture properties of

the human auditory system.

24

slide-25
SLIDE 25

http://jdsp.asu.edu

J-DSP Editor

The Units of Loudness

The ‘Phon’ scale

  • Derived relative to a 1 kHz

sine wave reference.

  • A signal has loudness level
  • f X phons when a 1 kHz

sinusoid of X dB SPL is perceived to be equally loud. The ‘Sone’ Scale

  • For a signal with loudness

level X phons, loudness in sones is:

  • A signal with L sones is

twice as loud as a signal of loudness L/2 sones.

10 40

2

X

L

25

slide-26
SLIDE 26

http://jdsp.asu.edu

J-DSP Editor

Loudness Estimation Methods

  • Neural Excitations (Auditory representations)

– Compute neural activity for stimuli

Outer & Middle Ear Filter Auditory Filters Specific Loudness Computation (Non-linear Transformation) Integrate Sound Excitation Pattern Specific Loudness MOORE-GLASBERG MODEL

  • Approved by ANSI as a new loudness standard.
  • Satisfactory performance for several kinds of spectra.

Loudness

26

slide-27
SLIDE 27

http://jdsp.asu.edu

J-DSP Editor

Loudness in Psychoacoustic Model Block

  • Spectrum, Specific Loudness Pattern and

Total Loudness Displayed in Psychoacoustic Model Block.

27

slide-28
SLIDE 28

http://jdsp.asu.edu

J-DSP Editor

Loudness Illustration

  • Two Signals with

same energy in dB level can have different loudness.

Signal with 3 sines Signal with 1 sine

28

slide-29
SLIDE 29

http://jdsp.asu.edu

J-DSP Editor

THANK YOU

29

slide-30
SLIDE 30

http://jdsp.asu.edu

J-DSP Editor

Loudness Estimation Methods

  • A, B, C Weighting

– Filters spectrum to compute loudness (derived from ELC)

  • Steven’s Law

– A power law for loudness of signal with intensity I:

Equal Loudness Contours (ELC) *

ReplayGain: A loudness leveler using ELC. SpotifyTM uses ReplayGain.

3 .

kI L 

* http://www.music.princeton.edu/~newton/teaching/music3/2007/fletchermunson.html

30