iJDSP Interactive Illustrations of Speech/Audio Processing Concepts - PowerPoint PPT Presentation

J-DSP Editor iJDSP – Interactive Illustrations of Speech/Audio Processing Concepts NSF Phase 3 J-DSP Workshop, UCy Presentation of an Independent Study By Girish Kalyanasundaram, MS by Thesis in EE Advisor: Dr. Andreas Spanias, Professor, Electrical Engineering, ASU http://jdsp.asu.edu 1

Block Diagram Based Learning in iJDSP J-DSP Editor • Effective for constructing basic systems for visualizing speech/audio DSP concepts. • Requirements: – Provision of speech/audio signals – Microphone Recording and Playback facility – Frame-by-Frame Processing Capability – Effective visualization tools http://jdsp.asu.edu 2

Extension of iJDSP Architecture J-DSP Editor • Added framework to create blocks capable of processing long signals. • These blocks can also interface with conventional blocks processing short signals. • Frame by frame processing functionality. • Paradigm designed for creating planned functions in iJDSP – instant setup of complex diagrams. http://jdsp.asu.edu 3

The Long Signal Generator J-DSP Editor • Hosts pre- defined speech, music and noise signals. • Frame-wise traversal and visualization facilitated. • Configurable Parameters: – Frame Size – Inter-frame overlap – Choice of signal – Gain applied to signal http://jdsp.asu.edu 4

Sound Recording & Rendering J-DSP Editor • Records sound at specified sampling rate (8, 16 or 44.1 kHz). • Frame-wise traversal of signal through playback buttons. • Sound player aggregates parsed data for rendering. http://jdsp.asu.edu 5

Frame-By-Frame Visualizations J-DSP Editor Plot Frequency Response • Playback buttons used to traverse through frames of incoming signal. http://jdsp.asu.edu 6

Speech/Audio DSP Functions J-DSP Editor • Spectrogram • Linear Predictive Coding (LPC) • Quantization • Line Spectral Pairs • MPEG I Layer 3 Psychoacoustic Model • Loudness Estimation http://jdsp.asu.edu 7

Spectrogram J-DSP Editor http://jdsp.asu.edu 8

Spectrogram (Contd.) J-DSP Editor • Enables visualization of spectrograms of long signals. • Can study the properties of spectrograms. • Effect of window length and overlap in time and frequency resolution. • Window types and effect of taper on resolution. http://jdsp.asu.edu 9

Linear Predictive Coding J-DSP Editor • LPC dashboard to view LPC coefficients, formants, residuals, and PZ Plot. http://jdsp.asu.edu 10

LPC & Quantization J-DSP Editor • Observation of quantization effects on LPC coefficients and residual. • Observation of overall SNR of signal. http://jdsp.asu.edu 11

Quantizer Block J-DSP Editor • Quantizer block can be use in quantizing LPC coefficients. http://jdsp.asu.edu 12

LPC & Quantization – SNR J-DSP Editor • The SNR block has been equipped with capability to aggregate SNR for entire signals. http://jdsp.asu.edu 13

Line Spectral Pairs (LSPs) J-DSP Editor • Block to display LSPs and Line spectral frequencies of input LPCs. • LPC – LSP Placement Demo. • Block demonstrating quantization effects of LPC. http://jdsp.asu.edu 14

LSP Block J-DSP Editor • Displays the LSP Filters Pole Zeros Plots and the line spectral frequency values. http://jdsp.asu.edu 15

LPC-LSP Demo J-DSP Editor • LPC poles can be placed and the LSPs will be shown dynamically. • Relation between the LPC pole locations and the LSP poles properties can be studied. http://jdsp.asu.edu 16

LPC-LSP Quantizer J-DSP Editor • This block can demonstrate that LSF quantization prevents unstable poles better than LPC quantization. http://jdsp.asu.edu 17

LPC-LSP Quantizer (Contd.) J-DSP Editor • The block can be used in combination with the LPC-LSP placement demo block to achieve this. http://jdsp.asu.edu 18

LPC-LSP Quantizer (Contd.) J-DSP Editor • Displays to view LPC poles and LSF Zeros http://jdsp.asu.edu 19

Psychoacoustic Model J-DSP Editor • The psychoacoustic model block implements the MPEG I Layer 3 psychoacoustic model . • A modified form of the ISO standard C source code, with improved efficiency for the psychoacoustic model is adopted. • Visualizations shown: – Global Masking Threshold (JND curve) – Psychoacoustically peak-picked spectrum – Comparison between original and psychoacoustically peak-picked signal http://jdsp.asu.edu 20

Psychoacoustic Model Interface J-DSP Editor • JND curve. http://jdsp.asu.edu 21

Psychoacoustic Model Interface J-DSP Editor • Psychoacoustically masked spectrum. http://jdsp.asu.edu 22

Psychoacoustic Model Interface J-DSP Editor • Original vs. peak-picked signal. http://jdsp.asu.edu 23

Loudness J-DSP Editor • A measure of perceived intensity. • A ‘psychophysical’ phenomenon. • A non-linear quantity. • Reflects properties of the auditory system. Human Auditory System: • The ear acts as a bank of band-pass filters detecting frequencies in the sound. • Perceptual models capture properties of the human auditory system. http://jdsp.asu.edu 24

The Units of Loudness J-DSP Editor The ‘ Phon ’ scale The ‘ Sone ’ Scale • Derived relative to a 1 kHz • For a signal with loudness sine wave reference. level X phons, loudness in sones is: • A signal has loudness level  X 40  10 of X phons when a 1 kHz L 2 sinusoid of X dB SPL is • A signal with L sones is perceived to be equally twice as loud as a signal of loud. loudness L/2 sones. 25 http://jdsp.asu.edu

Loudness Estimation Methods J-DSP Editor • Neural Excitations (Auditory representations) – Compute neural activity for stimuli Excitation Specific Loudness Outer & Pattern Auditory Computation Middle Ear Sound Filters (Non-linear Filter Transformation) Loudness Integrate Specific MOORE-GLASBERG MODEL Loudness • Approved by ANSI as a new loudness standard. • Satisfactory performance for several kinds of spectra. http://jdsp.asu.edu 26

Loudness in Psychoacoustic Model Block J-DSP Editor • Spectrum, Specific Loudness Pattern and Total Loudness Displayed in Psychoacoustic Model Block. http://jdsp.asu.edu 27

Loudness Illustration J-DSP Editor • Two Signals with same energy in dB level can have different loudness. Signal with 3 sines Signal with 1 sine http://jdsp.asu.edu 28

J-DSP Editor THANK YOU 29 http://jdsp.asu.edu

Loudness Estimation Methods J-DSP Editor • A, B, C Weighting – Filters spectrum to compute loudness (derived from ELC) • Steven’s Law – A power law for loudness of signal with intensity I: L  0 . 3 kI Equal Loudness Contours (ELC) * ReplayGain: A loudness leveler using ELC. Spotify TM uses ReplayGain. * http://www.music.princeton.edu/~newton/teaching/music3/2007/fletchermunson.html http://jdsp.asu.edu 30