Subspace Modeling and Selection Subspace Modeling and Selection for - PowerPoint PPT Presentation

Subspace Modeling and Selection Subspace Modeling and Selection for Noisy Speech Recognition for Noisy Speech Recognition Chuan-Wei Ting Advisor: Jen-Tzung Chien De pa rtme nt o f Co mpute r Sc ie nc e a nd I nfo rma tio n E ng ine e ring Na tio na l Che ng K ung Unive rsity

Outline Outline ► I ntro duc tio n ► No isy Spe e c h Re c o g nitio n ► Sub spa c e Mo de ling a nd Se le c tio n ► E xpe rime nts ► Co nc lusio ns a nd F uture Wo rks 2

Outline Outline ► Intr oduc tion ► No isy Spe e c h Re c o g nitio n ► Sub spa c e Mo de ling a nd Se le c tio n ► E xpe rime nts ► Co nc lusio ns a nd F uture Wo rks 3

Motivation Motivation ► Why ASR pe rfo rma nc e de g ra de s � Ac o ustic mo de l � I nsuffic ie nt tra ining da ta � Unsuita b le mo de l � Misma tc h o f e nviro nme nts Turn on the light OK! Dong Dong Dong Dong 4

Solutions for Noisy Speech Recognition Solutions for Noisy Speech Recognition ► Mo de l-b a se d c o mpe nsa tio n � MAP a da pta tio n � ML L R a da pta tio n ► Spe e c h e nha nc e me nt � Spe c tra l sub tra c tio n � SPL I CE � Sig na l sub spa c e 5

Outline Outline ► I ntro duc tio n ► Noisy Speec h R ec ognition ► Sub spa c e Mo de ling a nd Se le c tio n ► E xpe rime nts ► Co nc lusio ns a nd F uture Wo rks 6

Spectral Subtraction Spectral Subtraction ► T his a ppro a c h is wide ly use d b e c a use o f its simplic ity a nd e a se o f imple me nta tio n. ► Ho w to e stima te c le a n spe e c h � E stima te sho rt-te rm no ise spe c trum � Sub tra c t no ise spe c trum fro m no isy spe c trum 7

Flowchart of Spectral Subtraction Flowchart of Spectral Subtraction 8

Problems of Spectral Subtraction Problems of Spectral Subtraction ► Unde re stima tio n o r o ve re stima tio n o f no ise le ve l ► Whe n no isy spe e c h spe c trum is ne a r the e stima te d no ise spe c trum, spe c tra l sub tra c tio n ma y o b ta in ne g a tive va lue s a nd the se re sult in lo w-le ve l to ne s (“ music noise ”) in e stima te d c le a n sig na l. 9

SPLICE SPLICE ► SPL I CE (Ste re o -b a se d Pie c e wise L ine a r Co mpe nsa tio n fo r E nviro nme nts) is a te c hniq ue o f e stima ting the c e pstrum o f the c le a n spe e c h fro m the o b se rve d no isy spe e c h. 10

Assumptions for SPLICE Assumptions for SPLICE ► No isy spe e c h c e pstra l ve c to r ha s a distrib utio n o f mixture o f Ga ussia ns ∑ = z z ( ) ( | ) ( ) p p s p s s = z z μ Σ ( | ) ( ; , ) p s N whe re s s ► T he distrib utio n fo r c le a n ve c to r g ive n no isy spe e c h is Ga ussia n who se me a n ve c to r is a line a r tra nsfo rma tio n o f the no isy spe e c h ve c to r. = + y z y z r Γ ( | , ) ( ; , ) p s N s s correction vector expected variance 11

MMSE Speech Enhancement MMSE Speech Enhancement ► Assumptio ns o f SPL I CE a re due to the inhe re nt simplic itie s in de riving a nd imple me nting MMSEe stima tio n. ∑ [ ] [ ] = = y y z z y z ˆ MMSE | ( | ) | , E p s E s y y s [ ] = + y z z r ► Be c a use , we ha ve | , E s y s ∑ ∑ ∑ = + = + y z z z r z z r ˆ MMSE ( | ) ( | ) ( | ) p s p s p s s s s s s 12

Parameter Estimation Parameter Estimation ► I f ste re o da ta is a va ila b le , the pa ra me te rs y z ( | , ) p s o f c o nditio na l PDF c a n b e tra ine d r using ML c rite rio n s ∑ − z y z ( | )( ) p s i i i = r i ˆ ∑ s z ( | ) p s i i whe re z ( | ) ( ) p s p s = z i ( | ) p s ∑ i z ( | ) ( ) p s p s i s 13

Signal Subspace Approach Signal Subspace Approach ► Pro je c t no isy sig na l o nto two sub spa c e s � Sig na l Sub spa c e (c le a n spe e c h a nd no ise ) � No ise Sub spa c e (o nly no ise ) y ► L ine a r mo de l o f c le a n sig na l = ⋅ y x W SS SS × × x is 1 is M W K M whe re SS SS 14

Noisy Signal in Signal Subspace Noisy Signal in Signal Subspace z ► L ine a r mo de l o f no isy sig na l = ⋅ + = + z x n y n W SS SS SS SS � Co rre spo nding c o va ria nc e ma trix is = ⋅ + ⋅ + x n x n T [( )( ) ] R E W W ss ss ss ss ss ss z = + T W R W R ss x ss n 15

Clean Speech Estimation Clean Speech Estimation ► R emoving the c o mpo ne nts in no ise sub spa c e ► R etaining the c o mpo ne nts in sig na l sub spa c e Retain Speech in M K Signal Subspace = M K W T R z W K Remove Noise 16 Subspace

Perceptual Property on Human Auditory System Perceptual Property on Human Auditory System ► Huma n is mo re se nsitive to sig na l disto rtio n c o mpa re d to re sidua l no ise ► Ma sking pro pe rty � Ce rta in a udib le so und b e c o me s ina udib le in the pre se nc e o f a no the r so und 17

Demonstration Demonstration Noise Speech Noisy Speech Noisy Speech (Remove Silence) Distorted Noise Speech Distorted Noisy Speech (Remove Silence) 18

Masking Property Masking Property Clean Signal Noise Signal 19

Analysis of Estimated Signal Analysis of Estimated Signal ► E stima tio n e rro r Signal distortion ε = − = − ⋅ + ⋅ = ε + ε y y y n ˆ ( I) H H y n SS = + y y n ˆ ( ) H SS Residual noise ► E ne rg y o f e rro rs ( ) ε = ε ε = ε ε 2 T T [ ] tr [ ] E E y y y y y ( ) ε = ε ε = ε ε 2 T T [ ] tr [ ] E E n n n n n 20

Filter Estimation Filter Estimation ► I nc o rpo ra te d with pe rc e ptua l c rite rio n H ε y 2 min ε ≤ γσ 2 2 Sub je c t to n ► Sig na l sub spa c e filte r is o b ta ine d b y − = + μ 1 ( ) H R R R y y n opt 21

Implementation Implementation ► Re write the filte r using the de c o mpo sitio n = Λ T o f the c o va ria nc e ma trix R W W y y − = Λ Λ + μ T 1 T ( ) H W W R W W y y n opt − Λ Λ + μ 1 W T ( ) R W � We c a n re pre se nt a s a y y n G g a in func tio n μ ⎧ λ ( k ) ⎡ ⎤ ⎪ = 0 y G , 1 , 2 ,..., k M μ = = ⎨ λ + μ T g ( ) k ⎢ ⎥ H W W kk y opt ⎪ ⎣ ⎦ 0 0 = + ⎩ 0 , 1 ,..., k M K 22

Outline Outline ► I ntro duc tio n ► No isy Spe e c h Re c o g nitio n ► Subspac e Modeling and Selec tion ► E xpe rime nts ► Co nc lusio ns a nd F uture Wo rks 23

Factor Analysis Factor Analysis ► F ind la te nt va ria b le s in o b se rve d da ta Math Language Music Chemical Geography Memory Intellect Artistry 24

Factor Analysis Model Factor Analysis Model ► L ine a r mo de l fo r no isy sig na l Specific = Φ + z f r factors Factor Common loadings factors ► Assumptio ns o n F A mo de l = Ψ E rr T [ ] (diagonal) = ff I T [ ] E M = E fr T [ ] 0 = = = r f z [ ] 0 , [ ] 0 , [ ] 0 E E E 25

FA Solution Using PCA FA Solution Using PCA ► I n this wo rk, we find F A so lutio n b y e ig e nde c o mpo sitio n o f c o va ria nc e ma trix. = ΦΦ + Ψ = Λ T T R W W z = Λ Λ + Λ 1 2 1 2 T T W W W W p p p p m m m Λ = Λ Λ W = diag [ ] [ ] W W whe re p m p m 26

FA Solution FA Solution = W Λ z c 1 2 ► F o rmula te no isy sig na l in PCA fo rm, . c = cc I T ► T he white ne d sa mple with c a n [ ] E K − = Λ c 1 2 z T b e o b ta ine d b y a nd pa rtitio ne d W c = c c T T T b y . [ ] p m f r ► T he c o mmo n fa c to r a nd spe c ific fa c to r c a n b e o b ta ine d fro m = Λ + Λ = Φ + z 1 2 c 1 2 c f r W W p p p m m m 27

Covariance Matrix of Noisy Signal Covariance Matrix of Noisy Signal f ► Co mmo n fa c to rs in F A c o me fro m the f so urc e s o f c le a n sig na l a nd no ise y = + f f f f sig na l , . y n n r ► Spe c ific fa c to rs c a n b e e xpre sse d a s the r sum o f re sidua l c le a n sig na l a nd re sidua l y r = + r r r no ise , . n y n = Φ + + Φ + Φ + + Φ + f r f r f r f r T [( )( ) ] R E z y y n n y y n n = Φ Φ + + Φ Φ + = + T T R R R R R R fy ry fn rn y n 28

Analysis of Estimated Signal Analysis of Estimated Signal ► Using F A, c le a n sig na l is e stima te d fro m princ ipa l sub spa c e a nd mino r sub spa c e = + = + y y y z z ˆ ˆ ˆ H H p m p m ► F o r the c a se o f princ ipa l sub spa c e , the e stima tio n e rro r is g ive n b y = − = − + = + e y y I y n e e ˆ ( ) H H p p p p p p p py pn K Residual Signal Noise Distortion 29