Introduction to The HTK Toolkit Hsin-min Wang Reference: - The HTK - - PowerPoint PPT Presentation

Introduction to The HTK Toolkit

Hsin-min Wang

Reference:

• The HTK Book

2

Outline

An Overview of HTK HTK Processing Stages

– Data Preparation Tools – Training Tools – Testing Tools – Analysis Tools

A Tutorial Example

3

An Overview of HTK

HTK: A toolkit for building Hidden Markov Models HMMs can be used to model any time series and the core of HTK is similarly general-purpose HTK is primarily designed for building HMM-based speech processing tools, in particular speech recognizers

4

An Overview of HTK

Two major processing stages involved in HTK

– Training Phase: The training tools are used to estimate the parameters of a set of HMMs using training utterances and their associated transcriptions – Recognition Phase: Unknown utterances are transcribed using the HTK recognition tools

recognition output

5

An Overview of HTK

HTK Software Architecture

– Much of the functionality of HTK is built into the library modules

• Ensure that every tool interfaces to the outside world in exactly the

same way

Generic Properties of a HTK Tool

– HTK tools are designed to run with a traditional command line style interface

• The main use of configuration files

is to control the detailed behavior

• f the library modules on which all

HTK tools depend

HFoo -T 1 -f 34.3 -a -s myfile file1 file2 HFoo -C Config -f 34.3 -a -s myfile file1 file2

main arguments

6

HTK Processing Stages

Data Preparation Training Testing/Recognition Analysis

7

Data Preparation Phase

In order to build a set of HMMs for acoustic modeling, a set of speech data files and their associated transcriptions are required

– Convert the speech data files into an appropriate parametric format (or the appropriate acoustic feature format) – Convert the associated transcriptions of the speech data files into an appropriate format which consists of the required phone or word labels

HSLAB

– Used both to record the speech and to manually annotate it with any required transcriptions if the speech needs to be recorded or its transcriptions need to be built or modified

HCOPY

– Used to parameterize the speech waveforms to a variety of acoustic feature formats by setting the appropriate configuration variables

8

Data Preparation Phase

LPC linear prediction filter coefficients LPCREFC linear prediction reflection coefficients LPCEPSTRA LPC cepstral coefficients LPDELCEP LPC cepstra plus delta coefficients MFCC mel-frequency cepstral coefficients MELSPEC linear mel-filter bank channel outputs DISCRETE vector quantized data

HLIST

– Used to check the contents of any speech file as well as the results of any conversions before processing large quantities of speech data

HLED

– A script-driven text editor used to make the required transformations to label files, for example, the generation of context-dependent label files

9

Data Preparation Phase

HLSTATS

– Used to gather and display statistical information for the label files

HQUANT

– Used to build a VQ codebook in preparation for build discrete probability HMM systems

10

Training Phase

Prototype HMMs

– Define the topology required for each HMM by writing a prototype definition – HTK allows HMMs to be built with any desired topology – HMM definitions stored as simple text files – All of the HMM parameters (the means and variances of Gaussian distributions) given in the prototype definition are ignored only with exception of the transition probability

11

Training Phase

There are two different versions for acoustic model training which depend on whether the sub-word-level (e.g. the phone-level) boundary information exists in the transcription files or not

– If the training speech files are equipped the sub-word boundaries, i.e., the location of the sub-word boundaries have been marked, the tools HINIT and HREST can be used to train/generate each sub- word HMM model individually with all the speech training data

12

Training Phase

HINIT

– Iteratively computes an initial set of parameter value using the segmental k-means training procedure

• It reads in all of the bootstrap training data and cuts out all of the

examples of a specific phone

• On the first iteration cycle, the training data are uniformly segmented

with respective to its model state sequence, and each model state matching with the corresponding data segments and then means and variances are estimated. If mixture Gaussian models are being

trained, then a modified form of k-means clustering is used

• On the second and successive iteration cycles, the uniform

segmentation is replaced by Viterbi alignment

HREST

– Used to further re-estimate the HMM parameters initially computed by HINIT – Baum-Welch re-estimation procedure is used, instead of the segmental k-means training procedure for HINIT

13

Training Phase

14

Training Phase

15

Training Phase

On the other hand, if the training speech files are not equipped the sub-word-level boundary information, a so- called flat-start training scheme can be used

– In this case all of the phone models are initialized to be identical and have state means and variances equal to the global speech mean and variance. The tool HCOMPV can be used for this

HCOMPV

– Used to calculate the global mean and variance of a set of training data

16

Training Phase

17

Training Phase

Once the initial parameter set of HMMs has been created by either one of the two versions mentioned above, the tool HEREST is further used to perform embedded training

• n the whole set of the HMMs simultaneously using the

entire training set

– HINIT+HREST+HEREST – HCOMPV+HEREST

18

Training Phase

HEREST

– Performs a single Baum-Welch re- estimation of the whole set of the HMMs simultaneously

• For each training utterance, the

corresponding phone models are concatenated and the forward- backward algorithm is used to accumulate the statistics of state

• ccupation, means, variances, etc.,

for each HMM in the sequence

• When all of the training utterances

has been processed, the accumulated statistics are used to re-estimate the HMM parameters

– HEREST is the core HTK training tool

19

Training Phase

Model Refinement

– The philosophy of system construction in HTK is that HMMs should be refined incrementally – CI to CD: A typical progression is to start with a simple set of single Gaussian context-independent phone models and then iteratively refine them by expanding them to include context- dependency and use multiple mixture component Gaussian distributions – Tying: The tool HHED is a HMM definition editor which will clone models into context-dependent sets, apply a variety of parameter tyings and increment the number of mixture components in specified distributions – Adaptation: To improve performance for specific speakers the tools HEADAPT and HVITE can be used to adapt HMMs to better model the characteristics of particular speakers using a small amount of training or adaptation data

20

Recognition Phase

HVITE

– Performs Viterbi-based speech recognition. – Takes a network describing the allowable word sequences, a dictionary defining how each word is pronounced and a set of HMMs as inputs – Supports cross-word triphones, also can run with multiple tokens to generate lattices containing multiple hypotheses – Also can be configured to rescore lattices and perform forced alignments – The word networks needed to drive HVITE are usually either simple word loops in which any word can follow any other word

• r they are directed graphs representing a finite-state task

grammar

• HBUILD and HPARSE are supplied to create the word networks

21

Recognition Phase

22

Recognition Phase

Generating Forced Alignment

– HVITE computes a new network for each input utterance using the word level transcriptions and a dictionary – By default the output transcription will just contain the words and their boundaries. One of the main uses of forced alignment however is to determine the actual pronunciations used in the utterances used to train the HMM system

23

Analysis Phase

The final stage of the HTK Toolkit is the analysis stage

– When the HMM-based recognizer has been built, it is necessary to evaluate its performance by comparing the recognition results with the correct reference transcriptions. An analysis tool called HRESULTS is used for this purpose

HRESULTS

– Performs the comparison of recognition results and correct reference transcriptions by using dynamic programming to align them – The assessment criteria of HRESULTS are compatible with those used by the US National Institute of Standards and Technology (NIST)

24

A Tutorial Example

A Voice-operated interface for phone dialing Dial three three two six five four Dial nine zero four one oh nine Phone Woodland Call Steve Young

– $digit = ONE | TWO | THREE | FOUR | FIVE | SIX | SEVEN | EIGHT | NINE | OH | ZERO; $name = [ JOOP ] JANSEN | [ JULIAN ] ODELL | [ DAVE ] OLLASON | [ PHIL ] WOODLAND | [ STEVE ] YOUNG; ( SENT-START ( DIAL <$digit> | (PHONE|CALL) $name) SENT-END )

25

The Task Grammar

Grammar for Voice Dialing

26

The Word Network

The high level representation of a task grammar is provided for user convenience The HTK recognizer actually requires a word network to be defined using a low level notation called HTK Standard Lattice Format (SLF) in which each word instance and each word-to-word transition is listed explicitly HParse gram wdnet

The input task grammar file The output word network file

27

The Dictionary

The general format of each dictionary entry is WORD [outsym] p1 p2 p3 …

– Function words such as A and TO have multiple pronunciations – The entries for SENT-START and SENT-END have a silence model sil as their pronunciations and null output symbols

28

The Transcription Files

To train a set of HMMs, every file of training data must have an associated phone level transcription Master Label File (MLF) – a single file containing a complete set of transcriptions Once the word level MLF has been created, the phone level MLF can be generated using HLED

words.mlf

#!MLF!# "*/S0001.lab" ONE VALIDATED ACTS OF SCHOOL DISTRICTS . "*/S0002.lab" TWO OTHER CASES ALSO WERE UNDER ADVISEMENT . "*/S0003.lab" BOTH FIGURES (etc.) #!MLF!# "*/S0001.lab" sil w ah n v ae l ih d .. etc

phones0.mlf

29

The Transcription Files

HLEd -l '*' -d dict -i phones0.mlf mkphones0.led words.mlf

• The expand EX command replaces each word in

words.mlf by the corresponding pronunciation in the dictionary file dict.

• The IS command inserts a silence model sil

at the start and end of every utterance.

• The delete DE command deletes all short-pause sp labels,

which are not wanted in the transcription labels at this point.

mkphones0.led EX IS sil sil DE sp

30

Coding the Data

The raw speech waveforms is converted into sequences

• f feature vectors using HCOPY

– A configuration file (config) is needed # Coding parameters TARGETKIND = MFCC_0 TARGETRATE = 100000.0 SAVECOMPRESSED = T SAVEWITHCRC = T WINDOWSIZE = 250000.0 USEHAMMING = T PREEMCOEF = 0.97 NUMCHANS = 26 CEPLIFTER = 22 NUMCEPS = 12 ENORMALISE = F Use MFCC, c0 as the energy component 10ms (100000x100ns) 25ms use a Hamming window Pre-emphasis filter coefficient Filter bank number Cepstral Liftering Setting 12 MFCC coefficients Do not perform energy normalisation

31

Coding the Data

/root/waves/S0001.wav /root/mfc/S0001.mfc /root/waves/S0002.wav /root/mfc/S0002.mfc /root/waves/S0003.wav /root/mfc/S0003.mfc …

HCopy -T 1 -C config -S codetr.scp

32

Creating Monophone HMMs

33

Creating Monophone HMMs

• Creating Flat Start Monophones

HCompV -C config -f 0.01 -m -S train.scp -M hmm0 proto will create a new version of proto in the directory hmm0 in which the zero means and unit variances have been replaced by the global speech means and variances

34

Creating Monophone HMMs

• Embedded Reestimation

The flat start monophones stored in the directory hmm0 are re-estimated using the embedded re-estimation tool HEREST invoked as follows HERest -C config -I phones0.mlf -t 250.0 150.0 1000.0 \

• S train.scp -H hmm0/macros -H hmm0/hmmdefs -M hmm1 monophones0

hmm1 hmm2 hmm2 hmm3

35

Fixing the Silence Model

hmm3

36

Fixing the Silence Model – tee-model

hmm3

• Use a text editor on the file hmm3/hmmdefs to copy the centre state of the sil model

to make a new sp model and store the resulting MMF hmmdefs, which includes the new sp model, in the new directory hmm4.

• Run the HMM editor HHED to add the extra transitions required and tie the sp state

to the centre sil state HHEd -H hmm4/macros -H hmm4/hmmdefs -M hmm5 sil.hed monophones1 where sil.hed contains the following commands AT 2 4 0.2 {sil.transP} //add transitions AT 4 2 0.2 {sil.transP} AT 1 3 0.3 {sp.transP} TI silst {sil.state[3],sp.state[2]}

37

Fixing the Silence Model

• Embedded Reestimation

hmm3

• Finally, another two passes of HEREST are applied using the phone

transcriptions with sp models between words.

• This leaves the set of monophone HMMs created so far in the

directory hmm7.

38

Realigning the Training Data

• The dictionary contains multiple pronunciations for some words,

particularly function words.

• The phone models created so far can be used to realign the training data

and create new transcriptions.

• This can be done with a single invocation of the HTK recognition tool HVITE,

HVite -l '*' -o SWT -b silence -C config -a -H hmm7/macros \

• H hmm7/hmmdefs -i aligned.mlf -m -t 250.0 -y lab \
• I words.mlf -S train.scp dict monophones1
• This command uses the HMMs stored in hmm7 to transform the input word

level transcription words.mlf to the new phone level transcription aligned.mlf using the pronunciations stored in the dictionary dict

39

Making Triphones from Monophones

Convert the monophone transcriptions in aligned.mlf to an equivalent set of triphone transcriptions in wintri.mlf HLEd -n triphones1 -l '*' -i wintri.mlf mktri.led aligned.mlf

40

Making Triphones from Monophones

Reestimation should be again done twice, so that the resultant model sets will ultimately be saved in hmm12. HERest -B -C config -I wintri.mlf -t 250.0 150.0 1000.0 \

• s stats -S train.scp -H hmm11/macros \
• H hmm11/hmmdefs -M hmm12 triphones1

Context-dependent triphones can be made by simply cloning monophones HHEd -B -H hmm9/macros -H hmm9/hmmdefs \

• M hmm10 mktri.hed monophones1

41

Making Tied-State Triphones

• Once all state-tying has been completed

and new models synthesized, some models may share exactly the same 3 states and transition matrices and are thus identical. HHED will compact the model set by finding all identical models and tying them together, and produce a new list of models called tiedlist Decision tree state tying is performed by running HHED in the normal way, i.e. HHEd -B -H hmm12/macros \

• H hmm12/hmmdefs -M hmm13 \

tree.hed triphones1 > log

• The edit script tree.hed, which contains

the instructions regarding which contexts to examine for possible clustering, can be rather long and complex.

42

Recognizing the Test Data

Assuming that test.scp holds a list of the coded test files, then each test file will be recognized and its transcription output to an MLF called recout.mlf by executing the following HVite -H hmm15/macros -H hmm15/hmmdefs -S test.scp \

• l '*' -i recout.mlf -w wdnet \
• p 0.0 -s 5.0 dict tiedlist

43

Recognizing the Test Data - Evaluation

Assuming that the MLF testref.mlf contains word level transcriptions for each test file, the actual performance can be determined by running HRESULTS as follows HResults -I testref.mlf tiedlist recout.mlf the result would be a print-out of the form ====================== HTK Results Analysis ============== Date: Sun Oct 22 16:14:45 1995 Ref : testrefs.mlf Rec : recout.mlf

• ----------------------- Overall Results -----------------

SENT: %Correct=98.50 [H=197, S=3, N=200] WORD: %Corr=99.77, Acc=99.65 [H=853, D=1, S=1, I=1, N=855] ==========================================================