PocketSphinx: Open-Source Speech Recognition for Hand-held and - - PowerPoint PPT Presentation
PocketSphinx: Open-Source Speech Recognition for Hand-held and Embedded Devices David Huggins-Daines (dhuggins@cs.cmu.edu) Mohit Kumar (mohitkum@cs.cmu.edu) Arthur Chan (archan@cs.cmu.edu) Alan W Black (awb@cs.cmu.edu) Mosur Ravishankar
05/18/06 Language Technologies Institute, Carnegie Mellon University Slide 1
05/18/06 Language Technologies Institute, Carnegie Mellon University Slide 2
– Open source code under MIT-style license – Widely used in CMU and elsewhere – Mature and stable API
– Statistical Language Model support
– Medium-Large Vocabulary (1-10kwords) – Make it go faster
05/18/06 Language Technologies Institute, Carnegie Mellon University Slide 3
– 128-bit memory bus (6-10GB/sec) – 1.8-3GHz processor (5000 MIPS) – ATA, SATA, or SCSI storage (100-300MB/sec)
– 16 or 32-bit memory bus (100-400MB/sec) – 200-600MHz processor (200-700 MIPS) – SD/MMC or CF storage (1-16MB/sec) – no FPU or vector unit (sometimes a DSP...)
05/18/06 Language Technologies Institute, Carnegie Mellon University Slide 4
– My cell phone is pretty darn fast! – At least as fast as that DEC we
had a real-time 20k system on back in 1996!
– Sphinx benchmarks (shown to
the right) favor large caches and high memory bandwidth (Intel)
– Search, LM, and dictionary look-
up are highly memory-intensive
– We will have to deal with them
(Source: techreport.com)
05/18/06 Language Technologies Institute, Carnegie Mellon University Slide 5
10 1000 5000 0.25 0.5 0.75 1 1.25 1.5 1.75 2 2.25
Hand-held Desktop
# of words in vocabulary Speed (xRT)
05/18/06 Language Technologies Institute, Carnegie Mellon University Slide 6
– Front-end optimizations (fixed-point, logarithm) – Speeding up GMM computation – Old-fashioned beam tuning
– Memory optimization (+ model compression) – Machine-level optimization (assembly code)
– Search optimization – dynamic beam tuning – Language model compression and optimization
05/18/06 Language Technologies Institute, Carnegie Mellon University Slide 7
– 32-bit, 16.16 or 18.14 format – Using 64-bit multiply (SMULL) on ARM, 16.16
– MFCC calculated in log domain, using a
– Allows smaller order FFT and MFCC – Not as useful for large-vocabulary systems
05/18/06 Language Technologies Institute, Carnegie Mellon University Slide 8
– Use previous frame's top codewords to select
– Only update top-N every Mth frame – Can significantly affect accuracy
– Approximate nearest neighbor search in k
– 10% speedup, little or no effect on accuracy
05/18/06 Language Technologies Institute, Carnegie Mellon University Slide 9
– Approximations in the front end and GMM increase
– We would like to enforce a hard limit on the number
– Partition the beam width into bins – Dynamically recompute beam based on bin
– 30% speedup with 10% relative degradation in WER
05/18/06 Language Technologies Institute, Carnegie Mellon University Slide 10
– mmap(2)able, shareable between processes – leverage OS-level caching (virtual memory)
– Inherited from Sphinx-II – Adapted for memory-mapping – 5000+ vocabulary in <32M of RAM
– Pre-built radix tree of triphones->senones
05/18/06 Language Technologies Institute, Carnegie Mellon University Slide 11
Task Vocabulary Perplexity xReal-Time Word Error TIDIGITS 10 13.86 0.5 0.87% RM1 994 46.79 0.71 13.11% WSJ devel5k 4989 143.5 0.96 18.50%
– 206MHz StrongARM running Linux (FPU
– Other embedded Linux platforms – Analog Devices Blackfin, uClinux – WinCE using GNU toolchain (untested)
05/18/06 Language Technologies Institute, Carnegie Mellon University Slide 12
05/18/06 Language Technologies Institute, Carnegie Mellon University Slide 13
– Remove Sphinx-II codebook limitations
– Statistical profiling of LM access patterns
05/18/06 Language Technologies Institute, Carnegie Mellon University Slide 14
This work was supported by DARPA grant NB CH-D-03-0010. The content of the information in this publication does not necessarily reflect the position or the policy of the US Government, and no official endorsement should be inferred.