Voice Controlled Smart Spaces Florian Gratzer Advisor: Marc-Oliver - PowerPoint PPT Presentation

Chair of Network Architecures and Services Departments of Informatics Technical University of Munich Voice Controlled Smart Spaces Florian Gratzer Advisor: Marc-Oliver Pahl Garching, June 26 th 2015

Motivation • Controlling the environment via voice commands • Using • DS2OS as middleware • Devices with low energy consumption (Raspberry Pi, Arduino, …) • Pocketsphinx for voice recognition • Smartphones for voice input • Mapping of voice input to actions within DS2OS Florian Gratzer | Voice Controlled Smart Spaces 2

Voice recognition • Human speech waveform contains a large amount of information • Dependent on • Speaker • Speaking rate • Acoustic conditions • Hardly possible to match samples directly • Multiple processing steps are required • Using “Features” for matching Florian Gratzer | Voice Controlled Smart Spaces 3

Processing steps in voice recognition • Start and end time detection • Manual • (Semi-)automatic • Feature Extraction • Filtering • Windowing • Extracting features • Feature Matching • Change features before matching • Calculate distance between said utterance and known samples [B. Pfister – Sprachverarbeitung ISBN: 9788578110796] Florian Gratzer | Voice Controlled Smart Spaces 4

Interface of DS2OS VSL [http://pahl.de/?site=__publications#pahlDissertation_ds2os_2014] Florian Gratzer | Voice Controlled Smart Spaces 5

Requirements • Usability • Offline functionality • Runnable on low power hardware • Low error rate • Voice feedback Florian Gratzer | Voice Controlled Smart Spaces 6

Related work • Cloud based solutions [https://lh3.googleusercontent.com/E4UjLg_CtHlcAHKQlR X044ZtRERwWPqgD0VLL2TLo3LAGH8hZssK84SEwyNS [https:// upload.wikimedia.org/wikipedia/de/archive/8/86/20120916104811!Siri-Logo.png] sLFsfdO2=w300] [http://byticious.com/wp-content/uploads/2016/03/Google_Cloud_Speech_API.jpg] [http://www.windowsphonearea.com/wp- content/uploads/2014/04/cortana-face-logo.jpg] [http://img.scoop.it/4nryVafz-j2Vj_x-SlxB-Tl72eJkfbmt4t8yenImKBVvK0kTmF0xjctABnaLJIm9] [https://govivace.com/images/logo_govivac.png] Florian Gratzer | Voice Controlled Smart Spaces 7

Related work • MOVI Arduino Shield • 150 full-sentence voice commands • CMUSphinx for voice recognition • Debian based • System of c’t [http://www.audeme.com/movi.html] • Controlling Phillips Hue lamps • REST API • Jasper [https://cdn-cx-images.dynamite.myharmony.com/mh/ssv2/devices/philips-hue-glamour-v2.png] Florian Gratzer | Voice Controlled Smart Spaces 8

Design [http://img2.jalousiescout.de/products/20000020/20000020_i2_weiss.jpg] [http://cdn.idealo.com/folder/Product/1941/4/1941409/s1_produktbild_mid/falcon-appliances-classic-90-induktion.jpg] [http://d2rormqr1qwzpz.cloudfront.net/photos/2013/04/03/47185-arduino.jpg] [https://upload.wikimedia.org/wikipedia/commons/3/31/Raspberry_Pi_2_Model_B_v1.1_top_new_(bg_cut_out).jpg] [http://icons.iconarchive.com/icons/iconsmind/outline/128/Microphone-icon.png] [http://icons.iconarchive.com/icons/iconsmind/outline/128/Speaker-2-icon.png] Florian Gratzer | Voice Controlled Smart Spaces 9

Implementation • 5 types of services • Voice recognition services [https://upload.wikimedia.org/wikipedia/commons/3/31/Raspberry_Pi_2_Model_B_v1.1_top_new_(bg_cut_out).jpg] [http://icons.iconarchive.com/icons/iconsmind/outline/128/Microphone-icon.png] Florian Gratzer | Voice Controlled Smart Spaces 10

Implementation • 5 types of services • Voice recognition services • Voice output services [https://upload.wikimedia.org/wikipedia/commons/3/31/Raspberry_Pi_2_Model_B_v1.1_top_new_(bg_cut_out).jpg] [http://icons.iconarchive.com/icons/iconsmind/outline/128/Microphone-icon.png] [http://icons.iconarchive.com/icons/iconsmind/outline/128/Speaker-2-icon.png] Florian Gratzer | Voice Controlled Smart Spaces 11

Implementation • 5 types of services • Voice recognition services • Adaption Services • Voice output services [http://img2.jalousiescout.de/products/20000020/20000020_i2_weiss.jpg] [http://d2rormqr1qwzpz.cloudfront.net/photos/2013/04/03/47185-arduino.jpg] [https://upload.wikimedia.org/wikipedia/commons/3/31/Raspberry_Pi_2_Model_B_v1.1_top_new_(bg_cut_out).jpg] [http://icons.iconarchive.com/icons/iconsmind/outline/128/Microphone-icon.png] [http://icons.iconarchive.com/icons/iconsmind/outline/128/Speaker-2-icon.png] Florian Gratzer | Voice Controlled Smart Spaces 12

Implementation • 5 types of services • Command mapping service [http://img2.jalousiescout.de/products/20000020/20000020_i2_weiss.jpg] [http://cdn.idealo.com/folder/Product/1941/4/1941409/s1_produktbild_mid/falcon-appliances-classic-90-induktion.jpg] [http://d2rormqr1qwzpz.cloudfront.net/photos/2013/04/03/47185-arduino.jpg] [https://upload.wikimedia.org/wikipedia/commons/3/31/Raspberry_Pi_2_Model_B_v1.1_top_new_(bg_cut_out).jpg] [http://icons.iconarchive.com/icons/iconsmind/outline/128/Microphone-icon.png] [http://icons.iconarchive.com/icons/iconsmind/outline/128/Speaker-2-icon.png] Florian Gratzer | Voice Controlled Smart Spaces 13

Implementation • 5 types of services • Configuration interface for command mapping [http://img2.jalousiescout.de/products/20000020/20000020_i2_weiss.jpg] [http://cdn.idealo.com/folder/Product/1941/4/1941409/s1_produktbild_mid/falcon-appliances-classic-90-induktion.jpg] [http://d2rormqr1qwzpz.cloudfront.net/photos/2013/04/03/47185-arduino.jpg] [https://upload.wikimedia.org/wikipedia/commons/3/31/Raspberry_Pi_2_Model_B_v1.1_top_new_(bg_cut_out).jpg] [http://icons.iconarchive.com/icons/iconsmind/outline/128/Microphone-icon.png] [http://icons.iconarchive.com/icons/iconsmind/outline/128/Speaker-2-icon.png] Florian Gratzer | Voice Controlled Smart Spaces 14

Evaluation • Test the system in different scenarios • Model house with different scenarios • Real room at university • Does the system fulfill the requirements? [http://pahl.de/?site=projects/experimentalS2oSpaces] • Usability • Error Rate Florian Gratzer | Voice Controlled Smart Spaces 15

Thank you for your attention! Florian Gratzer | Voice Controlled Smart Spaces 16