Detecting State Changes of Indoor Everyday Objects using Wi-Fi Channel State Information Sep. 12 th , 2017 Kazuya Ohara*, Takuya Maekawa, and Yasuyuki Matsushita Osaka University, Japan 1
Background 2 Detecting State Changes of Indoor Everyday Objects (e.g. Open/Close door or window) Application Home automation (adaptive HVAC control) Monitoring an independently living elderly person
Background 3 Detecting State Changes of Indoor Everyday Objects (e.g. Open/Close door or window) Application Home automation (adaptive HVAC control) Monitoring an independently living elderly person － Usual Method － Attach sensors to each object High deployment and maintenance costs Use camera images Invasion of privacy
Background 4 Detecting State Changes of Indoor Everyday Objects (e.g. Open/Close door or window) Application door Home automation (adaptive HVAC control) Monitoring an independently living elderly person Wi-Fi signal － Usual Method － Attach sensors to each object High deployment and maintenance costs Use camera images PC AP Invasion of privacy Wi-Fi signal The path of Wi-Fi signal is greatly affected by state changes of the indoor everyday objects. Wi-Fi communications are used widely in indoor environments.
Purpose 5 Purpose Estimate states of indoor everyday objects from propagation information of Wi-Fi signals based on machine learning techniques. Estimating for each time slice door closed open opened close time open event close opened state closed OPEN closed opened receiver Propagation Information transmitter • Channel State Information (CSI) including the various effects such as reflection and path loss of signals high dimensional complex matrix
Channel State Information 6 ⋮ • MIMO (Multiple Input Multiple Output) using multiple antennas for transmitting and receiving signals • OFDM (Orthogonal Frequency Division Multiplexing) using subcarriers whose frequencies are different from each other subcarrier subcarrier
Channel State Information 7 1 ⋮ amplitude and phase change by reflection Im is obtained at each antenna and subcarrier : complex value CSI � : # of transmit antennas : amplitude change � : # of receive antennas : phase change � : # of subcarriers � �∠� � � � Re � �
Problem 8 Wi-Fi signals are very noisy Noise reduction CSI is complex information including reflection Difficult to design feature Existing approach Noise reduction: Aggregate CSI data and reduce the dimensionality using PCA Feature design: Researchers design manually (waveform, frequency of data) CSI data for each element Low dimensional data CSI PCA
Problem 9 Wi-Fi signals are very noisy Noise reduction CSI is complex information including reflection Difficult to design feature Existing approach Noise reduction: Aggregate CSI data and reduce the dimensionality using PCA Feature design: Researchers design manually (waveform, frequency of data) Activity of one person (e.g. walking, falling )
Problem 10 Wi-Fi signals are very noisy Noise reduction CSI is complex information including reflection Difficult to design feature Existing approach Noise reduction: Aggregate CSI data and reduce the dimensionality using PCA Feature design: Researchers design manually (waveform, frequency of data) States of multiple indoor objects separate effects caused by objects Mixed effects by objects
Approach 11 Noise reduction & Separation: Independent Component Analysis (ICA) Feature design: extracted automatically using Convolutional Neural Network (CNN)
Approach 12 Noise reduction & Separation: Independent Component Analysis (ICA) Blind source separation Aloha Aloha Aloha Hello Hello ICA Hello （ noise ） Independent source signals Mixed signals Decomposed signals Feature design: extracted automatically using Convolutional Neural Network (CNN)
Approach 13 Noise reduction & Separation: Independent Component Analysis (ICA) Signals reflected by objects are independently from door from window ICA (noise) Feature design: extracted automatically using Convolutional Neural Network (CNN)
Approach 14 Noise reduction & Separation: Independent Component Analysis (ICA) Signals reflected by objects are independently from door from window ICA (noise) Feature design: extracted automatically using Convolutional Neural Network (CNN) elements of CSI CSI time � � �
Overview of Proposed Method 15 CSI time-series data DNN HMM open close ICA opened Separating effects closed time Smoothing estimates Feature extraction for each time window elements of CSI feature States estimated by DNN include noise. Using HMM, the estimates can be smoothed. estimated state estimated state time time DNN time time estimated by DNN estimated by HMM
Overview of Proposed Method 16 DNN HMM open close ICA opened Separating effects closed for door CSI time-series data DNN HMM open close ICA time opened Separating effects closed for window DNN HMM open close ICA
ICA 17 ICA decomposes amplitude and phase of CSI for each subcarrier. Amplitude of CSI when door open/close Decomposed signals open close open close open close open close The number of Tx-Rx antenna pairs ||h 11 || Tx 1 – Rx 1 Component effected by door ICA ||h 12 || 0 40 80 0 40 80 time [sec.] time [sec.] noise Tx 1 – Rx 2 Can not find apparent changes in First component captures signal the signals when events occurred changes when events occurred
ICA 18 training data window door window door event event event event amplitude unmixing matrix ICA for door Tx 1 – Rx 1 amplitude unmixing matrix ICA for window Tx 1 – Rx 2
Overview of Proposed Method 19 CSI time-series data DNN HMM open close ICA opened Separating effects closed time Smoothing estimates Feature extraction for each time window elements of CSI feature States estimated by DNN include noise. Using HMM, the estimates can be smoothed. estimated state estimated state time time DNN time time estimated by DNN estimated by HMM
DNN 20 each time window Convolutional layer LSTM time CSI data • Amplitude • Phase Aggregate features • extract meaningful features from matrix of CSI data Decomposed amplitude • Decomposed phase
DNN 21 each time window Convolutional layer LSTM Input to HMM time CSI data • Amplitude • Phase Aggregate features • extract meaningful features from matrix of CSI data Decomposed amplitude • Decomposed phase
Overview of Proposed Method 22 CSI time-series data DNN HMM open close ICA opened Separating effects closed time Smoothing estimates Feature extraction for each time window elements of CSI feature States estimated by DNN include noise. Using HMM, the estimates can be smoothed. estimated state estimated state time time DNN time time estimated by DNN estimated by HMM
HMM 23 Knowledge of objects Output of DNN for closed → open → opened → ・・・ each time window Left-to-Right HMM closed DNN (door) open open open close opened opened closed
HMM 24 ground truth estimation DNN estimation closed open opened close HMM estimation w/o knowledge closed open opened close Proposed (HMM estimation w/ grammar) closed open opened close 0 240 60 120 180 time [sec]
Evaluation dataset 25 • Each event of object occurred in an arbitrary order 150 sessions (train: 90%, test: 10%) • Dimension of CSI: , CSI was obtained at the rate of 1000Hz. � � � • We estimate states of objects every 0.1 sec. 7.2 m 3.5 m window 1 window 2 3.5 m PC shade 1 shade 2 AP door cabinet 1 refrigerator cabinet 1 PC cabinet 2 9.5 m screen 7.5 m cabinet 2 10.2 m cabinet 3 cabinet 3 AP door shade shade AP PC window window environment 1 environment 2 environment 3
Evaluation dataset 26 • Each event of object occurred in an arbitrary order 150 sessions (train: 90%, test: 10%) • Dimension of CSI: , CSI was obtained at the rate of 1000Hz. � � � • We estimate states of objects every 0.1 sec. 7.2 m 3.5 m window 1 window 2 3.5 m PC shade 1 shade 2 AP door cabinet 1 window refrigerator shade cabinet 1 PC cabinet 2 9.5 m screen 7.5 m cabinet 2 10.2 m cabinet 3 cabinet 3 AP door shade shade AP PC window window screen door environment 1 environment 2 environment 3
Result for each objects 27 window 1 window 2 door shade 1 shade 2 AP AP cabinet 1 PC PC 1.0 Averaged F-measure cabinet 2 screen 0.9 0.8 cabinet 3 0.7 0.6 0.5 AP AP 0.4 shade 0.3 door 0.2 window PC PC environment 2 0.1 0.0 environment 1 door window1 window2 shade1 shade2 screen door window shade cabinet1 cabinet2 cabinet3 window shade cabinet1 cabinet2 cabinet3 refrigerator window refrigerator shade PC PC cabinet 3 cabinet 1 environment 1 environment 2 environment 3 AP AP cabinet 2 environment 3
F-measure 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 door window1 open environment 1 window2 shade1 shade2 close Result for each event/ state screen door opened window environment 2 shade cabinet1 closed cabinet2 cabinet3 window shade environment 3 cabinet1 cabinet2 cabinet3 refrigerator 28
Recommend
More recommend
