Unmanned Aerial Vehicles Using WiFi Chitra R. Karanam and Yasamin - - PowerPoint PPT Presentation

3D Through-Wall Imaging With Unmanned Aerial Vehicles Using WiFi

Chitra R. Karanam and Yasamin Mostofi

Department of Electrical and Computer Engineering University of California Santa Barbara

16th ACM/IEEE International Conference on Information Processing in Sensor Networks, Pittsburgh, PA USA

Sensing with RF Signals

2

Counting people Elderly fall detection Smart home Through-wall imaging

Imaging and Robotics

3

Why imaging? Why robots?

• Search and rescue
• Archaeological exploration
• Surveillance
• Mapping
• Soon to be part of our society
• Can go to hazardous places
• Form autonomous networks
• Allow for autonomous and
• ptimized antenna positioning

Robotic Through-Wall 3D Imaging with WiFi RSSI

4

X1.5 Speed

Robotic Through-Wall 3D Imaging with WiFi RSSI

5

State-of-the-art: 2D Imaging

• Y. Mostofi. "Cooperative wireless-based obstacle/object mapping and see-through

capabilities in robotic networks." IEEE Transactions on Mobile Computing 12.5 (2013): 817-829.

• S. Depatla, L. Buckland, and Y. Mostofi. "X-ray vision with only wifi power measurements

using Rytov wave models." IEEE Transactions on Vehicular Technology 64.4 (2015): 1376-1387.

Challenges in 3D Imaging

• Considerably more under-determined system
• 3D path planning and design
• Localization of air vehicles considerably more challenging

Outline

6

• Proposed 3D Through-wall Imaging Pipeline

− System Modeling and Sparse Signal Processing − Markov Random Field Modeling and Belief Propagation

• Route Design for UAVs: Robotic Paths in 3D
• Experimental Testbed: UAVs and Google

Tangos

• 3D Imaging Experimental Results
• Conclusions

Outline

7

• Proposed 3D Through-wall Imaging Pipeline

− System Modeling and Sparse Signal Processing − Markov Random Field Modeling and Belief Propagation

• Route Design for UAVs: Robotic Paths in 3D
• Experimental Testbed: UAVs and Google

Tangos

• 3D Imaging Experimental Results
• Conclusions

Measurement Model

8

𝑄𝑆 𝒒𝒋, 𝒓𝒋 = 𝑄

𝑄𝑀 𝒒𝒋, 𝒓𝒋 + 𝛿 ෍ 𝑘

𝑒𝑗𝑘𝜃𝑗𝑘 + 𝜂 𝒒𝒋, 𝒓𝒋 = 𝑄𝑄𝑀 𝒒𝒋, 𝒓𝒋 + 𝛿 ෍

𝑘∈ℒ(𝑞𝑗,𝑟𝑗)

𝜃 𝒔𝒌 Δ𝑒 + 𝜂 𝒒𝒋, 𝒓𝒋

WiFi RSSI measurement Path loss Shadowing Modeling error TX pos. RX pos. Decay rate of signal due to object at 𝒔𝑘 Cell dimension Cells along 𝑗𝑢ℎ measurement line

Using Wentzel-Kramers-Brillouin (WKB) wave approximation

𝑏𝑗𝑘 = ቊ1 if 𝑘𝑢ℎ cell is on 𝑗𝑢ℎ measurement line

• therwise

Measurement Model (cont.)

9

Linear measurement model

𝑸 = 𝑩𝑷

𝑄𝑗 ≡ 𝑄𝑆 𝒒𝒋, 𝒓𝒋 − 𝑄𝑄𝑀(𝒒𝒋, 𝒓𝒋) 𝛿 Δ𝑒 ≅ ෍

𝑘∈ℒ 𝑞𝑗,𝑟𝑗

𝜃 𝒔𝒌

𝑄

1

𝑄2 ⋮ 𝑄𝑁 = 𝑏11 𝑏12 𝑏21 𝑏22 ⋯ 𝑏1𝑂 𝑏2𝑂 ⋮ ⋱ ⋮ 𝑏𝑁1 𝑏𝑁2 ⋯ 𝑏𝑁𝑂 𝜃(𝒔1) 𝜃(𝒔2) ⋮ 𝜃(𝒔𝑂) Observation vector: 𝑸

Cells along a measurement line

Vector of signal decay rate: 𝑷

3D domain

Sparse Signal Processing

10

𝑸 = 𝑩 × 𝑷

Linear measurement model

• M ≪ N - severely under-determined system
• Real spaces – generally sparse in spatial variations
• Regularization – Total Variation minimization

𝑛𝑗𝑜𝑗𝑛𝑗𝑨𝑓 𝑈𝑊 𝑷 , 𝑡𝑣𝑐𝑘𝑓𝑑𝑢 𝑢𝑝 𝑸 = 𝑩𝑷 𝑈𝑊 𝑷 = ෍

𝑛=1 𝑂

𝐸𝑛 𝑷 2 ,

where,

𝐸𝑛𝑷 = 𝐽𝑗+1,𝑘,𝑙 − 𝐽𝑗,𝑘,𝑙 𝐽𝑗,𝑘+1,𝑙 − 𝐽𝑗,𝑘,𝑙 𝐽𝑗,𝑘,𝑙+1 − 𝐽𝑗,𝑘,𝑙 , 𝐽 = 3D matrix version of 𝑷.

Ground-truth image 2D slice of 3D image from TV Min.

Challenging to image the area!

𝑁 × 1 𝑁 × 𝑂 𝑂 × 1

Markov Random Field Modeling

• Model spatial dependencies
• Decision at each cell depends on

− Observed intensity from TV min. − Decision at neighboring cells

• MRF model:
• Using Hammersley-Clifford theorem:

11

𝑄 𝑌𝑗 = 𝑦𝑗 𝑌

𝑘 = 𝑦𝑘, ∀ 𝑘 ≠ 𝑗 ) = 𝑄(𝑌𝑗 = 𝑦𝑗 | 𝑌 𝑘 = 𝑦𝑘, ∀ 𝑘 ∈ 𝒪 𝑗 )

𝑄 𝒀 = 𝒚 𝒁 = 𝒛) ∝ exp −𝐹 𝒚, 𝒛 , 𝐹 𝒚, 𝒛 = ෍

𝑗

Φ𝑗 𝑦𝑗, 𝑧𝑗 + ෍

𝑗,𝑘 ∈Ɛ

Φ𝑗𝑘(𝑦𝑗, 𝑦𝑘)

Data cost Discontinuity cost Total cost

Neighborhood of node 𝑗

𝑌𝑗 − Binary label at 𝑗𝑢ℎ node 𝑍

𝑗 − TV min. intensity

• bserved at 𝑗𝑢ℎ node

Φ𝑗𝑘 𝑦𝑗, 𝑦𝑘 = 𝑦𝑗 − 𝑦𝑘

2

Φ𝑗 𝑦𝑗, 𝑧𝑗 = ൝ 1 − 𝑦𝑗 boundary 𝑦𝑗 − 𝑧𝑗 2 else Cost function definition

𝑌𝑗 ∈ 0, 1 , 0 ≤ 𝑍

𝑗 ≤ 1

Markov Random Field Modeling (cont.)

• Interested in finding solution 𝒀 = 𝒚 that maximizes
• Challenging: NP hard problem in loopy graphs
• Sum-product Loopy Belief Propagation

− Computes marginal probability at each cell 𝑄(𝑌𝑗 = 𝑦𝑗|𝒁 = 𝒛) − Estimated label: ො 𝑦𝑗 = arg max

𝑦𝑗

𝑄(𝑌𝑗 = 𝑦𝑗|𝒁 = 𝒛)

− Distributed approximated solution and computationally efficient

12

Ground-truth image

𝑄 𝒀 = 𝒚 𝒁 = 𝒛) ∝ exp −𝐹 𝒚, 𝒛

2D slice of 3D image from TV Min. 2D slice of 3D image - MRF and LBP

Outline

13

• Proposed 3D Through-wall Imaging Pipeline

− System Modeling and Sparse Signal Processing − Markov Random Field Modeling and Belief Propagation

• Route Design for UAVs: Robotic Paths in 3D
• Experimental Testbed: UAVs and Google

Tangos

• 3D Imaging Experimental Results
• Conclusions

UAV Path Planning - Challenges

14

Example 3D scenario

3 example x-z plane cross-sections at different y’s

• Different planes contain different information about the area
• Need to design routes that capture all possible information

UAV Path Planning – Route Design

15

Top view Front view Top view

• Parallel routes (angles: 0°, 90°, 45° and 135°)
• 4 angles for diverse perspectives of area

Front view

• Horizontal routes capture x-y variations at
• ne height
• Sloped routes capture variations in z with

small number of measurements X1.5 Speed

X1.5 Speed

Outline

16

• Proposed 3D Through-wall Imaging Pipeline

− System Modeling and Sparse Signal Processing − Markov Random Field Modeling and Belief Propagation

• Route Design for UAVs: Robotic Paths in 3D
• Experimental Testbed: UAVs and Google

Tangos

• 3D Imaging Experimental Results
• Conclusions

Experimental Testbed

Requirements:

• 3D path planning
• Accurate localization
• Coordination between UAVs
• RSSI measurements

17

Experimental Testbed (cont.)

18

GOOGLE TANGO GOOGLE TANGO

Wireless comm. link Wireless comm. link Wireless comm. link WiFi router

• n TX UAV

Raspberry Pi

• n RX UAV

TX UAV RX UAV TX Tango RX Tango Remote PC

Experimental Testbed (cont.)

19

Route Control WiFi RSSI

• Google Tango
• Cameras and IR sensors
• Uses features of environment
• Real-time accurate positioning
• Streams position information to

UAV for route control

• RMSE of localization: 0.067 m
• Way-points: position goals
• TX UAV: WiFi router
• RX UAV:

− Raspberry Pi − WLAN card

• Location-stamped RSSI

measurements

Not yet. Wait for me Reached way-point 3?

Localization Coordination

Outline

20

• Proposed 3D Through-wall Imaging Pipeline

− System Modeling and Sparse Signal Processing − Markov Random Field Modeling and Belief Propagation

• Route Design for UAVs: Robotic Paths in 3D
• Experimental Testbed: UAVs and Google

Tangos

• 3D Imaging Experimental Results
• Conclusions

Experimental Scenario: Two-Cube Area

21

3D Through-Wall Imaging: Experimental Results

22

Two-Cube Area

• 3D high-quality imaging with WiFi
• Empty and occupied places imaged well
• Accurate localization of center of top block
• Variation along z is captured in reconstruction

Experimental Scenario: L-shape Area

23

3D Through-Wall Imaging: Experimental Results

24

L-shape Area

• 3D high-quality imaging with WiFi
• Empty and occupied places imaged well
• Accurate localization of center of top block
• Variation along z is captured in reconstruction

Conclusions

• 3D Through-Wall Imaging

− Aerial vehicles and WiFi − Sparsity, MRF modeling and Loopy Belief Propagation for binary map of area

• Efficient Informative Route Design for UAVs
• Experimental Testbed

− Google Tangos for localization and coordination − Motion planning and control

• Experimental Results

− Accurate through-wall imaging of 3D objects using very few measurements

25

26

Thank you!

This work is funded by NSF CCSS award # 1611254