RF-Wear Wearable Everyday Body-Frame Tracking using Passive RFIDs - PowerPoint PPT Presentation

RF-Wear Wearable Everyday Body-Frame Tracking using Passive RFIDs Haojian Jin Zhijian Yang Swarun Kumar Jason Hong 1

RF-Wear turns a regular clothing into a body-frame aware garment using low-cost , light weight , machine washable , battery-free RFID tags. 2

Commercial Tracking Wearables 3

How do these devices track? + Pulse Sensor Pedometer (Accelerometer) 4

many times, we want more than heart rate and steps…. 5

Personal Trainer in Fitness 6

Gait Tracking in Rehabilitation 7

Gesture Input in VR/AR 8

how can we do body-frame today? 9

Optitrack 10

Infrastructure-based sensing Kinect Leap Motion Openpose (CMU) 11

Wearable Electronics inertial sensors Neuron 12

Smart fabrics Google jacquard [UIST 2016] 13

RF-Wear mobile, ad-hoc v.s. infrastructure solutions washable, durable, low cost v.s. wearable electronics continuous rich tracking v.s. smart fabrics (limited gestures) 14

RF-Wear skeleton tracking for daily use. using low-cost, machine washable, lightweight, battery-free RFIDs 15

d. Waist 135° 45° 75° 105° 80° 180° 96° a. Shoulder b. Knee c. Elbow e. Thigh RF-Wear: average joint angle tracking accuracy of 8~21°, 20~60 Hz 16

17

research contributions 1 A fj ne-grained mobile RFID tag positioning 2 A RFID sensing primitive for joint tracking 3 A practical body-worn RFID tag placement solution 4 A detailed prototype implementation and evaluation 18

background RFID sensing, phase measurement, triangulation 19

RFID Sensing Con fj guration RFID Tags RFID Antenna RFID Reader 20

RFID Backscatter Communication RFID Tags (Re fm ector) RFID Antenna (Transmitter) 21

RFID Backscatter Communication RFID Tags (Re fm ector) RFID Antenna (Transmitter) 22

d Phase Ranging resolution: θ T θ 1 LESS THAN 0.1 mm θ Tag θ 2 θ R λ Direct wave Backscatter wave Phase in Backscatter Communication 23

Tagoram [MOBICOM 2014] 24

Stationary RFID Sensing A 4 Static multiple antennas d 4 d 1 at known positions A 1 d 3 d 2 Use triangulation to calculate the tag position A 3 A 2 25

Mobile/Wearable Stationary RFID Sensing A 4 Static multiple antennas d 4 d 1 at known positions A 1 d 3 d 2 Use triangulation to calculate the tag position A 3 A 2 26

θ 1 θ 2 RF-Wear Key Primitives 27

θ 1 θ past work RF-Wear reversing the tag-antenna relationship 28

θ 6 5 1 4 3 2 measure the radio signal time-of-arrival delay 29

θ 6 5 1 4 3 2 the tag placement l is known 30

θ 1 the antenna is in the pocket the position may change when the user moves 31

θ 1 θ 2 θ 1 θ 2 knee joint angle = 32

ideally… in reality… multipath 33

Eigenspace method (MUSIC algorithm) 0 Incoming signal power distribution -5 across 0° to 180° θ = α -10 p( α ) -15 θ 6 5 1 4 3 2 -20 -25 10 30 50 70 90 110 130 150 170 α Broadside Angle (degrees) 34

Real-world Spectrum θ 1 Incoming signal power distribution across 0° to 180° 0 -10 p( α ) -20 -30 -40 10 30 50 70 90 110 130 150 170 α Broadside Angle (degrees) 35

Incoming signal power distribution across 0° to 180° 0 -10 θ 1 p( α ) -20 -30 θ 2 -40 0 68° -20 p( α ) -40 -60 10 30 50 70 90 110 130 150 170 α Broadside Angle (degrees) measure the o ff set of two spectrum to counter multipath signals 36

RF-Wear on Body 37

challenges on-body 1 2D sensing primitives to 3D space 2 Two Degree of Freedom Joints θ 3 Fabric fm exibility 38

implementation RFID tags, RFID readers, Software 39

RFID Tags 5.2 x 1.2 x 0.1 (cm) fm exible, washable 25Hz on the body (1m) 40

Software implemented in Python computation time: 0.03s => live demo (15 Hz) raw signal rate at 20~60 Hz continuous skeleton tracking Context: RapID [CHI’16] - 200 ms IDSense [CHI’15] - 2s discrete gesture recognition 41

evaluation 1) Array geometry 2) Fabric fm exibility 3) Motion capture experiment 42

microbenchmark 1m away on the fm oor facing the same direction 30 seconds/repetition 43

example: 2x4 6 tag array dimensions [2x3; 2x4; 2x5; 3x3; 4x4; 5x5] X 3 aperture [3cm, 4cm, 5cm] aperture: 5cm X 6 relative angles [30°, 60°, 90°, 120°, 150°, 180°] X 3 repetitions = 324 experiments repetitions 44

microbenchmark accuracy 20 azimuth elevation worst! 15 angle error (°) 10 best! 5 0 3x2_5cm 4x2_5cm 4x2_4cm 5x2_4cm 3x3_5cm 3x3_4cm 3x3_3cm 4x4_5cm 4x4_4cm 5x5_5cm 5x5_4cm 45

fabric fm exibility test 46

1 tag array con fj guration [2x4 with an aperture at 5 cm] X 3 fabrics [cotton, wool, polyester] X 6 relative angles [30°, 60°, 90°, 120°, 150°, 180°] X 3 repetitions = 54 experiments (30 sec each data collection) repetitions 47

fabric fm exibility test ) s e e r 8.69 g e context: d ( r cardboard: 4° o 6.56 r r 5.24 E y c a r u c c A Cotton pant Wool Sweater Jacket 48

motion capture 8 cameras on the ceiling sub-millimeter accuracy 49

knee elbow shoulder 50

walk in place (50s) + walk around (50s) 51

knee joint angle trace Walk in-place knee joint angle (°) Walk around time (sec) GroundTruth RF-Wear (KF) RF-Wear (Raw) 52

hand movement (160 sec) 53

elbow joint angle trace elbow joint angle (°) time (sec) GroundTruth RF-Wear (KF) RF-Wear (Raw) 54 54

shoulder rotation (3 x 20 sec) 55

shoulder joint angle trace shoulder joint angle (°) Horizontal DOF RF-Wear (Raw) RF-Wear (KF) GroundTruth Vertical DOF RF-Wear (Raw) RF-Wear (KF) GroundTruth time (sec) 56

Evaluation Summary If we use a tag array for 4X2 with an 5cm aperture, Card board accuracy: 4° On fabric: 6°-9° On body: knee 9° (walk in place), 12° (walk around). elbow 12°, shoulder (21° and 8°) Context (Kinect): knee joint angle accuracy in a gait cycle: 28.5° Accuracy of the Microsoft Kinect™ for measuring gait parameters during treadmill walking [Gait & Posture 2015] 57

discussion 58

number of tags? 64 on four limbs + 48 on the main body = 112 tags 59

on the fabric in the fabric on the body (tattoo) in the body (implant) 60

follow-up work 61

WiSh: Towards a Wireless Shape-aware World using Passive RFIDs (MobiSys’18)

conclusion 63

body-frame tracking for daily use turns a regular clothing into a body-frame aware garment using low-cost, light weight, machine washable, battery-free RFID tags tracks joint angle at 8~21°, 20~60 Hz RF-Wear 64

RF-Wear Wearable Everyday Body-Frame Tracking using Passive RFIDs Haojian Jin http://haojianj.in/ 65

Q & A 66

d θ T θ 1 θ Tag θ 2 θ R λ Direct wave Backscatter wave Phase in Backscatter Communication 67

The speed of radio in the air is 3x10^8 m/s. The 900 MHz radio will have 9x10^8 cycles in one second. The wavelength (the length of a cycle) would be 33 cm. The resolution of phase reading is 0.0015 radians. The distance resolution = = 0.0079 cm. LESS THAN 0.1 mm Phase to Super Resolution Distance 68

Mobile Reader (battery up to 8 hours)

Refresh rate Hardware limit reader: 1,100 tags/second. RFID tags backscatter frequency on body: 20 Hz. Software limit: MUSIC algorithm is computing expensive: 15 Hz. 70 https://www.atlasr fj dstore.com/impinj-speedway-revolution-r420-uhf-r fj d-reader-4-port/

Moving antenna Each angle computation was run independently based on one observation. we can do 30~60 Hz with commercial RFID readers given the reader moves at human speeds. 71

Context, accuracy of Microsoft Kinect knee in a gait cycle RMSD: 28.5° hip RMSD: 11.8° Accuracy of the Microsoft Kinect™ for measuring gait parameters during treadmill walking [Gait & Posture 2015] 72

Privacy (radio awareness) Traditional architecture: Stationary readers + Mobile Tags RFWear, WiSh Mobile readers + Mobile/Stationary Tags Users will have the control and awareness the reader status. 73

Body-frame v.s. skeleton RF-Wear tracks the body-frame by tracking the way clothes move as the body moves. Advantage: We can also track stomach spasms, belly movement. :) Limitation: RF-Wear can only track the joints covered by clothing. 74