SMU Classification: Restricted HDR-Nets, October 13, 2020 Collaborative Edge-based Machine Intelligence: Promise and Challenges Archan Misra Acknowledge the creative contributions of: PhD : Amit Sharma, Dulanga Weerakoon Post-Docs : Tran Huy Vu, Kasthuri Jayarajah, Manoj Gulati, Meera Radhakrishnan Post-doc & Engineers : Vengat Subramaniam, Dhanuja Wanniarachchige Collaborators : Vigneshwaran Subbaraju, Tarek Abdelzaher, Rajesh Balan
SMU Classification: Restricted My Research History 2022 2010 - 2015 2015 2014 - 2018 2018-2022 2010 2014 2018 Mobile Sensing & Analytics Wearable + IoT Systems • Indoor Location Wearable Sensing & • Batteryless Wearables • Group Detection Systems • Queuing Detection • Wireless/RFID Sensing • Eating ( Annapurna) • In-Store Shopping (IRIS, I4S) • Fine-grained Gestural • VR+ mobile (Empath-D) Tracking Key Research Thrusts • Fusion of multi-modal sensing Key Research Thrusts Key Research Thrusts (inertial) • Make Batteryless (or Utlra-Low • Optimize (Energy, Accuracy, • Adaptive sampling & triggered Power) Sensing possible Latency) tradeoffs sensing • Method: Utilize new sensing • Multi-modal sensor fusion • Multiple live deployments modalities ( video, wireless ) & (inertial, image) (campus, malls, museums) + collaborative ML at edge licensing
SMU Classification: Restricted W8-Scope: Exercise Monitoring using IoT Sensors Percom 2020 Goals : • Quantified insights on weight stack-based exercises provide personalized digital coaching Techniques: • Simple weight stack sensor (accelerometer+ magnetometer) to track & understand exercises Results: • Longitudinal Data Collection at 2 gyms 95+% accuracy & adaptation to medium-term evolutionary behavior 96 % 95 % 97 % 99 % Magnetic Sensor on Wt. Stack {Weight, Type, User}
SMU Classification: Restricted ERICA: Earable-based Real-Time Feedback for Free-weights Exercises Sensys 2020 Goals: • Associate User’s Earable with Dumbbell-mounted IoT sensors • Perform exercise recognition & real-time mistake detection • Provide “live” corrective feedback Feedback after every ~4 repetitions results in lower mistakes during set 4
SMU Classification: Restricted Some Lessons Learnt Pure Wearable/Mobile Sensing or Infrastructure Sensing isn’t Enough • Need to fuse inputs from personal and ambient sensors Computation vs. Communication Tradeoffs are Changing • Comms getting cheaper; computation more complex Source: doi: 10.1109/MIC.2018.011581520 Source: A. Canziani, A. Paszke, E. Culurciello, An Analysis of Deep Neural Network Models for Practical Applications ,, CoRR, May 2016
SMU Classification: Restricted Resource Bottlenecks & Trends 1. Where’s the Resource Bottleneck? 2. The Rise of the “Edge”
SMU Classification: Restricted This Talk: Summary of Collaborative Machine Intelligence (CMI) Collaboration is the Key to Realizing this Vision . Among: • Wearable devices & Edge infrastructure • Multiple IoT devices & Edge infrastructure DS: Distributed & Triggered Sensing Tightly coordinate Cheaper Expensive Sensor Triggering CMI: Collaborative ML-based Edge Intelligence Distribute Inferencing Pipelines across multiple pervasive devices & across modalities (Accuracy, Energy, Latency)
SMU Classification: Restricted RF/Wireless: A Swiss-Army Knife Energy Harvesting Sensing • Use Radio signal reflections to capture gestures • Multiple emerging modalities: light > • WiSee: Doppler Shifts Movement vibration > temperature > RF Frequency • Factors: size/form factor, on-body • Human Motion Artefacts position, intrusiveness. • WiBreathe: Breathing Rate • Doppler Shift • Object Composition • RFID Phase Shift Shape & Liquid Detector Ambient light Vibration Thermal gradient
SMU Classification: Restricted DS1. Battery Free Wearable/IoT Sensors Percom 2019 Access Point AP estimate AoA of “ping” Vision Device sends • Utilize battery-free sensors on AP transmits power packets “ping” packet wearables & IoT devices to provide Wearable operate on fine-grained tracking harvested energy, record • Key breakthrough : Charge devices sensor readings, transmit wirelessly via WiFi “power packet” data back at appropriate Data time transmissions Applications Device harvests and stores energy in a super capacitor • Activity Tracking of Workers & Moving Equipment Person with Wearables • Product Monitoring in Warehouses 20 Harvested Power (µW) • Elderly Monitoring in smart homes 15 10 Challenges • Low energy density using omnidirectional WiFi antenna 5 (< 1 m W at 1.5m) 0 • WiFi AP coordination to charge multiple devices 0.5 1 1.5 Distance (m)
The Wearable + AP System Power Management Motion Trigger • • The Harvester: Rectifier Matching Circuit Super Capacitor Storage Micro-controller Accelerometer RF Comm. SMU Classification: Restricted The Wearable AP Beamforming The 1 st WARP Pkt det • • • Raw Rx Buffer for DoA Tx Phase Sync for Beamforming Rx Buffer for AoA Ping detection (nRF24L01+) Ant A s Tx Phy & Tx Ant B h Mac syn i Ant C f Rx Phy & Mac Ant D t Sync Cable Sync Cable 2 nd WARP Pkt det Raw Rx Buffer for DoA Ant A s Tx Phy & Tx Ant B h Mac syn i Ant C f Rx Phy & Mac t Ant D
SMU Classification: Restricted The WiWear++: Low-power downlink (LPD) Under submission • Base version: Ping triggered by significant motion;No MAC • New: Use Wake-up Receivers to support low-power downlink (AP to device) • Proactive ping request (update orientation) • Content-free uplink trx START STOP LPUART compatible (1 START + 2 STOP bits) Raw OOK signal (From AP) Wakeup µController Receiver (LPUART) WiWear++ Prototype µController only wakes up to read LPUART data register
SMU Classification: Restricted The Cloud RAN & The Future of Multi-AP Operation • Lots of distributed transmitters (915/964 MHz • Harvesting Power levels drop with multiple wearable devices channels) surrounding the target. • Adjust phase distributed beamforming • 24 Trx (1.7W) in 20X20 m 2 0.6-0.7mW power harvested EnergyBall, Ubicomp’19 Power harvested (4 devices, 0.2m) • Future: What about multiple APs, that coordinate their transmissions? • Complex balance between sensing, communication and energy transfer capacity
SMU Classification: Restricted Takeaways & Reflections New opportunities : • Edge-Coordinated Activation of sensing on wearable devices. • Combination of passive RF sensing+ battery-less wearable/IoT devices • Edge ML needed to perform real- time multi-modal inferencing
SMU Classification: Restricted Collaborative MI: The Solution for Dependable Machine Intelligence Key Idea: Overcome limitations in resource & fidelity by performing machine intelligence jointly • Real-time decision making • Complex ML pipelines being executed on individual IoT devices or with edge-assistance • Key Resource & Performance Bottlenecks Latency of DNN execution 550 msec+ for person recognition/frame on a Movidius co-processor (1W) Low Accuracy Individual sensors subject to environmental artefacts Energy Overhead Need to support battery-less operations
SMU Classification: Restricted EA1: Collaborative IoT & The Edge: Ongoing Work • Collaborative Sensing A’s view is partly Spatial and/or occluded temporal overlap among sensors Sensor Multiplicity Learning from B can improve Adjust Inferencing A’s accuracy Pipeline on-the-fly Malicious C can purposefully • Dependable Systems perturb shared Resilience to inferences Adversarial Attacks
SMU Classification: Restricted Collaborative IoT & The Edge: Ongoing Work Design Goals 1. Requires NO re-training of the DNN models High Accuracy on complexity/very 2. Backward compatibility to non- Learned deep models collaborative mode when no Task Collaborative collaborators are available models 3. Minimal latency and bandwidth overhead for infusing collaborative input Low complexity/ deep models Latency Closing the accuracy gap with collaboration
SMU Classification: Restricted Approach #1: Run-Time Collaborative Inferencing (b) CSSD: Collaboration at Input Stage (a ) CNMS: Collaboration at Decision Stage Trusted Inferences Concat Concat Prior Prior Masks from N Reputation Masks Cameras Score Update from N Prior Masks from Neighboring Prior Masks from Neighboring Cameras Cameras Cameras High accuracy improvement with minimal latency PETS Dataset (8 cameras) • Person detection using SSD300; Homographic View mapping Inference Time Accuracy SSD Baseline 80ms 71% Collaborative 85ms 82.2% SSD 100ms 75.5% CNMS
SMU Classification: Restricted Approach #2.1: Adapting the ML Pipelines “On the Fly” for Improved Accuracy Exploration #1 4 Increase in TP% Increase in FP% Improving Accuracy through Sensor Multiplicity Gain in TP for nominal increase in FP Cam 1 3 Improved 2 Detections Detections 1 Class Confidence 0 boosting Detections S o N f Cam 2 M t Same person object, S m Convolutional perceptively clearer in a Layers the collaborator view x View 8+5 (33% overlap) View 8+6 (62%) View 8+7 (38%)
Recommend
More recommend
