SLIDE 1
SHALINI GUPTA, PAVLO MOLCHANOV, KIHWAN KIM, KARI PULLI, JAN KAUTZ NVIDIA RESEARCH
GESTURE RECOGNITION: USING A MULTI SENSOR APPROACH SHALINI GUPTA, - - PowerPoint PPT Presentation
GESTURE RECOGNITION: USING A MULTI SENSOR APPROACH SHALINI GUPTA, - - PowerPoint PPT Presentation
GESTURE RECOGNITION: USING A MULTI SENSOR APPROACH SHALINI GUPTA, PAVLO MOLCHANOV, KIHWAN KIM, KARI PULLI, JAN KAUTZ NVIDIA RESEARCH DRIVER DISTRACTION GESTURE INTERFACE (http://www.softkinetic.com) DAY DAY NIGHT NO SUNLIGHT NO SUNLIGHT
SLIDE 2
SLIDE 3
GESTURE INTERFACE
(http://www.softkinetic.com)
SLIDE 4
DAY
SLIDE 5
DAY NIGHT
SLIDE 6
NO SUNLIGHT
SLIDE 7
NO SUNLIGHT SUNLIGHT
SLIDE 8
sensors
COLOR + DEPTH RADAR
MULTI-SENSOR SOLUTION
SLIDE 9
gesture UI sensors
COLOR + DEPTH RADAR
MULTI-SENSOR SOLUTION
SLIDE 10
3D shape color velocity
3.2% INCREASED ACCURACY
+1.5m/s
- 1.5m/s
0 m/s
SLIDE 11
v t 0.15W velocity power
16X POWER EFFICIENCY
SLIDE 12
v t 0.15W vT gesture gesture velocity power 2.5 W
16X POWER EFFICIENCY
SLIDE 13
radar prototype
SHORT RANGE FMCW RADAR
SLIDE 14
x y z v radar prototype 4D vector
SHORT RANGE FMCW RADAR
+1.5m/s
- 1.5m/s
0 m/s
SLIDE 15
POSITION RESULTS
SLIDE 16
VELOCITY RESULTS
+1.5m/s
- 1.5m/s
0 m/s
SLIDE 17
GESTURE NETWORK
3D convolutional layer fully connected NN logistic regression subsampling layer 60 frames Trained on GPU
SLIDE 18
10 GESTURES
PALM SWIPE SHAKE CALL left right up down left right clockwise counter-clockwise ROTATION
SLIDE 19
INDOOR CAR SIMULATOR
SLIDE 20
OUTDOOR CAR
SLIDE 21
ERROR RATE
C D R D+C R+D R+C R+D+C 39.90% 9.10% 10.90%
D – depth C – color R - radar
SLIDE 22
ERROR RATE
C D R D+C R+D R+C R+D+C 39.90% 9.10% 10.90% 7.90% 8.30% 7.40%
D – depth C – color R - radar
SLIDE 23
ERROR RATE
C D R D+C R+D R+C R+D+C 39.90% 9.10% 10.90% 7.90% 8.30% 7.40% 5.90%
D – depth C – color R - radar
SLIDE 24
ERROR RATE
D – depth C – color R - radar
Night Evening Day (shadow) Day (sunlight)
6.70% 3.00% 9.70% 20.90%
D+R (CNN) D+R+C (CNN) D+C (HOG)
SLIDE 25
ERROR RATE
D – depth C – color R - radar
Night Evening Day (shadow) Day (sunlight)
6.70% 3.00% 9.70% 20.90% 6.70% 1.50% 8.30% 7.50%
D+R (CNN) D+R+C (CNN) D+C (HOG)
SLIDE 26
ERROR RATE
D – depth C – color R - radar
Night Evening Day (shadow) Day (sunlight)
6.70% 1.50% 8.30% 7.50% 22.20% 2.45% 13.00% 20.90%
D+R (CNN) D+R+C (CNN) D+C (HOG*)
*Ohn-Bar and Trivedi, IEEE Trans. on
Intelligent Transportation Systems, 2014.
SLIDE 27
52ms on Quadro 6000
DEMO
SLIDE 28
CONCLUSION
GESTURE UI2 COLOR + DEPTH RADAR1
1Multi-sensor System for Driver’s Hand-Gesture Recognition, IEEE Automatic Face and Gesture Recognition, May 2015. 2Short-Range FMCW Monopulse Radar for Hand-Gesture Sensing, IEEE International Radar Conference, May 2015.
SLIDE 29
THANK YOU
SHALINIG@NVIDIA.COM
SLIDE 30