Sensor Subsystem Pitfalls and Considerations Jen Costillo - PowerPoint PPT Presentation

Designing An Android Sensor Subsystem Pitfalls and Considerations Jen Costillo jen@rebelbot.com

Simple Choices User Battery experience performance 7/15/2012 Costillo- OSCON 2012 2

Established or Innovative Product? Established Innovation-Driven • Will I be making another • Do I have new sensors new product in 6 types? months? • Are features more • Is the reference design important than release considered good enough date? for the application? • Are money and resources no problem? 7/15/2012 Costillo- OSCON 2012 3

Forsaking Reference Designs 7/15/2012 Costillo- OSCON 2012 4

Going On Your Own • But… • If you make your own, – power ↓ – You’re on your own – Control code size – Integration pains – Control mechanical – Test time ↑ footprint – Gesture testing – In-house expertise becomes a challenge – Calibration blues – Larger mechanical footprint 7/15/2012 Costillo- OSCON 2012 5

Android Universe Application Android Application SensorManager Frameworks Sensor JNI Sensor Service Sensor Manager Libraries Sensor HAL Interface Kernel Linux Kernel Sensor Driver Driver Sensor Hub/ Hardware Sensors Coprocessor 7/15/2012 Costillo- OSCON 2012 6

Application Frameworks Libraries Linux Kernel HARDWARE Hardware 7/15/2012 Costillo- OSCON 2012 7

Hardware Architecture 7/15/2012 Costillo- OSCON 2012 8

Sampling Rates: The 3 Rates Under-sampling Over-sampling • Inaccurate, sluggish • Accurate, smooth response response • Slight power savings • Power-hungry Sampling Rate 7/15/2012 Costillo- OSCON 2012 9

Wake up events and power considerations Application Internal External Processor only Coprocessor Processor D C Reference supported Reference supported More processor selection Most power hungry Most work done for More outcome control you Most customized Footprint impact 7/15/2012 Costillo- OSCON 2012 10

Hardware Summary Power Latency = Consumption Max(sensors n ) Sensor = Σ sensors n + + dedicated Solution any dedicated processing processor time • Use tie-breaker criteria 7/15/2012 Costillo- OSCON 2012 11

Application Frameworks Libraries Linux Kernel KERNEL Hardware 7/15/2012 Costillo- OSCON 2012 12

Kernel Driver Application Processor Peripheral Shared Interface Memory Coprocessor Sensor Microcontroller 7/15/2012 Costillo- OSCON 2012 13

Application Frameworks Libraries Linux Kernel LIBRARIES AND SERVICES Hardware 7/15/2012 Costillo- OSCON 2012 14

Sensor HAL and Services • HAL device/<vendor>/<board name>/libsensors • Service frameworks/base/services/sensorservice • Manager frameworks/base/libs/gui 7/15/2012 Costillo- OSCON 2012 15

Sensor Fusion Libraries Linux Kernel Sensor Hub Sensors http://en.wikipedia.org/wiki/Sensor_fusion https://www.llnl.gov/news/newsreleases/2010/NR-10-01-06.html 7/15/2012 Costillo- OSCON 2012 16

Gesture Detection Algorithm Application Android Processor SensorService Co- Sensor Hub Processor Sensors MPU with Barometer Proximity Gyro/Accel Compass 7/15/2012 Costillo- OSCON 2012 17

Gesture Detection Comparison Make Buy Application • Powerful processor Minimal Off-load AppPro Schedule Impact In-house Already Tested expertise &tuned Complete Compact code Sensor hub solution • Off-load to cheaper power • Wake up Event Handling 7/15/2012 Costillo- OSCON 2012 18

Calibration Use of Calibration Gesture in the Compass App By Catch.com 7/15/2012 19 Costillo- OSCON 2012

Application Frameworks Libraries Linux Kernel FRAMEWORKS Hardware 7/15/2012 Costillo- OSCON 2012 20

Virtual Sensors • Leverages 1+ physical or other virtual sensors • Multiple Options – Google (version 3) – Reference Vendor – Sensor Vendors 7/15/2012 Costillo- OSCON 2012 21

Android Virtual Sensors Accel Gravity Linear Accel Gravity Accel 7/15/2012 Costillo- OSCON 2012 22

Android Virtual Sensors Gyro Accel Orientation Accel Rotation Compass 7/15/2012 Costillo- OSCON 2012 23

Virtual Sensors Challenges Sensor 1 • Garbage In- Garbage Out • Latency Sensor Virtual 2 Output • Non-Synchronized samples Sensor 1 • Implementation Virtual Dependencies 1 • Multi-vendor problems, Virtual Sensor 2 2 verify Vendor ID http://www.invensense.com/midc/presentations/James%20Lim.pdf 7/15/2012 Costillo- OSCON 2012 24

Application Frameworks Libraries Linux Kernel APPLICATIONS Hardware 7/15/2012 Costillo- OSCON 2012 25

Using Sensors mSensorManager = getSystemService (Context.SENSOR_SERVICE); mSensorManager.registerListener ( mSensorListener, mSensorManager. getDefaultSensor ( Sensor.TYPE_ACCELEROMETER), SensorManager.SENSOR_DELAY_NORMAL); void onSensorChanged (SensorEvent event) { // get sensor data float x = event.values[SensorManager.DATA_X]; } 7/15/2012 Costillo- OSCON 2012 26

Using Sensor (Continued) SensorManager. getRotationMatrix ( m_rotationMatrix, null, m_Mag, m_Accels); SensorManager. getOrientation ( m_rotationMatrix, m_orientation); float yaw_deg = m_orientation[0] * 57.2957795f; float pitch_deg = m_orientation[1] * 57.2957795f; float roll_deg = m_orientation[2] * 57.2957795f; 7/15/2012 Costillo- OSCON 2012 27

Types of Sensor Problems • Bias • Drift • Settling Time • Jitter/Noise • Environmental Interference 7/15/2012 Costillo- OSCON 2012 28

Bias • Problem: Data is off by a constant value. • Sources: – static calibration failure • Solutions: – Calculate linear offset at start of application – Recalibrate locally 7/15/2012 Costillo- OSCON 2012 29

Drift • Problem: Shift of data without cause • Sources: – Magnetic interference – Poor HW calibration • Solutions: – Increase smoothing techniques 7/15/2012 Costillo- OSCON 2012 30

Settling Time • Problem: Extended time before finalized steady data. • Sources: – Latency – Sensitivity • Solutions: – Limit additional processing 7/15/2012 Costillo- OSCON 2012 31

Noise • Problem: Data jumps around constantly • Sources: – Sensor – Calibration – Poor filtering • Solutions: – High pass filter – Linear averaging – FFT 7/15/2012 Costillo- OSCON 2012 32

Environmental Interference • Problem: Inconsistent results • Sources: – Magnetometer – EMI • Solutions: – Reference Device – Calibration gesture 7/15/2012 Costillo- OSCON 2012 33

Best Practices in Application Development • Select the right sensor for the job. • Use the Correct Data Rate. – UI or GAMING are the most common. • Use Sensor In Context • Customize for your hardware and system capabilities • Magnetometer-based sensors are the most touchy. • Keep the Gesture UI simple. 7/15/2012 Costillo- OSCON 2012 34

QUESTIONS? JEN@REBELBOT.COM Additional resources http://processors.wiki.ti.com/index.php/Android_Sensor_PortingGuide http://www.kandroid.org/online-pdk/guide/sensors.html http://invensense.com/midc/ http://developer.android.com/reference/android/hardware/SensorEvent.html 7/15/2012 Costillo- OSCON 2012 35