A method of personal positioning based on sensor data fusion of wearable camera and self-contained sensors. *1 *2 *1 Makasakatsu Kourogi and Takeshi Kurata *1 National Institute of Advanced Industrial Science and Technology (AIST) *2 HIT Lab., University of Washington
Purpose Personal positioning for wearable PC user � Without external source of information such as IR beacon or ultrasonic receivers. � Light-weight equipments without using expensive highly specialized devices such as Inertial Measurement Unit (IMU).
Requirements What are required for personal positioning? � Direction of the user ’ s head (camera) � A wearable camera is attached to the head. � Direction of the user ’ s body � The direction to which the user moves. � Position of the user
Proposed method Attitude sensors Wearable camera (head tracker) Dead-reckoning module Heading direction based on walking Source of information Accelerometers (3-axis) Gravitational direction Gyro-sensors (3-axis) � A wearable camera Magnetometers (3-axis) Inclinometers (3-axis) � A 3-DOF attitude sensor (head) � A 3-DOF attitude sensor (torso) � Walking distance measurement unit � Walking behavior is recognized and measured. � A kind of a smart pedometer.
Proposed method: head direction A head tracker (InterTrax2) is used to acquire the direction of user ’ s head. � Incremental error is accumulated. � Direction initialization is required. A problem to be solved
Proposed method: Position and head direction Image taken from the camera is used to acquire the position and head direction. � Image is matched with pre-stored image database of which position and direction of capturing is precisely known. Position, direction initialized Image registration “ Not matched ” Wearable camera No initialization occurs Image database
Proposed method: body direction A self-contained sensors are used to acquire the body direction. � Gyro-sensors (3-axis) � Incrementally accumulate errors. � Magnetometers (3-axis) � Enable us to get absolute reference. � Accelerometers (3-axis) � Used to detect and measure walking behavior. These sensors are attached to the user ’ s torso.
Problems to be solved Earth ’ s magnetic field (EMF) � The magnetic field is easily disturbed by electronic devices and appliances. � Thus, it is difficult to use magnetometers to reliably acquire the absolute direction.
Proposed method: EMF How to estimate reliability of the magnetic field? � Dip angle should be constant at given longitude and latitude. Horizontal direction Dip Earth’s magnetic field (EMF) Gravitational direction
Proposed method: EMF How to estimate reliability of the magnetic field? � We take a strategy to see if the EMF ’ s dip angle is continuously correct while in motion to test the reliability of the EMF. EMF can be reliable if its dip angle is correct while the user is walking. Walking behavior can be detected by accelerometers.
Proposed method: EMF Sequence of dip angle while in walking � Computed from actual output of magnetometers 70 Data range when the magnetic field is reliable 60 True dip angle 50 Angle [deg] 40 30 20 10 0 0 5000 10000 15000 20000 Time [msec]
Proposed method: EMF What if EMF seems unreliable? � Gyro-sensors are used to compute the direction by accumulation of attitude.
Proposed method: relative position Walking detection � A typical pattern of forward and vertical acceleration while in walking behavior. 0.6 Vertical acceleration 0.5 Forward acceleration 0.4 0.3 0.2 Acceleration [G] 0.1 0 -0.1 -0.2 -0.3 -0.4 -0.5 10000 10500 11000 11500 12000 12500 13000 13500 14000 Time [msec] Actual output of accelerometers
Proposed method: relative position Dead-reckoning is used. � Position movement is decided with relative position with dead-reckoning of combination of pedometer and the body direction.
Proposed method: data integration How to integrate sensory data? � Kalman filter framework is used. � Sensor data are fed into the framework and statistically consistent results are acquired on each discrete time.
Proposed method Kalman filter framework buffer Image registration Image module Attitude angle (head) DB Update Accumulation Angular velocity (torso) State vector Generate Magnetic vector measurement Test Covariance matrix reliability Measurement Gravitational vector Input Kalman filter Acceleration vector Output update loop Walking distance Update Update gain Covariance matrix estimation module covariance of measurement error
Proposed method Kalman filter: state vector and measurement vector ⎡ ˆ ⎤ x ⎡ ⎤ x k k Position ⎢ ⎥ ⎢ ⎥ ˆ y y ⎢ ⎥ ⎢ ⎥ k k ⎢ ⎥ α ˆ ⎢ ⎥ α k ⎢ ⎥ k ⎢ ⎥ ˆ β β ⎢ ⎥ Attitude (head) ⎢ ⎥ = k ˆ y = k x ⎢ ⎥ k γ ⎢ ⎥ ˆ k γ k ⎢ ⎥ k ⎢ ⎥ ψ ˆ ψ ⎢ ⎥ ⎢ ⎥ k k ⎢ ⎥ ˆ ⎢ ⎥ θ θ Attitude (torso) ⎢ ⎥ k ⎢ ⎥ k ˆ φ ⎢ ⎥ φ ⎣ ⎦ ⎢ ⎥ ⎣ ⎦ k k
Proposed method ψ ⎡ cos ⎤ Kalman filter: equations of state vector k ⎢ ⎥ ψ sin ⎢ ⎥ Walking distance (as input vector) k Prediction: ⎢ ⎥ 0 ⎢ ⎥ = + ⋅ 0 x x D ( x ) u ⎢ ⎥ = + D ( x ) k 1 | k k | k k | k k where ⎢ ⎥ k | k 0 ⎢ ⎥ Update: 0 ⎢ ⎥ ⎢ ⎥ 0 = + − ˆ x x K ( y x ) ⎢ ⎥ − − k | k k | k 1 k k k | k 1 ⎢ 0 ⎥ ⎣ ⎦ Kalman gain: ˆ ˆ − = Σ Σ + Σ 1 K ( ) − − k k | k 1 k | k 1 vk Covariance matrix of measurement noise
Proposed method Kalman filter: equations of covariance matrix Prediction: ˆ ˆ Σ + = Σ + Σ k 1 | k k | k wk Covariance matrix of process noise Update: ˆ ˆ ˆ Σ = Σ − Σ K − − k | k k | k 1 k | k 1
Experimental results: Positioning Position of resetting by image registration Starting position End position
Applications Personal navigation Annotation overlay � Annotative information is overlaid with input video frame to wearable display.
Implementation System diagram IEEE802.11b Database Wireless LAN Database client server VGA MicroOptical ClipOn Display Access points USB InterTrax2 Software Ethernet Web CCD camera Image server Server RS-232C 3DM-G Wearable PC
Experimental results: Positioning and annotative information � Demo video:
Conclusions: EMF can be reliably used to estimate the direction. Positioning is achieved by the proposed method. The method can be used to overlay annotative information on input frame.
Recommend
More recommend
