SLIDE 1 A PODS-based Extended Kalman Filter: Quantifying Sensing Uncertainties in Automatic Bird Species Detection
Dezhen Song Associate Professor
- Dept. of Computer Science and Engineering,
Texas A&M University
IEEE ICRA Workshop on Uncertainty in Automation, May 9, 2011
SLIDE 2
Thanks to:
Ni Qin, Yiliang Xu, Chang Young Kim, Wen Li, TAMU Ken Goldberg, UC Berkeley Ron Rohrbach, Cornell Lab of Ornithology John Fitzpatrick, Cornell Lab of Ornithology David Luneau, U Arkansas Hopeng Wang and Jingtain Liu, Nankai University John Rappole, Smithsonian Selma Glasscock, Welder Wildlife Foundation National Science Foundation The Nature Conservancy Arkansas Game and Fish Commission U.S. Fish and Wildlife Service Arkansas Electric Cooperative Cache River National Wildlife Refuge
SLIDE 3
Biological observation is arduous, expensive, dangerous, lonely
SLIDE 4
Assisting the search for IBWO
SLIDE 5 Detecting Rare Birds
- Low occurrence (e.g., <10 times per year)
- Short duration (e.g., < 1 sec. in FOV)
- Huge video data for human identification.
- Setup and maintenance in remote environments.
SLIDE 6 Design Goals
– low false negative
– filtering the targeted bird
- Easy to setup and maintain
– monocular vision system
SLIDE 7 Related Work
– DeerCam – Africa web cams at the Tembe – Elephant part – Tiger web cams – James Reserve Wildlife Observatory – Crane Cam – Swan Cam
SLIDE 8 Related Work
- Motion detection and tracking
– Elgammal, Grimson, Isard …
- Periodic motion detection
– Culter, Ran, Briassouli …
- 3D inference using monocular vision
– Ribnick, Hoiem, Saxena …
SLIDE 9 Related Work
– SLAM, tracking, recognition … – Convergence
- ample observation data
- manageable noise
- less than 11 data points
- significant image noise
SLIDE 10 Bird detection problem
– targeted bird body length lb and speed range V=[vmin,vmax]. – a sequence of n images containing a moving object
– to determine if the object is a bird of targeted species
SLIDE 11 Assumptions
– High resolution – Narrow FOV
– Motion segmentation
– High flying speed – Narrow camera FOV
SLIDE 12
Observation 1: Invariant body length in Steady flight
SLIDE 13 Invariant body length in steady flight
[ut,vt]T [uh,vh]T z=[uh,vh,ut,vt]T (observation)
SLIDE 14 Bird Body Axis Filtering
- Observation 2: Body axis orientation close to tangent
line of trajectory during steady flight
Bird body axis Flying trajectory θ θ Difference between θ and θ on 61bird sequences:
0.8 ; 8.3
b b
µ σ = =
, ) ( , )
z=argmax , s.t. [ 2 , 2 ]
h h t t
b b u v B u v B
l θ θ σ θ σ
∈ ∈
∈ − +
B
SLIDE 15 Modeling A Flying Bird
camera center
Image plane
[x,y,z]T [ut,vt]T [uh,vh]T x z y Ptail Kinematics: Pin-hole model: Tail:
b t t t T tail b b
x xl x y z y yl z zl − = = − − / || v || P [ , , ] / || v || / || v || & & &
SLIDE 16 Extended Kalman Filter
camera center Image plane x(k+1) z(k) z(k+1) x z y x(k)
SLIDE 17 Determine Species for Noise-free Cases
camera center Image plane
Targeted range
False True
SLIDE 18 Estimation with Observation Noises
camera center Image plane
SLIDE 19 Probable Observation Data Set (PODS)
camera center Image plane
1( )
[ ( ) ]
h
S k u k τ = ±
2( )
[ ( ) ]
h
S k v k τ = ±
3( )
[ ( ) ]
t
S k u k τ = ±
4( )
[ ( ) ]
t
S k v k τ = ±
PODS:
1 1 1
and
n n n
k k ε δ = ∈ <
: : :
{Z | z( ) ( ) (X ) } Z S
Targeted range
1 2 3 4
k S k S k S k S k = × × × ( ) ( ) ( ) ( ) ( ) S
SLIDE 20
EKF Convergence Metrics
SLIDE 21 PODS-EKF
PODS:
1 1 1
and
n n n
k k ε δ = ∈ <
: : :
{Z | z( ) ( ) (X ) } Z S
Targeted range
Dezhen Song and Yiliang Yu, Low False Negative Filter for Detecting Rare Bird Species from Short Video Segments using a Probable Observation Data Set-based EKF Method, IEEE Transactions on Image Processing, vol. 19, no. 9, Sept. 2010,
1 1
Decision-making: 1 (accept) if and 0 (reject) otherwise
n n
I ∩ ≠ Φ ≠ Φ
: :
(Z ) = V Z V
SLIDE 22 PODS-EKF Approximate Computation
Targeted range
1 1 n k n k
ε
∈
=
( ) : z( : )
argmin (X Z ) %
S
Subject to:
1 1 1
and
n n n
k k ε δ = ∈ <
: : :
{Z | z( ) ( ) (X ) } Z S
Dezhen Song and Yiliang Yu, Low False Negative Filter for Detecting Rare Bird Species from Short Video Segments using a Probable Observation Data Set-based EKF Method, IEEE Transactions on Image Processing, vol. 19, no. 9, Sept. 2010,
SLIDE 23 Dezhen Song and Yiliang Yu, Low False Negative Filter for Detecting Rare Bird Species from Short Video Segments using a Probable Observation Data Set-based EKF Method, IEEE Transactions on Image Processing, vol. 19, no. 9, Sept. 2010,
SLIDE 24
Algorithm
SLIDE 25 Experiments
- Both simulated and real data
- A desktop PC with an Intel Core 2 Duo 2.13GHz CPU
and 2GB RAM
- Matlab 7.0 (motion detection) and Visual C++ 8.0
(PODS-EKF)
- Arecont AV3100 camera
- Bird species tested:
SLIDE 26
Convergence of different EFKs on Rock Pigeon
SLIDE 27
Simulation on three birds
SLIDE 28 Physical Experiment on Rock Pigeon
Insects, falling leaves,
SLIDE 29 ROC Curves for Rock Pigeon
Area under ROC curve: 91.5% in Simulation; 95.0% in Experiment.
SLIDE 30
SLIDE 31
SLIDE 32
SLIDE 33
SLIDE 34
SLIDE 35
SLIDE 36
SLIDE 37 Data reduction
- Oct. 2006 – Oct. 2007
- Motion detection: 29.41 TB to 27.42 GB
- PODS-EKF: 27.42 GB to 146.7 MB (~960 images)
- Overall reduction rate: 99.9995%
SLIDE 38
What we found
Pileated woodpecker (cousin of IBWO)
SLIDE 39
Northern flicker (smaller than IBWO)
SLIDE 40
Red-tailed Hawk (larger than IBWO)
SLIDE 41 Conclusion
- Low false negative bird filter: PODS-EKF
- Cope with insufficient noisy observation data
- 95% area under ROC curve
- 99.9995% data reduction
SLIDE 42 Current and Future Work
- Examine wing-flapping motion
– Wingbeat frequency is unique for each species
SLIDE 43
Wing Kinematic Model
SLIDE 44 Current & Future Work: AnyFish
Collaborators: Mr. Ji Zhang, Dr. Gil Rosenthal, and Dr. Wei Yan
SLIDE 45
Thanks! Websites: http://telerobot.cs.tamu.edu http://rbt.cs.tamu.edu/
SLIDE 46
SLIDE 47 Harmonic component Wingbeat frequency component Gliding component
Seagull: Mean 2.74 Hz S.D. 0.22 Hz
