Data Fusion in Sensor Networks Hugh Durrant-Whyte ARC Federation - - PowerPoint PPT Presentation

data fusion in sensor networks
SMART_READER_LITE
LIVE PREVIEW

Data Fusion in Sensor Networks Hugh Durrant-Whyte ARC Federation - - PowerPoint PPT Presentation

Data Fusion in Sensor Networks Hugh Durrant-Whyte ARC Federation Fellow, Research Director ARC Centre of Excellence for Autonomous Systems The University of Sydney, Australia hugh@cas.edu.au ARC=Australian version of NSF Slide 1 IPAM Sensor


slide-1
SLIDE 1

Slide 1 IPAM Sensor Networks Jan. 2007

Data Fusion in Sensor Networks

Hugh Durrant-Whyte ARC Federation Fellow, Research Director ARC Centre of Excellence for Autonomous Systems The University of Sydney, Australia hugh@cas.edu.au

ARC=Australian version of NSF

slide-2
SLIDE 2

Slide 2 IPAM Sensor Networks Jan. 2007

Data Fusion in Autonomous Networks

  • Decentralised Data Fusion (DDF)
  • The DDF paradigm
  • Decentralised Bayes and information fusion
  • Fusion Challenges (ANSER II)
  • Learning mixed sensor features
  • Building general density models
  • Control Challenges (RCS-18)
  • Sensor and communication management
  • Information in system design
  • Future Challenges
slide-3
SLIDE 3

Slide 3 IPAM Sensor Networks Jan. 2007

Decentralised Data Fusion (DDF)

  • A set of Network Data Fusion

Methods:

  • Ad Hoc Network
  • Fusion at Sensor/Platform
  • No Central Fusion Site
  • Fully Scalable
  • Decentralised Algorithms

Using the Information Filter:

  • For Target Tracking
  • For Cooperative Control
  • For Cooperative Navigation

Target states (different bandwidths) Sensor Fusion Sensor Fusion Sensor Fusion To other network components Sensor Fusion

slide-4
SLIDE 4

Slide 4 IPAM Sensor Networks Jan. 2007

Bayes and Information Fusion

Bayes:

( ) ( ) ( )

=

j j k k

k k z P k P C k P ) ( | ) ( | ) ( . | ) (

1

x Z x Z x

Posterior Prior Sensor Likelihoods

  • x(k) is the state at time k
  • zj(k) is the jth sensor observation at time k
  • Zk is the sequence of observations up to k

)) ( ( )) ( | ) ( ( k k k z P

j j

x x Λ =

Sensors contribute Likelihoods on x(k)

slide-5
SLIDE 5

Slide 5 IPAM Sensor Networks Jan. 2007

Data Fusion: Distributed Sensing

...... ......

X

z1 zn zi Fusion Centre P(zi|x)

Sensor i

P(z1|x)

Sensor 1

P(zn|x)

Sensor n

Λ1(x) Λi(x) Λn(x)

P(x|Zn)=C P(x) Π Λi(x) n

i=1

P(x)

slide-6
SLIDE 6

Slide 6 IPAM Sensor Networks Jan. 2007

Data Fusion: Distributed Fusion

......

X

z1 zn Fusion Centre P(z1|x)

Sensor 1

P(xk|z1) P1(xk|z) P(zn|x)

Sensor n

P(xk|zn) Pn(xk|z) P(xk|Zn) P(xk|Zn)=C P(xk-1) Π n

i=1

Pi(xk|z) P(xk-1|Zn) k k-1

slide-7
SLIDE 7

Slide 7 IPAM Sensor Networks Jan. 2007

Data Fusion: Decentralised Fusion

......

X

z1 zn P(z1|x)

Sensor 1

P(xk|z1) P1(xk|z) P(zn|x)

Sensor n

P(xk|zn) Pn(xk|z) P(xk|Zn) Fusion Centre P(xk|Zn)=C P(xk-1) Π n

i=1

Pi(xk|z) P(xk-1|Zn) k k-1 Fusion Centre P(xk|Zn)=C P(xk-1) Π n

i=1

Pi(xk|z) P(xk-1|Zn) k k-1 P(xk|Zn)

slide-8
SLIDE 8

Slide 8 IPAM Sensor Networks Jan. 2007

Bayes and Information Fusion

Bayes:

( ) ( ) ( )

=

j j k k

k k z P k P C k P ) ( | ) ( | ) ( . | ) (

1

x Z x Z x

Log-Likelihood:

( ) ( ) ( )

K k k z P k P k P

j j k k

+ + =

) ( | ) ( ln | ) ( ln | ) ( ln

1

x Z x Z x

Information, by construction, is additive

[ ]

) ( log ) ( x x P E H − =

Information: A measure of compactness:

⎥ ⎦ ⎤ ⎢ ⎣ ⎡ − = ) ( ) | ( log ) : ( x z x z x P P E I

Mutual Information: a priori measure

  • f contribution to compaction:
slide-9
SLIDE 9

Slide 9 IPAM Sensor Networks Jan. 2007

Bayes and Information Fusion

Bayes:

( ) ( ) ( )

=

j j k k

k k z P k P C k P ) ( | ) ( | ) ( . | ) (

1

x Z x Z x

Log-Likelihood:

( ) ( ) ( )

K k k z P k P k P

j j k k

+ + =

) ( | ) ( ln | ) ( ln | ) ( ln

1

x Z x Z x

Information, by construction, is additive Gaussian Case: (Fisher or Canonical) Information Form

) | ( ) | (

1

k k k k

= P Y

j j T j j k

H R H I

1

) (

=

) | ( ˆ ) | ( ) | ( ˆ

1

k k k k k k x P y

=

) ( ) (

1

k k

j j T j j

z R H i

=

slide-10
SLIDE 10

Slide 10 IPAM Sensor Networks Jan. 2007

The Information Filter

Observation updates are simple sums (unlike KF):

+ − =

j j k

k k k k ) ( ) 1 | ( ˆ ) | ( ˆ i y y

+ − =

j j k

k k k k ) ( ) 1 | ( ) | ( I Y Y

[ ]

) ( ) | ( ) | ( ˆ ) | ( ˆ ) | 1 ( ˆ k k k k k k k k k Bu Y y Ω y y + + = +

) ( ) ( ) ( ) | ( ) | 1 ( k k k k k k k

T

Ω Σ Ω Y Y − = +

Time/Structure updates are Dual to state (KF) Observation Updates :

slide-11
SLIDE 11

Slide 11 IPAM Sensor Networks Jan. 2007

DDF in Operation

  • Network communicates Information
  • Nodes fuse local observation and

communicated Information

  • Channels communicate local

information gain (mutual information)

Sensor Fusion Sensor Fusion Sensor Fusion To other network components

Σ Σ

i(k) yi(k|k) yi(k|k-1) yi(k|k) ~

∆ypj(k|k)

Channel Filters Prediction Preprocess

∆ypi(k|k)

Channel Manager

Sensor Node

∆yqj(k|k)

slide-12
SLIDE 12

Slide 12 IPAM Sensor Networks Jan. 2007

ANSERII: All-Source Bayesian Fusion

  • Full Bayes DDF for fusion of heterogeneous data from UAVs,

UGVs, human and data base sources

  • Model general feature types; trees, buildings, dams, etc
  • Identify and label features, integrate human inferences
  • Real-time exploitation of network data by air, ground, human
slide-13
SLIDE 13

Slide 13 IPAM Sensor Networks Jan. 2007

Bayesian DDF Node Structure

zi(k)

Time Update (Convolution) Preprocess and Feature Extraction

Sensor Node P Q

Observation Update (Multiplication) Channel Filter Channel Filter Density Fitting Likelihood Model Channel Manager Assimilation (Multiplication)

Pi(z|x) Pi(z=zi(k)|x) P(xk|Zk-1,zi(k)) P(xk|Zk-1) P(xk|Zk-1,zi(k)) P(xk|Zk) P(xk|zQ,zP)

slide-14
SLIDE 14

Slide 14 IPAM Sensor Networks Jan. 2007

Learning and Inference Process

slide-15
SLIDE 15

Slide 15 IPAM Sensor Networks Jan. 2007

Step I: Model Generation

  • NLDR on image patches
  • Find low-D representation for

significant patches of interest

  • Use VBEM to find number and

centre for mixture model

s z x

( ) ( ) ( ) ( )

s s x s x z s x z P P P P | , | , , =

slide-16
SLIDE 16

Slide 16 IPAM Sensor Networks Jan. 2007

VBEM to Estimate Classifications

  • Need to estimate number

and location of classes

  • Variational Bayes

Procedures:

  • Simultaneous

estimation of class number and centre

  • Can deal with order 105

state dimensions

  • Unsupervised/semi-

supervised classification

  • f sensory data
  • Automatic generation of

Likelihood Function

slide-17
SLIDE 17

Slide 17 IPAM Sensor Networks Jan. 2007

Resulting Location Likelihood Models

ANSER 2

  • Mixture Model for Location

Parameters and Likelihoods

slide-18
SLIDE 18

Slide 18 IPAM Sensor Networks Jan. 2007

Resulting Class Likelihood Models

( ) ( ) ( )

s s x s x z P P P | , |

slide-19
SLIDE 19

IPAM Sensor Networks Jan. 2007

Step II: Model Inference (Air)

P(z | x,s)=P(z|x=[x,s]) is the required likelihood for inference

slide-20
SLIDE 20

Slide 20 IPAM Sensor Networks Jan. 2007

Step II: Model Inference (Ground)

P(z | x,s)=P(z|x=[x,s]) is the required likelihood for inference

slide-21
SLIDE 21

Slide 21 IPAM Sensor Networks Jan. 2007

ANSER II Also Uses Data-Base and Human-derived Information

  • Human operator input:
  • Metric Information
  • Labels, Context
  • On-line estimation of

“operator likelihood”

  • Hyperspectral data “node”
slide-22
SLIDE 22

Slide 22 IPAM Sensor Networks Jan. 2007

Component-Based Middleware for Deployment (ORCA)

slide-23
SLIDE 23

Slide 23 IPAM Sensor Networks Jan. 2007

Mission System Implementation

System Video

slide-24
SLIDE 24

Slide 24 IPAM Sensor Networks Jan. 2007

ANSER II Common Operating Picture

slide-25
SLIDE 25

Slide 25 IPAM Sensor Networks Jan. 2007

Fusion: Future Challenges

  • Feature Modelling: Finding “x”
  • Generalising single-sensor feature models
  • Learning mixed-modality feature models
  • Learning and refining abstractions (MI?)
  • Probabilistic Fusion: Finding “P”
  • High-dimensional density estimation
  • Find a general and efficient density family,

closed under fusion operations

  • Data association with general densities
  • Systems design: Finding “z”
  • Use information to do system design,

assembly and reconfiguration

slide-26
SLIDE 26

Slide 26 IPAM Sensor Networks Jan. 2007

Mutual Information & Control

  • Mutual Information or information gain, is

exactly what is communicated in the DDF

  • Can be exploited in sensor management,

communications and platform control

Σ Σ

i(k) yi(k|k) yi(k|k-1) yi(k|k) ~

∆ypj(k|k)

Channel Filters Prediction Preprocess

∆ypi(k|k)

Channel Manager

Sensor Node

∆yqj(k|k)

slide-27
SLIDE 27

Slide 27 IPAM Sensor Networks Jan. 2007

Example Cooperative Control

  • Inherits DDF

properties:

  • Scalability
  • Survivability
  • The trajectory that

maximises information

  • Information shared (DDF)
slide-28
SLIDE 28

Slide 28 IPAM Sensor Networks Jan. 2007

RCS-18: Future Cooperative UAVs

  • How best to use tactical UAV

fleets ?

  • A list of candidate targets of

interest

  • Coordinate a UAV fleet with

mixed sensors to:

  • Locate,
  • identify and
  • prosecute targets
  • Demonstrate this
slide-29
SLIDE 29

Slide 29 IPAM Sensor Networks Jan. 2007

Set-up

  • DDF Enabled on all platforms
  • Mutual information on target

location and IDs

  • A set of UAV manoeuvres:
  • Point-to-Point
  • Orbits
  • K-step look-ahead

e

Orbit s

R d V

ϕ ϕ

ω ϕ =

I I

slide-30
SLIDE 30

Slide 30 IPAM Sensor Networks Jan. 2007

Targets

slide-31
SLIDE 31

Slide 31 IPAM Sensor Networks Jan. 2007

Two Vehicle Demonstration

slide-32
SLIDE 32

Slide 32 IPAM Sensor Networks Jan. 2007

Extending the Paradigm

  • Other information-maximising

controllers

  • Resource use (platform,

communications)

  • Target cuing, hand-off, etc
  • Search, Exploration
slide-33
SLIDE 33

Slide 33 IPAM Sensor Networks Jan. 2007

Control: Future Challenges

  • Dealing with Constraints
  • Process constraints
  • Time constraints (rendezvous)
  • Cooperative Planning
  • Heterogeneous platforms:

Who does what and when

  • Re-tasking
slide-34
SLIDE 34

Slide 34 IPAM Sensor Networks Jan. 2007

Future Challenges

  • Theory
  • Spatial Scales
  • Temporal Scales
  • The autonomous sensor
  • The application
  • Mine picture compilation
  • Large-scale sub-sea surveys
slide-35
SLIDE 35

Slide 35 IPAM Sensor Networks Jan. 2007

Mine Picture Compilation

Multi-Spectral Geophysical Airborne Laser Face Radar

slide-36
SLIDE 36

Slide 36 IPAM Sensor Networks Jan. 2007

Large-Scale Sub-Sea Surveys

Acoustic Visual Laser Chemical Temperature