Minimum Cost Data Aggregation with Localized Processing for - - PowerPoint PPT Presentation

Minimum Cost Data Aggregation with Localized Processing for Statistical Inference

• A. Anandkumar1
• L. Tong1
• A. Swami2
• A. Ephremides3

1ECE Dept., Cornell University, Ithaca, NY 14853 2Army Research Laboratory, Adelphi MD 20783 3EE Dept., University of Maryland College Park, MD 20742

IEEE INFOCOM 2008 .

Supported by Army Research Laboratory CTA Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 1 / 21

Distributed Statistical Inference

Sensor Network Applications

Detection Estimation

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 2 / 21

Distributed Statistical Inference

Sensor Network Applications

Detection Estimation

Fusion Center

Yn = [Y1, . . . , Yn]

Classical Distributed Inference

Sensors: take measurements Fusion Center: Final decision Statistical Model

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 2 / 21

Routing for Inference

Raw Data: Yn

Fusion Center Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 3 / 21

Routing for Inference

Raw Data: Yn

Fusion Center

20 40 60 80 100 120 140 160 180

Raw (SPT) Independent (MST)

Number of nodes n Total cost

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 3 / 21

Routing for Inference

Raw Data: Yn

Fusion Center

20 40 60 80 100 120 140 160 180

Raw (SPT) Independent (MST)

Number of nodes n Total cost

20 40 60 80 100 120 140 160 180

Raw (SPT) Independent (MST)

Number of nodes n

• Avg. cost

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 3 / 21

Routing for Inference

Raw Data: Yn

Fusion Center

20 40 60 80 100 120 140 160 180

Raw (SPT) Independent (MST)

Number of nodes n Total cost

20 40 60 80 100 120 140 160 180

Raw (SPT) Independent (MST)

Number of nodes n

• Avg. cost

Sufficient Statistics for Mean Estimation Y1, . . . , Yn

i.i.d.

∼ N(θ, 1)

• i Yi sufficient to estimate θ: no performance loss

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 3 / 21

Routing for Inference

Raw Data: Yn

Fusion Center

Sum Function

Fusion Center

20 40 60 80 100 120 140 160 180

Raw (SPT) Independent (MST)

Number of nodes n Total cost

20 40 60 80 100 120 140 160 180

Raw (SPT) Independent (MST)

Number of nodes n

• Avg. cost

Sufficient Statistics for Mean Estimation Y1, . . . , Yn

i.i.d.

∼ N(θ, 1)

• i Yi sufficient to estimate θ: no performance loss

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 3 / 21

Routing for Inference

Raw Data: Yn

Fusion Center

Sum Function

Fusion Center

20 40 60 80 100 120 140 160 180

Raw (SPT) Independent (MST)

Number of nodes n Total cost

20 40 60 80 100 120 140 160 180

Raw (SPT) Independent (MST)

Number of nodes n

• Avg. cost

Sufficient Statistics for Mean Estimation Y1, . . . , Yn

i.i.d.

∼ N(θ, 1)

• i Yi sufficient to estimate θ: no performance loss

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 3 / 21

Routing for Inference

Raw Data: Yn

Fusion Center

Sum Function

Fusion Center

20 40 60 80 100 120 140 160 180

Raw (SPT) Independent (MST)

Number of nodes n Total cost

20 40 60 80 100 120 140 160 180

Raw (SPT) Independent (MST)

Number of nodes n

• Avg. cost

Sufficient Statistics for Mean Estimation Y1, . . . , Yn

i.i.d.

∼ N(θ, 1)

• i Yi sufficient to estimate θ: no performance loss

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 3 / 21

Routing for Inference

Raw Data: Yn

Fusion Center

Sum Function

Fusion Center

20 40 60 80 100 120 140 160 180

Raw (SPT) Independent (MST)

Number of nodes n Total cost

20 40 60 80 100 120 140 160 180

Raw (SPT) Independent (MST)

Number of nodes n

• Avg. cost

Sufficient Statistics for Mean Estimation Y1, . . . , Yn

i.i.d.

∼ N(θ, 1)

• i Yi sufficient to estimate θ: no performance loss

Binary Hypothesis Test: Y1, . . . , Yn

i.i.d.

∼ f(y; H0) or f(y; H1)

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 3 / 21

Routing for Inference

Raw Data: Yn

Fusion Center

Sum Function

Fusion Center

20 40 60 80 100 120 140 160 180

Raw (SPT) Independent (MST)

Number of nodes n Total cost

20 40 60 80 100 120 140 160 180

Raw (SPT) Independent (MST)

Number of nodes n

• Avg. cost

Sufficient Statistics for Mean Estimation Y1, . . . , Yn

i.i.d.

∼ N(θ, 1)

• i Yi sufficient to estimate θ: no performance loss

Binary Hypothesis Test: Y1, . . . , Yn

i.i.d.

∼ f(y; H0) or f(y; H1) [

i log f(Yi; H0), i log f(Yi; H1)] sufficient to decide hypothesis

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 3 / 21

Routing for Inference

Raw Data: Yn

Fusion Center

Sum Function

Fusion Center

20 40 60 80 100 120 140 160 180

Raw (SPT) Independent (MST)

Number of nodes n Total cost

20 40 60 80 100 120 140 160 180

Raw (SPT) Independent (MST)

Number of nodes n

• Avg. cost

Sufficient Statistics for Mean Estimation Y1, . . . , Yn

i.i.d.

∼ N(θ, 1)

• i Yi sufficient to estimate θ: no performance loss

Binary Hypothesis Test: Y1, . . . , Yn

i.i.d.

∼ f(y; H0) or f(y; H1) [

i log f(Yi; H0), i log f(Yi; H1)] sufficient to decide hypothesis

LLR=

i log f(Yi;H1) log f(Yi;H1) minimally sufficient to decide hypothesis

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 3 / 21

Minimum Cost In-Network Processing for Inference

Minimal Sufficient Statistic for Binary Hypothesis Testing

Log Likelihood Ratio: LLR(YV ) = log f(YV ; H1) f(YV ; H0)

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 4 / 21

Minimum Cost In-Network Processing for Inference

Minimal Sufficient Statistic for Binary Hypothesis Testing

Log Likelihood Ratio: LLR(YV ) = log f(YV ; H1) f(YV ; H0)

?

LLR(YV ) = log f(YV ;H1)

log f(YV ;H0)

Fusion Center

Extent of Processing?

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 4 / 21

Minimum Cost In-Network Processing for Inference

Minimal Sufficient Statistic for Binary Hypothesis Testing

Log Likelihood Ratio: LLR(YV ) = log f(YV ; H1) f(YV ; H0)

?

LLR(YV ) = log f(YV ;H1)

log f(YV ;H0)

Fusion Center

Extent of Processing? Fusion Scheme?

Minimum Cost Data Fusion for Inference

Min total costs s.t. LLR is delivered to fusion center

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 4 / 21

Minimum Cost In-Network Processing for Inference

Minimal Sufficient Statistic for Binary Hypothesis Testing

Log Likelihood Ratio: LLR(YV ) = log f(YV ; H1) f(YV ; H0)

?

LLR(YV ) = log f(YV ;H1)

log f(YV ;H0)

Fusion Center

Extent of Processing? Fusion Scheme?

Minimum Cost Data Fusion for Inference

Min total costs s.t. LLR is delivered to fusion center

Spatial Correlation Model: Should Capture Full Correlation Range

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 4 / 21

Minimum Cost In-Network Processing for Inference

Minimal Sufficient Statistic for Binary Hypothesis Testing

Log Likelihood Ratio: LLR(YV ) = log f(YV ; H1) f(YV ; H0)

?

LLR(YV ) = log f(YV ;H1)

log f(YV ;H0)

Fusion Center

Extent of Processing? Fusion Scheme?

Minimum Cost Data Fusion for Inference

Min total costs s.t. LLR is delivered to fusion center

Spatial Correlation Model: Should Capture Full Correlation Range Markov random field with dependency graph

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 4 / 21

Minimum Cost In-Network Processing for Inference

Minimal Sufficient Statistic for Binary Hypothesis Testing

Log Likelihood Ratio: LLR(YV ) = log f(YV ; H1) f(YV ; H0)

?

LLR(YV ) = log f(YV ;H1)

log f(YV ;H0)

Fusion Center

Extent of Processing? Fusion Scheme?

Minimum Cost Data Fusion for Inference

Min total costs s.t. LLR is delivered to fusion center

Spatial Correlation Model: Should Capture Full Correlation Range Markov random field with dependency graph Structured LLR: sum over dependency graph cliques

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 4 / 21

Minimum Cost In-Network Processing for Inference

Minimal Sufficient Statistic for Binary Hypothesis Testing

Log Likelihood Ratio: LLR(YV ) = log f(YV ; H1) f(YV ; H0)

?

LLR(YV ) = log f(YV ;H1)

log f(YV ;H0)

Fusion Center

Extent of Processing? Fusion Scheme?

Minimum Cost Data Fusion for Inference

Min total costs s.t. LLR is delivered to fusion center

Spatial Correlation Model: Should Capture Full Correlation Range Markov random field with dependency graph Structured LLR: sum over dependency graph cliques Local processing of clique data

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 4 / 21

Summary of Results

Minimum Cost Data Fusion for Inference

Min total routing costs s.t. likelihood ratio is delivered to fusion center

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 5 / 21

Summary of Results

Minimum Cost Data Fusion for Inference

Min total routing costs s.t. likelihood ratio is delivered to fusion center

AggMST: MST-based Heuristic

Separation of local processor selection and aggregation Approximation Ratio of 2 for Nearest-Neighbor Dependency

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 5 / 21

Summary of Results

Minimum Cost Data Fusion for Inference

Min total routing costs s.t. likelihood ratio is delivered to fusion center

AggMST: MST-based Heuristic

Separation of local processor selection and aggregation Approximation Ratio of 2 for Nearest-Neighbor Dependency

Steiner Tree Reduction

Joint design of local processors and aggregation Optimal Cost is given by Steiner tree on expanded graph Approximation-factor preserving reduction: best known is 1.55

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 5 / 21

Summary of Results

Minimum Cost Data Fusion for Inference

Min total routing costs s.t. likelihood ratio is delivered to fusion center

AggMST: MST-based Heuristic

Separation of local processor selection and aggregation Approximation Ratio of 2 for Nearest-Neighbor Dependency

Steiner Tree Reduction

Joint design of local processors and aggregation Optimal Cost is given by Steiner tree on expanded graph Approximation-factor preserving reduction: best known is 1.55

Constant Average Cost Scaling (Allerton ‘07)

k-NNG Dependency in Random Large Constant Density Networks

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 5 / 21

Outline

1

Introduction

2

Markov Random Field

3

Minimum Cost Fusion

4

Heuristics and Approximations

5

Conclusion

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 6 / 21

Outline

1

Introduction

2

Markov Random Field

3

Minimum Cost Fusion

4

Heuristics and Approximations

5

Conclusion

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 7 / 21

Markov Random Field (MRF)

Dependency Graph

A A C B

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 8 / 21

Markov Random Field (MRF)

Dependency Graph YA ⊥ YB|YC

B C A A

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 8 / 21

Markov Random Field (MRF)

Dependency Graph YA ⊥ YB|YC

B C A A

Hammersley-Clifford Theorem ‘71 Log-Likelihood: sum of potentials of maximal cliques of dependency graph − log f(YV ; Gd) =

• c∈C

Ψc(Yc)

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 8 / 21

Markov Random Field (MRF)

Dependency Graph YA ⊥ YB|YC

B C A A

Hammersley-Clifford Theorem ‘71 Log-Likelihood: sum of potentials of maximal cliques of dependency graph − log f(YV ; Gd) =

• c∈C

Ψc(Yc) Independent: Cliques=Nodes − log f(YV ; Gd) =

i∈V

Ψi(Yi)

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 8 / 21

Markov Random Field (MRF)

Dependency Graph YA ⊥ YB|YC

B C A A

Hammersley-Clifford Theorem ‘71 Log-Likelihood: sum of potentials of maximal cliques of dependency graph − log f(YV ; Gd) =

• c∈C

Ψc(Yc) Independent: Cliques=Nodes − log f(YV ; Gd) =

i∈V

Ψi(Yi) Chain Dependency Graph Psi(3,4 logf(Y;G) Psi(1,2 Psi(2,3

1 2 3 4

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 8 / 21

Markov Random Field (MRF)

Dependency Graph YA ⊥ YB|YC

B C A A

Hammersley-Clifford Theorem ‘71 Log-Likelihood: sum of potentials of maximal cliques of dependency graph − log f(YV ; Gd) =

• c∈C

Ψc(Yc) Independent: Cliques=Nodes − log f(YV ; Gd) =

i∈V

Ψi(Yi) Chain Dependency Graph Psi(3, Psi(1, logf(Y; Psi(2,

1 2 3 4

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 8 / 21

Markov Random Field (MRF)

Dependency Graph YA ⊥ YB|YC

B C A A

Hammersley-Clifford Theorem ‘71 Log-Likelihood: sum of potentials of maximal cliques of dependency graph − log f(YV ; Gd) =

• c∈C

Ψc(Yc) Independent: Cliques=Nodes − log f(YV ; Gd) =

i∈V

Ψi(Yi) Chain Dependency Graph Psi(1, Psi(2,3 Psi(3,

− log f(YV ; Gd) = 1 2 3 4

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 8 / 21

Markov Random Field (MRF)

Dependency Graph YA ⊥ YB|YC

B C A A

Hammersley-Clifford Theorem ‘71 Log-Likelihood: sum of potentials of maximal cliques of dependency graph − log f(YV ; Gd) =

• c∈C

Ψc(Yc) Independent: Cliques=Nodes − log f(YV ; Gd) =

i∈V

Ψi(Yi) Chain Dependency Graph Psi(2, Psi(3,

− log f(YV ; Gd) = Ψ1,2(Y1, Y2)+ 1 2 3 4

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 8 / 21

Markov Random Field (MRF)

Dependency Graph YA ⊥ YB|YC

B C A A

Hammersley-Clifford Theorem ‘71 Log-Likelihood: sum of potentials of maximal cliques of dependency graph − log f(YV ; Gd) =

• c∈C

Ψc(Yc) Independent: Cliques=Nodes − log f(YV ; Gd) =

i∈V

Ψi(Yi) Chain Dependency Graph Psi(3,

− log f(YV ; Gd) = Ψ1,2(Y1, Y2)+ Ψ2,3(Y2, Y3)+ 1 2 3 4

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 8 / 21

Markov Random Field (MRF)

Dependency Graph YA ⊥ YB|YC

B C A A

Hammersley-Clifford Theorem ‘71 Log-Likelihood: sum of potentials of maximal cliques of dependency graph − log f(YV ; Gd) =

• c∈C

Ψc(Yc) Independent: Cliques=Nodes − log f(YV ; Gd) =

i∈V

Ψi(Yi) Chain Dependency Graph

− log f(YV ; Gd) = Ψ1,2(Y1, Y2)+ Ψ2,3(Y2, Y3)+ Ψ3,4(Y3, Y4) 1 2 3 4

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 8 / 21

Binary Hypothesis Testing of MRFs

Null Hypothesis H0 : {G0(V ), C0, Ψ0} Ψ0 Alternative Hypothesis H1 : {G1(V ), C1, Ψ1} Ψ1

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 9 / 21

Binary Hypothesis Testing of MRFs

Null Hypothesis H0 : {G0(V ), C0, Ψ0} Ψ0 Alternative Hypothesis H1 : {G1(V ), C1, Ψ1} Ψ1

Minimal Sufficient Statistic for Binary Hypothesis Testing

Log Likelihood Ratio: LLR(YV ) = log f(YV ; H1) f(YV ; H0)

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 9 / 21

Binary Hypothesis Testing of MRFs

Null Hypothesis H0 : {G0(V ), C0, Ψ0} Ψ0 Alternative Hypothesis H1 : {G1(V ), C1, Ψ1} Ψ1 Effective MRF For LLR {G0(V )∪G1(V ), max(C0∪C1), Φ} Φ

Minimal Sufficient Statistic for Binary Hypothesis Testing

Log Likelihood Ratio: LLR(YV ) = log f(YV ; H1) f(YV ; H0)

LLR in MRF= Log-Likelihood of Effective MRF

LLR(YV ) =

c∈C

Φc(Yc)

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 9 / 21

Outline

1

Introduction

2

Markov Random Field

3

Minimum Cost Fusion

4

Heuristics and Approximations

5

Conclusion

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 10 / 21

Minimum Cost Fusion for Inference

Problem Statement

Minimize sum routing costs s.t. LLR(Yn) =

c∈C

Φc(Yc) is delivered

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 11 / 21

Minimum Cost Fusion for Inference

Problem Statement

Minimize sum routing costs s.t. LLR(Yn) =

c∈C

Φc(Yc) is delivered

Network & Communication Model

Connected Network, Bidirectional Links, Unicast Mode

• Comm. Graph with Link Costs

Cost C Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 11 / 21

Minimum Cost Fusion for Inference

Problem Statement

Minimize sum routing costs s.t. LLR(Yn) =

c∈C

Φc(Yc) is delivered

Network & Communication Model

Connected Network, Bidirectional Links, Unicast Mode

• Comm. Graph with Link Costs

Cost C Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 11 / 21

Minimum Cost Fusion for Inference

Problem Statement

Minimize sum routing costs s.t. LLR(Yn) =

c∈C

Φc(Yc) is delivered

Network & Communication Model

Connected Network, Bidirectional Links, Unicast Mode

• Comm. Graph with Link Costs

Cost C

Cliques of Dependency Graph

c4 c2 c3

c1

• Min. Cost Fusion

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 11 / 21

Stages of LLR Computation: LLR(Yn) =

c∈C

Φc(Yc)

c4 c2 c3 c1

Forwarding graph Dependency graph Aggregation graph Processor Fusion center

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 12 / 21

Stages of LLR Computation: LLR(Yn) =

c∈C

Φc(Yc)

c4 c2 c3 c1

Forwarding graph Dependency graph Aggregation graph Processor Fusion center

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 12 / 21

Stages of LLR Computation: LLR(Yn) =

c∈C

Φc(Yc)

Raw Data: Yi

Forwarding graph Dependency graph Aggregation graph Processor Fusion center

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 12 / 21

Stages of LLR Computation: LLR(Yn) =

c∈C

Φc(Yc)

Φ(c1) + Φ(c2)

Φ(c3) Φ(c4)

Forwarding graph Dependency graph Aggregation graph Processor Fusion center

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 12 / 21

Stages of LLR Computation: LLR(Yn) =

c∈C

Φc(Yc)

+

3

• i=1

Φ(ci) LLR =

4

• i=1

Φ(ci)

Forwarding graph Dependency graph Aggregation graph Processor Fusion center

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 12 / 21

Stages of LLR Computation: LLR(Yn) =

c∈C

Φc(Yc)

Forwarding graph Dependency graph Aggregation graph Processor Fusion center

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 12 / 21

Stages of LLR Computation: LLR(Yn) =

c∈C

Φc(Yc)

Forwarding graph Dependency graph Aggregation graph Processor Fusion center

Local Computation of Clique Potentials: Processor is a Clique Member

Simplifies optimization problem Local knowledge of function parameters

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 12 / 21

Stages of LLR Computation: LLR(Yn) =

c∈C

Φc(Yc)

Forwarding graph Dependency graph Aggregation graph Processor Fusion center

Local Computation of Clique Potentials: Processor is a Clique Member

Simplifies optimization problem Local knowledge of function parameters

c1

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 12 / 21

Stages of LLR Computation: LLR(Yn) =

c∈C

Φc(Yc)

Forwarding graph Dependency graph Aggregation graph Processor Fusion center

Local Computation of Clique Potentials: Processor is a Clique Member

Simplifies optimization problem Local knowledge of function parameters

c1

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 12 / 21

Stages of LLR Computation: LLR(Yn) =

c∈C

Φc(Yc)

Forwarding graph Dependency graph Aggregation graph Processor Fusion center

Local Computation of Clique Potentials: Processor is a Clique Member

Simplifies optimization problem Local knowledge of function parameters

c1

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 12 / 21

Stages of LLR Computation: LLR(Yn) =

c∈C

Φc(Yc)

Forwarding graph Dependency graph Aggregation graph Processor Fusion center

Local Computation of Clique Potentials: Processor is a Clique Member

Simplifies optimization problem Local knowledge of function parameters

c1

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 12 / 21

Outline

1

Introduction

2

Markov Random Field

3

Minimum Cost Fusion

4

Heuristics and Approximations

5

Conclusion

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 13 / 21

Lower Bound of MST & AggMST Heuristic

Minimum Spanning Tree: Lower Bound for Min Routing Cost

Independent data: achieves bound ⇒ Correlation increases cost

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 14 / 21

Lower Bound of MST & AggMST Heuristic

Minimum Spanning Tree: Lower Bound for Min Routing Cost

Independent data: achieves bound ⇒ Correlation increases cost

AggMST Heuristic

c4 c2 c3

c1

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 14 / 21

Lower Bound of MST & AggMST Heuristic

Minimum Spanning Tree: Lower Bound for Min Routing Cost

Independent data: achieves bound ⇒ Correlation increases cost

AggMST Heuristic

Processor Assignment: Any clique member c4 c2 c3

c1

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 14 / 21

Lower Bound of MST & AggMST Heuristic

Minimum Spanning Tree: Lower Bound for Min Routing Cost

Independent data: achieves bound ⇒ Correlation increases cost

AggMST Heuristic

Processor Assignment: Any clique member Forwarding: Other members to processor

Raw Data: Yi

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 14 / 21

Lower Bound of MST & AggMST Heuristic

Minimum Spanning Tree: Lower Bound for Min Routing Cost

Independent data: achieves bound ⇒ Correlation increases cost

AggMST Heuristic

Processor Assignment: Any clique member Forwarding: Other members to processor

Φ(c1) + Φ(c2) Φ(c3) Φ(c4)

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 14 / 21

Lower Bound of MST & AggMST Heuristic

Minimum Spanning Tree: Lower Bound for Min Routing Cost

Independent data: achieves bound ⇒ Correlation increases cost

AggMST Heuristic

Processor Assignment: Any clique member Forwarding: Other members to processor Aggregation: MST, towards fusion center +

LLR No useful data

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 14 / 21

Lower Bound of MST & AggMST Heuristic

Minimum Spanning Tree: Lower Bound for Min Routing Cost

Independent data: achieves bound ⇒ Correlation increases cost

AggMST Heuristic

Processor Assignment: Any clique member Forwarding: Other members to processor Aggregation: MST, towards fusion center +

LLR No useful data

Approximation Algorithm with Ratio ρ

Routing cost no worse than ρ times optimal, runs in polynomial time

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 14 / 21

Lower Bound of MST & AggMST Heuristic

Minimum Spanning Tree: Lower Bound for Min Routing Cost

Independent data: achieves bound ⇒ Correlation increases cost

AggMST Heuristic

Processor Assignment: Any clique member Forwarding: Other members to processor Aggregation: MST, towards fusion center +

LLR No useful data

Approximation Algorithm with Ratio ρ

Routing cost no worse than ρ times optimal, runs in polynomial time

Approximation Ratio of AggMST = 2 for Nearest-Neighbor Graph

C(AggMST) C(G∗) ≤ C(AggMST) C(MST) ≤ 2

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 14 / 21

Simulation Results

20 40 60 80 100 120 140 160 180 2 4 6 8 10 12

Number of nodes n

• Avg. cost per node

NNG dependency 3-NNG dependency 2-NNG dependency Shortest path Lower Bound (MST)

Implications

Scalable in network size for k-NNG dependency Fusion cost sensitive to No. of cliques in dependency graph

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 15 / 21

Simulation Results

20 40 60 80 100 120 140 160 180 2 4 6 8 10 12

Number of nodes n

• Avg. cost per node

NNG dependency 3-NNG dependency 2-NNG dependency Shortest path Lower Bound (MST)

20 40 60 80 100 120 140 160 180 2 4 6 8 10 12

Number of nodes n

• Avg. cost per node

NNG dependency 3-NNG dependency 2-NNG dependency Shortest path Lower Bound (MST)

Implications

Scalable in network size for k-NNG dependency Fusion cost sensitive to No. of cliques in dependency graph

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 15 / 21

Simulation Results

20 40 60 80 100 120 140 160 180 2 4 6 8 10 12

Number of nodes n

• Avg. cost per node

NNG dependency 3-NNG dependency 2-NNG dependency Shortest path Lower Bound (MST)

20 40 60 80 100 120 140 160 180 2 4 6 8 10 12

Number of nodes n

• Avg. cost per node

NNG dependency 3-NNG dependency 2-NNG dependency Shortest path Lower Bound (MST)

20 40 60 80 100 120 140 160 180 2 4 6 8 10 12

Number of nodes n

• Avg. cost per node

NNG dependency 3-NNG dependency 2-NNG dependency Shortest path Lower Bound (MST)

Implications

Scalable in network size for k-NNG dependency Fusion cost sensitive to No. of cliques in dependency graph

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 15 / 21

Simulation Results

20 40 60 80 100 120 140 160 180 2 4 6 8 10 12

Number of nodes n

• Avg. cost per node

NNG dependency 3-NNG dependency 2-NNG dependency Shortest path Lower Bound (MST)

20 40 60 80 100 120 140 160 180 2 4 6 8 10 12

Number of nodes n

• Avg. cost per node

NNG dependency 3-NNG dependency 2-NNG dependency Shortest path Lower Bound (MST)

20 40 60 80 100 120 140 160 180 2 4 6 8 10 12

Number of nodes n

• Avg. cost per node

NNG dependency 3-NNG dependency 2-NNG dependency Shortest path Lower Bound (MST)

20 40 60 80 100 120 140 160 180 2 4 6 8 10 12

Number of nodes n

• Avg. cost per node

NNG dependency 3-NNG dependency 2-NNG dependency Shortest path Lower Bound (MST)

Implications

Scalable in network size for k-NNG dependency Fusion cost sensitive to No. of cliques in dependency graph

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 15 / 21

Simulation Results

20 40 60 80 100 120 140 160 180 2 4 6 8 10 12

Number of nodes n

• Avg. cost per node

NNG dependency 3-NNG dependency 2-NNG dependency Shortest path Lower Bound (MST)

20 40 60 80 100 120 140 160 180 2 4 6 8 10 12

Number of nodes n

• Avg. cost per node

NNG dependency 3-NNG dependency 2-NNG dependency Shortest path Lower Bound (MST)

20 40 60 80 100 120 140 160 180 2 4 6 8 10 12

Number of nodes n

• Avg. cost per node

NNG dependency 3-NNG dependency 2-NNG dependency Shortest path Lower Bound (MST)

20 40 60 80 100 120 140 160 180 2 4 6 8 10 12

Number of nodes n

• Avg. cost per node

NNG dependency 3-NNG dependency 2-NNG dependency Shortest path Lower Bound (MST)

20 40 60 80 100 120 140 160 180 2 4 6 8 10 12

Number of nodes n

• Avg. cost per node

NNG dependency 3-NNG dependency 2-NNG dependency Shortest path Lower Bound (MST)

Implications

Scalable in network size for k-NNG dependency Fusion cost sensitive to No. of cliques in dependency graph

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 15 / 21

Steiner-Tree Reduction

Steiner Tree

Minimum cost tree containing a required set of nodes called terminals NP-hard problem, currently the best approximation is 1.55

Terminals

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 16 / 21

Steiner-Tree Reduction

Steiner Tree

Minimum cost tree containing a required set of nodes called terminals NP-hard problem, currently the best approximation is 1.55

Terminals

Main result

Min cost fusion has approx. ratio preserving Steiner tree reduction

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 16 / 21

Example : Chain dependency graph

1 2 3 4

Fusion Center Graph transformation and building Steiner tree.

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 17 / 21

Example : Chain dependency graph

1 2 3 4

Fusion Center Graph transformation and building Steiner tree.

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 17 / 21

Example : Chain dependency graph

1 2 3 4

Fusion Center v12 v23 v34 Graph transformation and building Steiner tree.

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 17 / 21

Example : Chain dependency graph

1 2 3 4

Fusion Center v12 v23 v34 Graph transformation and building Steiner tree.

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 17 / 21

Example : Chain dependency graph

1 2 3 4

Fusion Center v12 v23 v34 Graph transformation and building Steiner tree.

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 17 / 21

Example : Chain dependency graph

1 2 3 4

Fusion Center v12 v23 v34 Graph transformation and building Steiner tree.

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 17 / 21

Example : Chain dependency graph

1 2 3 4

Fusion Center v12 v23 v34 Graph transformation and building Steiner tree.

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 17 / 21

Example : Chain dependency graph

1 2 3 4

Fusion Center v12 v23 v34 Graph transformation and building Steiner tree.

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 17 / 21

Example : Chain dependency graph

Proc(12) Proc(23) Proc(34)

1 2 3 4

v12 v23 v34 Graph transformation and building Steiner tree.

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 17 / 21

Example : Chain dependency graph

FG FG FG

1 2 3 4

v12 v23 v34 Graph transformation and building Steiner tree.

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 17 / 21

Example : Chain dependency graph

FG FG FG AG AG

1 2 3 4

Graph transformation and building Steiner tree.

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 17 / 21

Example : Chain dependency graph

1 2 3 4

Y1 Y2 Y3 Graph transformation and building Steiner tree.

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 17 / 21

Example : Chain dependency graph

1 2 3 4

Φ(Y1, Y2) Graph transformation and building Steiner tree.

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 17 / 21

Example : Chain dependency graph

1 2 3 4

Φ(Y1, Y2) + Φ(Y2, Y3) Graph transformation and building Steiner tree.

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 17 / 21

Example : Chain dependency graph

1 2 3 4

LLR

Graph transformation and building Steiner tree.

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 17 / 21

Outline

1

Introduction

2

Markov Random Field

3

Minimum Cost Fusion

4

Heuristics and Approximations

5

Conclusion

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 18 / 21

Conclusion

Summary

Minimum cost data fusion for inference Concept of dependency graph based routing

◮ Exploit correlation structure to fuse data efficiently

Proposed MST-based heuristic: AggMST

◮ 2-approximation for NNG, simple construction

Steiner-tree reduction

◮ 1.55-approximation, Use of Steiner tree algorithms Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 19 / 21

Conclusion

Summary

Minimum cost data fusion for inference Concept of dependency graph based routing

◮ Exploit correlation structure to fuse data efficiently

Proposed MST-based heuristic: AggMST

◮ 2-approximation for NNG, simple construction

Steiner-tree reduction

◮ 1.55-approximation, Use of Steiner tree algorithms

Outlook

Incorporating physical layer issues

◮ Effect of interference, Broadcast nature of wireless medium

Spatial probability approach for large random networks (Allerton 07) Tradeoff between routing costs and inference performance

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 19 / 21

Previous Works on Correlated Data Routing

Spatial Correlation Models: All incorporated under MRF framework

Joint-Gaussian, distance based correlation (Marco et al. 03, Yoon & Shahabi 07) Joint entropy (Pattem et al. 04), spl. MRF (Jindal & Psounis 06)

Correlated Data Gathering (Cristescu et al. 06, Scaglione&Servetto 02)

Raw data not needed at fusion center, only the likelihood function for optimal inference

In-network Function Computation (Giridar & Kumar 06)

Valid for symmetric functions, likelihood function may not have this form

Routing for Inference: For Special Correlation Models

Independent Measurements: (Yang & Blum 07, Yu & Ephremides 06) 1-D Gauss-Markov process: (Sung et al. 06, Chamberland & Veeravalli 06)

Routing for Belief Propagation (Kreidl & Willsky 06, Williams et al. 05)

Local MAP estimate of raw data at each node: not global decision at fusion center

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 20 / 21

Thank You !

Anandkumar, Tong, Swami, Ephremides Minimum Cost Fusion for Inference IEEE INFOCOM ‘08 21 / 21