Scalable Algorithms for Distributed Statistical Inference - - PowerPoint PPT Presentation

▶
scalable algorithms for distributed statistical inference
SMART_READER_LITE
LIVE PREVIEW

Scalable Algorithms for Distributed Statistical Inference - - PowerPoint PPT Presentation

Nov 25, 2022 751 likes •2.39k views

Scalable Algorithms for Distributed Statistical Inference Animashree Anandkumar School of Electrical and Computer Engineering Cornell University, Ithaca, NY 14853 Currently visiting EECS, MIT, Cambridge, MA 02139 PhD Committee: Lang Tong,

slide-1
SLIDE 1

Scalable Algorithms for Distributed Statistical Inference

Animashree Anandkumar

School of Electrical and Computer Engineering Cornell University, Ithaca, NY 14853 Currently visiting EECS, MIT, Cambridge, MA 02139 PhD Committee: Lang Tong, Aaron Wagner, Kevin Tang David Williamson, Ananthram Swami.

Supported by ARL-CTA, ARO, IBM PhD Fellowship

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 1 / 59

slide-2
SLIDE 2

Introduction

Internet PSTN

Traditional Wire-line Networks

Fixed networks Over-provisioned links Layered architecture

Emerging Networks

Large, complex, ubiquitous Resource constraints

◮ e.g., Energy, Bandwidth

Heterogeneous nodes Interaction between different networks

Sensor Networks

Decision Node Whitespace Spectrum Primary Secondary

Cognitive Networks Biological Networks Social Networks

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 2 / 59

slide-3
SLIDE 3

Network Data: Integrated View

Characteristics

Large number of samples, multi-modal Noisy, imperfect or missing data

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 3 / 59

slide-4
SLIDE 4

Network Data: Integrated View

Characteristics

Large number of samples, multi-modal Noisy, imperfect or missing data Data Locality: relationship between data at nearby nodes

◮ e.g., Temperature & other environmental data Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 3 / 59

slide-5
SLIDE 5

Network Data: Integrated View

Characteristics

Large number of samples, multi-modal Noisy, imperfect or missing data Data Locality: relationship between data at nearby nodes

◮ e.g., Temperature & other environmental data

Data to Knowledge: Specific Goals of Networks

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 3 / 59

slide-6
SLIDE 6

Network Data: Integrated View

Characteristics

Large number of samples, multi-modal Noisy, imperfect or missing data Data Locality: relationship between data at nearby nodes

◮ e.g., Temperature & other environmental data

Data to Knowledge: Specific Goals of Networks Distributed Statistical Inference

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 3 / 59

slide-7
SLIDE 7

Distributed Statistical Inference

Inference about a random population made from its samples

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 4 / 59

slide-8
SLIDE 8

Distributed Statistical Inference

Inference about a random population made from its samples

  • Node 1

C1 Cn Fusion center Node n

Classical Inference Quantization and inference rules Fixed configuration (one hop) Independent data at nodes

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 4 / 59

slide-9
SLIDE 9

Distributed Statistical Inference

Inference about a random population made from its samples

  • Node 1

C1 Cn Fusion center Node n

Classical Inference Quantization and inference rules Fixed configuration (one hop) Independent data at nodes

Fusion center

Wireless Sensor Networks for Inference Multihop data fusion Constraints on fusion costs Transmission and fusion policies Correlated data: local dependence

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 4 / 59

slide-10
SLIDE 10

Distributed Statistical Inference

Inference about a random population made from its samples

  • Node 1

C1 Cn Fusion center Node n

Classical Inference Quantization and inference rules Fixed configuration (one hop) Independent data at nodes

Fusion center

Wireless Sensor Networks for Inference Multihop data fusion Constraints on fusion costs Transmission and fusion policies Correlated data: local dependence Scaling of Fusion Costs & Inference Accuracy with Network Size

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 4 / 59

slide-11
SLIDE 11

Setup: Fusion of Sensor Data & Fusion Cost

Setup

Consider n randomly distributed sensors Vi ∈ Vn making random

  • bservations YVn.

Fusion center makes decision on underlying hypothesis using data The fusion policy πn schedules transmissions and computations at sensor nodes in Vn

(Vi, Yi)

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 5 / 59

slide-12
SLIDE 12

Setup: Fusion of Sensor Data & Fusion Cost

Setup

Consider n randomly distributed sensors Vi ∈ Vn making random

  • bservations YVn.

Fusion center makes decision on underlying hypothesis using data The fusion policy πn schedules transmissions and computations at sensor nodes in Vn

(Vi, Yi)

πn

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 5 / 59

slide-13
SLIDE 13

Setup: Fusion of Sensor Data & Fusion Cost

Setup

Consider n randomly distributed sensors Vi ∈ Vn making random

  • bservations YVn.

Fusion center makes decision on underlying hypothesis using data The fusion policy πn schedules transmissions and computations at sensor nodes in Vn

Cost of a Fusion Policy

The average fusion cost ¯ E(πn)∆ = 1 n

  • Vi∈Vn

Ei(πn)

(Vi, Yi)

πn

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 5 / 59

slide-14
SLIDE 14

Scaling of Fusion Cost & Lossless Fusion

Cost of a Fusion Policy

The fusion policy πn schedules transmissions of sensor nodes The average fusion cost ¯ E(πn)∆ = 1 n

  • Vi∈Vn

Ei(πn) How does ¯ E(πn) behave?

  • A. Anandkumar, J.E. Yukich, L. Tong, A. Swami, “Energy scaling laws for distributed inference

in random networks,” accepted to IEEE JSAC: Special Issues on Stochastic Geometry and Random Graphs for Wireless Networks, Dec. 2008 (on ArXiv)

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 6 / 59

slide-15
SLIDE 15

Scaling of Fusion Cost & Lossless Fusion

Cost of a Fusion Policy

The fusion policy πn schedules transmissions of sensor nodes The average fusion cost ¯ E(πn)∆ = 1 n

  • Vi∈Vn

Ei(πn) How does ¯ E(πn) behave?

¯ E(πn) n O(nν/2) O(n) O(√n) O(1)

  • A. Anandkumar, J.E. Yukich, L. Tong, A. Swami, “Energy scaling laws for distributed inference

in random networks,” accepted to IEEE JSAC: Special Issues on Stochastic Geometry and Random Graphs for Wireless Networks, Dec. 2008 (on ArXiv)

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 6 / 59

slide-16
SLIDE 16

Scaling of Fusion Cost & Lossless Fusion

Cost of a Fusion Policy

The fusion policy πn schedules transmissions of sensor nodes The average fusion cost ¯ E(πn)∆ = 1 n

  • Vi∈Vn

Ei(πn) How does ¯ E(πn) behave?

¯ E(πn) n O(nν/2) O(n) O(√n) O(1)

Constraint: No Loss in Inference Performance

A fusion policy is lossless if it results in no loss of inference performance at fusion center- as if all raw data available at fusion center

  • A. Anandkumar, J.E. Yukich, L. Tong, A. Swami, “Energy scaling laws for distributed inference

in random networks,” accepted to IEEE JSAC: Special Issues on Stochastic Geometry and Random Graphs for Wireless Networks, Dec. 2008 (on ArXiv)

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 6 / 59

slide-17
SLIDE 17

Problem Statement-I : Energy Scaling Laws

Fusion policy graph

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 7 / 59

slide-18
SLIDE 18

Problem Statement-I : Energy Scaling Laws

Fusion policy graph Network graph

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 7 / 59

slide-19
SLIDE 19

Problem Statement-I : Energy Scaling Laws

Fusion policy graph Network graph Dependency graph

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 7 / 59

slide-20
SLIDE 20

Problem Statement-I : Energy Scaling Laws

Fusion policy graph Network graph Dependency graph

Scalable Lossless Fusion Policy

Find a sequence of scalable policies {πn}, i.e., lim sup

n→∞

1 n

  • Vi∈Vn

Ei(πn) L2 = ¯ Eπ

∞ < ∞,

with small scaling constant ¯ Eπ

∞ such that optimal inference is achieved at

fusion center (lossless) for a class of node configurations.

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 7 / 59

slide-21
SLIDE 21

Problem II: Optimal Node Placement Distribution

Clustered

−0.5 −0.4 −0.3 −0.2 −0.1 0.1 0.2 0.3 0.4 0.5 −0.5 −0.4 −0.3 −0.2 −0.1 0.1 0.2 0.3 0.4 0.5

Uniform

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Spread-out

−0.5 −0.4 −0.3 −0.2 −0.1 0.1 0.2 0.3 0.4 0.5 −0.5 −0.4 −0.3 −0.2 −0.1 0.1 0.2 0.3 0.4 0.5

Goal: what placement strategy has best asymptotic average energy ¯ Eπ

∞?

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 8 / 59

slide-22
SLIDE 22

Problem II: Optimal Node Placement Distribution

Clustered

−0.5 −0.4 −0.3 −0.2 −0.1 0.1 0.2 0.3 0.4 0.5 −0.5 −0.4 −0.3 −0.2 −0.1 0.1 0.2 0.3 0.4 0.5

Uniform

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Spread-out

−0.5 −0.4 −0.3 −0.2 −0.1 0.1 0.2 0.3 0.4 0.5 −0.5 −0.4 −0.3 −0.2 −0.1 0.1 0.2 0.3 0.4 0.5

Scalable N

  • t

s c a l a b l e

¯ En n O(n) O(√n) O(1) Goal: what placement strategy has best asymptotic average energy ¯ Eπ

∞?

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 8 / 59

slide-23
SLIDE 23

Problem II: Optimal Node Placement Distribution

Clustered

−0.5 −0.4 −0.3 −0.2 −0.1 0.1 0.2 0.3 0.4 0.5 −0.5 −0.4 −0.3 −0.2 −0.1 0.1 0.2 0.3 0.4 0.5

Uniform

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Spread-out

−0.5 −0.4 −0.3 −0.2 −0.1 0.1 0.2 0.3 0.4 0.5 −0.5 −0.4 −0.3 −0.2 −0.1 0.1 0.2 0.3 0.4 0.5

Scalable

¯ En n ¯ Eπ

∞

Goal: what placement strategy has best asymptotic average energy ¯ Eπ

∞?

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 8 / 59

slide-24
SLIDE 24

Problem II: Optimal Node Placement Distribution

Clustered

−0.5 −0.4 −0.3 −0.2 −0.1 0.1 0.2 0.3 0.4 0.5 −0.5 −0.4 −0.3 −0.2 −0.1 0.1 0.2 0.3 0.4 0.5

Uniform

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Spread-out

−0.5 −0.4 −0.3 −0.2 −0.1 0.1 0.2 0.3 0.4 0.5 −0.5 −0.4 −0.3 −0.2 −0.1 0.1 0.2 0.3 0.4 0.5

?

Scalable

¯ En n ¯ Eπ

∞

Goal: what placement strategy has best asymptotic average energy ¯ Eπ

∞?

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 8 / 59

slide-25
SLIDE 25

Problem II: Optimal Node Placement Distribution

Clustered

−0.5 −0.4 −0.3 −0.2 −0.1 0.1 0.2 0.3 0.4 0.5 −0.5 −0.4 −0.3 −0.2 −0.1 0.1 0.2 0.3 0.4 0.5

Uniform

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Spread-out

−0.5 −0.4 −0.3 −0.2 −0.1 0.1 0.2 0.3 0.4 0.5 −0.5 −0.4 −0.3 −0.2 −0.1 0.1 0.2 0.3 0.4 0.5

?

Scalable

¯ En n ¯ Eπ

∞

Goal: what placement strategy has best asymptotic average energy ¯ Eπ

∞?

Challenge: Network & dependency graphs influenced by node locations

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 8 / 59

slide-26
SLIDE 26

Related Work: Scaling Laws in Networks

Capacity Scaling in Wireless Networks (Gupta & Kumar, IT ‘00)

Information flow between nodes, O(

1 √n log n) scaling

Routing Correlated Data

Algorithms for gathering correlated data (Cristescu, B. Beferull-Lozano & Vetterli, TON ‘06)

Function Computation

Rate scaling for Computation of separable functions at a sink (Giridhar & Kumar, JSAC ‘05) Bounds on time required to achieve a distortion level for distributed computation (Ayaso, Dahleh & Shah, ISIT ‘08)

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 9 / 59

slide-27
SLIDE 27

Outline

Models, assumptions, and problem formulations

◮ Propagation, network, and inference models

Insights from special cases Markov random fields Scalable data fusion for Markov random field Some related problems Conclusion and future work

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 10 / 59

slide-28
SLIDE 28

Propagation Model and Assumptions

Pr Pt (dB)

log d Transmitter Receiver d Cost for perfect reception: ET = O(dν). ν: path-loss exponent. Scheduling to avoid interference. Quantization effects ignored. Berkeley Mote Characteristics

Transmission range: 500-1000 ft. Current draw: 25mA (tx), 8mA (rx) Rate: 38.4 Kbaud.

  • A. Ephremides, “Energy concerns in wireless networks,” IEEE Wireless Comm., no. 4, Aug. 2002

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 11 / 59

slide-29
SLIDE 29

Network Graph Model For Communication

Random Node Placement

Points Xi

i.i.d.

∼ κ(x) on unit ball Q1 κ(x) bounded away from 0 and ∞ Network scaled to a fixed density λ: Vi = n

λXi

.... . . ... . . . .

κ(x) R = n

πλ

Vi Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 12 / 59

slide-30
SLIDE 30

Network Graph Model For Communication

Random Node Placement

Points Xi

i.i.d.

∼ κ(x) on unit ball Q1 κ(x) bounded away from 0 and ∞ Network scaled to a fixed density λ: Vi = n

λXi

.... . . ... . . . .

κ(x) R = n

πλ

Vi

Network Graph for Communication

Connected set of comm. links Energy & interference constraints

◮ Disc graph above critical radius

Adjustable transmission power

Vi

R = n

πλ

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 12 / 59

slide-31
SLIDE 31

Routing Strategies With No Fusion Are Not Scalable

¯ E(πn) n O(nν/2) O(n) O(√n) O(1)

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 13 / 59

slide-32
SLIDE 32

Routing Strategies With No Fusion Are Not Scalable

¯ E(πn) n O(nν/2) O(n) O(√n) O(1)

Single Hop

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 13 / 59

slide-33
SLIDE 33

Routing Strategies With No Fusion Are Not Scalable

¯ E(πn) n O(nν/2) O(n) O(√n) O(1)

Single Hop Shortest path

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 13 / 59

slide-34
SLIDE 34

Routing Strategies With No Fusion Are Not Scalable

?

¯ E(πn) n O(nν/2) O(n) O(√n) O(1)

Single Hop Shortest path

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 13 / 59

slide-35
SLIDE 35

Routing Strategies With No Fusion Are Not Scalable

?

¯ E(πn) n O(nν/2) O(n) O(√n) O(1)

Single Hop Shortest path Incorporate inference model (dependency graph) for scalable fusion policy

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 13 / 59

slide-36
SLIDE 36

Distributed Computation of Sufficient Statistic

Example: Sufficient Statistic for Mean Estimation Y1, . . . , Yn

i.i.d.

∼ N(θ, 1)

  • i Yi sufficient to estimate θ: no performance loss
  • E. Dynkin, “Necessary and sufficient statistics for a family of probability distributions,” Tran.

Math, Stat. and Prob., vol. 1, pp. 23-41, 1961

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 14 / 59

slide-37
SLIDE 37

Distributed Computation of Sufficient Statistic

Example: Sufficient Statistic for Mean Estimation Y1, . . . , Yn

i.i.d.

∼ N(θ, 1)

  • i Yi sufficient to estimate θ: no performance loss

Sufficient Statistic For Inference: No Performance Loss Dimensionality reduction: lower communication costs Minimal Sufficiency: Maximum dimensionality reduction

  • E. Dynkin, “Necessary and sufficient statistics for a family of probability distributions,” Tran.

Math, Stat. and Prob., vol. 1, pp. 23-41, 1961

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 14 / 59

slide-38
SLIDE 38

Distributed Computation of Sufficient Statistic

Example: Sufficient Statistic for Mean Estimation Y1, . . . , Yn

i.i.d.

∼ N(θ, 1)

  • i Yi sufficient to estimate θ: no performance loss

Sufficient Statistic For Inference: No Performance Loss Dimensionality reduction: lower communication costs Minimal Sufficiency: Maximum dimensionality reduction Binary Hypothesis Testing: Decide Y1, . . . , Yn ∼ f0(Yn) or f1(Yn)

  • E. Dynkin, “Necessary and sufficient statistics for a family of probability distributions,” Tran.

Math, Stat. and Prob., vol. 1, pp. 23-41, 1961

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 14 / 59

slide-39
SLIDE 39

Distributed Computation of Sufficient Statistic

Example: Sufficient Statistic for Mean Estimation Y1, . . . , Yn

i.i.d.

∼ N(θ, 1)

  • i Yi sufficient to estimate θ: no performance loss

Sufficient Statistic For Inference: No Performance Loss Dimensionality reduction: lower communication costs Minimal Sufficiency: Maximum dimensionality reduction Binary Hypothesis Testing: Decide Y1, . . . , Yn ∼ f0(Yn) or f1(Yn) Minimal Sufficient Statistic for Binary Hypothesis Testing (Dynkin 61) Log Likelihood Ratio: LG(Yn) = log f0(Yn) f1(Yn)

  • E. Dynkin, “Necessary and sufficient statistics for a family of probability distributions,” Tran.

Math, Stat. and Prob., vol. 1, pp. 23-41, 1961

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 14 / 59

slide-40
SLIDE 40

Distributed Computation of Sufficient Statistic

Example: Sufficient Statistic for Mean Estimation Y1, . . . , Yn

i.i.d.

∼ N(θ, 1)

  • i Yi sufficient to estimate θ: no performance loss

Sufficient Statistic For Inference: No Performance Loss Dimensionality reduction: lower communication costs Minimal Sufficiency: Maximum dimensionality reduction Binary Hypothesis Testing: Decide Y1, . . . , Yn ∼ f0(Yn) or f1(Yn) Minimal Sufficient Statistic for Binary Hypothesis Testing (Dynkin 61) Log Likelihood Ratio: LG(Yn) = log f0(Yn) f1(Yn)

Is there a scalable fusion policy for computing likelihood ratio?

  • E. Dynkin, “Necessary and sufficient statistics for a family of probability distributions,” Tran.

Math, Stat. and Prob., vol. 1, pp. 23-41, 1961

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 14 / 59

slide-41
SLIDE 41

Inference Model and Assumptions

Random location Vn

∆

=(V1, · · · , Vn) and sensor data YVn. Binary hypothesis: H0 vs. H1: Hk : YVn ∼ f(yvn|Vn = vn; Hk)

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 15 / 59

slide-42
SLIDE 42

Inference Model and Assumptions

Random location Vn

∆

=(V1, · · · , Vn) and sensor data YVn. Binary hypothesis: H0 vs. H1: Hk : YVn ∼ f(yvn|Vn = vn; Hk) YVn: Markov random field with dependency graph Gk(Vn)

Fusion center Yi Yj

Dependency neighbor condition: No direct “interaction” between two nodes unless they are neighbors in dependency graph

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 15 / 59

slide-43
SLIDE 43

Outline

Models, assumptions, and problem formulations

◮ Propagation, network, and inference models

Insights from special cases Markov random fields Scalable data fusion for Markov random field Some related problems Conclusion and future work

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 16 / 59

slide-44
SLIDE 44

Optimal Fusion: the IID Case

Consider i.i.d. observations

Hk : YV ∼

  • i∈V

fk(Yi)

Sufficient statistic

L(YV) = log f0(YV) f1(YV) =

  • i∈V

L(Yi)

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 17 / 59

slide-45
SLIDE 45

Optimal Fusion: the IID Case

Consider i.i.d. observations

Hk : YV ∼

  • i∈V

fk(Yi)

Sufficient statistic

L(YV) = log f0(YV) f1(YV) =

  • i∈V

L(Yi)

The optimal data fusion is the LLR aggregation over the MST (why?)

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 17 / 59

slide-46
SLIDE 46

Optimal Fusion: the IID Case

Consider i.i.d. observations

Hk : YV ∼

  • i∈V

fk(Yi)

Sufficient statistic

L(YV) = log f0(YV) f1(YV) =

  • i∈V

L(Yi)

The optimal data fusion is the LLR aggregation over the MST (why?)

each node must transmit at least once MST minimizes power-weighted edge sum: min

i |ei|ν

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 17 / 59

slide-47
SLIDE 47

Optimal Fusion: the IID Case

Consider i.i.d. observations

Hk : YV ∼

  • i∈V

fk(Yi)

Sufficient statistic

L(YV) = log f0(YV) f1(YV) =

  • i∈V

L(Yi)

The optimal data fusion is the LLR aggregation over the MST (why?)

each node must transmit at least once MST minimizes power-weighted edge sum: min

i |ei|ν

Assume network graph contains MST

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 17 / 59

slide-48
SLIDE 48

Optimal Fusion: Energy Analysis

Energy per node is

¯ E(πMST

n

) = 1 n

  • e∈MSTn

|e|ν Steele’88, Yukich’00 1 n

  • e∈MSTn

|e|ν L2 → ¯ E MST

∞

< ∞ Scalable fusion along MST for independent data

  • J. E. Yukich,“Asymptotics for weighted minimal spanning trees on random points,” Stochastic

Processes and their Applications, vol. 85, No. 1, pp. 123-138, Jan. 2000.

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 18 / 59

slide-49
SLIDE 49

Role of Sensor Location Distribution

Better scaling constant ¯ E MST

∞

= ζ(ν; MST)

  • Q1

κ(x)1− ν

2 dx?

Clustered

−0.5 −0.4 −0.3 −0.2 −0.1 0.1 0.2 0.3 0.4 0.5 −0.5 −0.4 −0.3 −0.2 −0.1 0.1 0.2 0.3 0.4 0.5

Uniform

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Spread-out

−0.5 −0.4 −0.3 −0.2 −0.1 0.1 0.2 0.3 0.4 0.5 −0.5 −0.4 −0.3 −0.2 −0.1 0.1 0.2 0.3 0.4 0.5

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 19 / 59

slide-50
SLIDE 50

Role of Sensor Location Distribution

Better scaling constant ¯ E MST

∞

= ζ(ν; MST)

  • Q1

κ(x)1− ν

2 dx?

Clustered

−0.5 −0.4 −0.3 −0.2 −0.1 0.1 0.2 0.3 0.4 0.5 −0.5 −0.4 −0.3 −0.2 −0.1 0.1 0.2 0.3 0.4 0.5

Uniform

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Spread-out

−0.5 −0.4 −0.3 −0.2 −0.1 0.1 0.2 0.3 0.4 0.5 −0.5 −0.4 −0.3 −0.2 −0.1 0.1 0.2 0.3 0.4 0.5

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 19 / 59

slide-51
SLIDE 51

Role of Sensor Location Distribution

Better scaling constant ¯ E MST

∞

= ζ(ν; MST)

  • Q1

κ(x)1− ν

2 dx?

Clustered

−0.5 −0.4 −0.3 −0.2 −0.1 0.1 0.2 0.3 0.4 0.5 −0.5 −0.4 −0.3 −0.2 −0.1 0.1 0.2 0.3 0.4 0.5

Uniform

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Spread-out

−0.5 −0.4 −0.3 −0.2 −0.1 0.1 0.2 0.3 0.4 0.5 −0.5 −0.4 −0.3 −0.2 −0.1 0.1 0.2 0.3 0.4 0.5

Ratio of ¯ EMST

∞

  • f clustered and spread-out placements with respect to uniform

1 2 3 4 5 0.5 1 1.5 2 2.5 Uniform is Worst−Case Uniform is Optimal

Path-loss Exponent ν

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 19 / 59

slide-52
SLIDE 52

Outline

Models, assumptions, and problem formulations

◮ Propagation, network, and inference models

Insights from special cases Markov random fields

◮ Conditional-independence Relationships ◮ Hammersley-Clifford Theorem ◮ Form of Likelihood Ratio

Scalable data fusion for Markov random field Some related problems Conclusion and future work

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 20 / 59

slide-53
SLIDE 53

Inference Model and Assumptions

Random location Vn

∆

=(V1, · · · , Vn) and samples YVn. Binary hypothesis: H0 vs. H1: Hk : YVn ∼ f(yvn|Vn = vn; Hk)

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 21 / 59

slide-54
SLIDE 54

Inference Model and Assumptions

Random location Vn

∆

=(V1, · · · , Vn) and samples YVn. Binary hypothesis: H0 vs. H1: Hk : YVn ∼ f(yvn|Vn = vn; Hk) YVn: Markov random field with dependency graph Gk(Vn)

Fusion center Yi Yj Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 21 / 59

slide-55
SLIDE 55

Dependency Graph and Markov Random Field

Consider an undirected graph G(V), each vertex Vi ∈ V is associated with a random variable Yi Yi Yk Yj

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 22 / 59

slide-56
SLIDE 56

Dependency Graph and Markov Random Field

Consider an undirected graph G(V), each vertex Vi ∈ V is associated with a random variable Yi V\{Nbd(i) ∪ i} i Nbd(i) Yi ⊥ ⊥ YV\{Nbd(i)∪i}|YNbd(i)

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 22 / 59

slide-57
SLIDE 57

Dependency Graph and Markov Random Field

Consider an undirected graph G(V), each vertex Vi ∈ V is associated with a random variable Yi For any disjoint sets A, B, C such that C separates A and B, V\{Nbd(i) ∪ i} i Nbd(i) Yi ⊥ ⊥ YV\{Nbd(i)∪i}|YNbd(i) A B C YA ⊥ ⊥ YB|YC

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 22 / 59

slide-58
SLIDE 58

Likelihood Function of MRF

Hammersley-Clifford Theorem’71

Let f be joint pdf of MRF with graph G(V), − log f(YV) =

  • c∈C

Ψc(Yc) where C is the set of maximal cliques.

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 23 / 59

slide-59
SLIDE 59

Likelihood Function of MRF

Hammersley-Clifford Theorem’71

Let f be joint pdf of MRF with graph G(V), − log f(YV) =

  • c∈C

Ψc(Yc) where C is the set of maximal cliques.

Gaussian MRF: − log f(YV) = 1

2

  • −n log 2π −log |ΣV|+
  • (i,j)∈G

Σ−1

V (i, j)YiYj + i∈V

Σ−1

V (i, i)Y 2 i

  • Dependency

Graph

1 2 3 4 5 6 7 8

1 2 3 4 5 6 7 1 2 3 7 6 5 4

X X X X X X X X X X X X X X X X

8 8

X X X

Inverse of Covariance Matrix

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 23 / 59

slide-60
SLIDE 60

Inference Model and Assumptions

Random location Vn

∆

=(V1, · · · , Vn) and samples YVn. Binary hypothesis: H0 vs. H1: Hk : YVn ∼ f(yvn|Vn = vn, Hk)

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 24 / 59

slide-61
SLIDE 61

Inference Model and Assumptions

Random location Vn

∆

=(V1, · · · , Vn) and samples YVn. Binary hypothesis: H0 vs. H1: Hk : YVn ∼ f(yvn|Vn = vn, Hk) YVn: Markov random field with dependency graph Gk(Vn) − log f(YVn|Gk, Hk) =

  • c∈Ck

Ψk,c(Yc) where Cn,k is the collection of maximal cliques Ψk,c clique potentials.

H0 : G0 H1 : G1

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 24 / 59

slide-62
SLIDE 62

Dependency Graph Model

H0 : G0 H1 : G1

Recall Hammersley-Clifford Theorem

− log f(YVn|Gk, Hk) =

c∈Ck

Ψk,c(Yc)

Minimal sufficient statistic

LG(YV) = log f(YV|G0, H0) f(YV|G1, H1)

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 25 / 59

slide-63
SLIDE 63

Dependency Graph Model

H0 : G0 H1 : G1 Joint:G0 ∪ G1

Recall Hammersley-Clifford Theorem

− log f(YVn|Gk, Hk) =

c∈Ck

Ψk,c(Yc)

Minimal sufficient statistic

LG(YV) = log f(YV|G0, H0) f(YV|G1, H1)

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 25 / 59

slide-64
SLIDE 64

Dependency Graph Model

H0 : G0 H1 : G1 Joint:G0 ∪ G1

Recall Hammersley-Clifford Theorem

− log f(YVn|Gk, Hk) =

c∈Ck

Ψk,c(Yc)

Minimal sufficient statistic

LG(YV) = log f(YV|G0, H0) f(YV|G1, H1)

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 25 / 59

slide-65
SLIDE 65

Dependency Graph Model

H0 : G0 H1 : G1 Joint:G0 ∪ G1

Recall Hammersley-Clifford Theorem

− log f(YVn|Gk, Hk) =

c∈Ck

Ψk,c(Yc)

Minimal sufficient statistic

LG(YV) = log f(YV|G0, H0) f(YV|G1, H1)=

  • c∈C

φ(Yc)

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 25 / 59

slide-66
SLIDE 66

Outline

Models, assumptions, and problem formulations

◮ Propagation, network, and inference models

Insights from special cases Markov random fields Scalable data fusion for Markov random field

◮ A suboptimal scalable policy ◮ Effects of sparsity on scalability ◮ Energy scaling analysis

Some related problems Conclusion and future work

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 26 / 59

slide-67
SLIDE 67

Fusion for Markov Random Field

Network graph Dependency graph Fusion policy graph

Lossless Fusion Policies

Given the network and dependency graphs (N, G), FG,N

∆

={π : LG(YV) =

c∈C

φ(Yc) computable at the fusion center}.

Optimal fusion Policy: E(π∗

n) =

min

π∈FGn,Nn

  • i Ei(πn)

NP-hard: Steiner-tree reduction (INFOCOM ‘08)

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 27 / 59

slide-68
SLIDE 68

Data Fusion for Markov Random Field (DFMRF)

Log-likelihood Ratio LG(YV) =

c∈C

φ(Yc) Step I: Data forwarding and local computation: Given dependency graph G and network graph N. Randomly select a representative (processor) in each clique of G. Clique members forward data to processor via SPR on N

c4 c2 c3

c1

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 28 / 59

slide-69
SLIDE 69

Data Fusion for Markov Random Field (DFMRF)

Log-likelihood Ratio LG(YV) =

c∈C

φ(Yc) Step I: Data forwarding and local computation: Given dependency graph G and network graph N. Randomly select a representative (processor) in each clique of G. Clique members forward data to processor via SPR on N

c4 c2 c3

c1

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 28 / 59

slide-70
SLIDE 70

Data Fusion for Markov Random Field (DFMRF)

Log-likelihood Ratio LG(YV) =

c∈C

φ(Yc) Step I: Data forwarding and local computation: Given dependency graph G and network graph N. Randomly select a representative (processor) in each clique of G. Clique members forward data to processor via SPR on N

Raw Data: Yi

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 28 / 59

slide-71
SLIDE 71

Data Fusion for Markov Random Field (DFMRF)

Log-likelihood Ratio LG(YV) =

c∈C

φ(Yc) Step I: Data forwarding and local computation: Given dependency graph G and network graph N. Randomly select a representative (processor) in each clique of G. Clique members forward data to processor via SPR on N

φ(Yc1) + φ(Yc2) φ(Yc3) φ(Yc4)

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 28 / 59

slide-72
SLIDE 72

Data Fusion for Markov Random Field (DFMRF)

Log-likelihood Ratio LG(YV) =

c∈C

φ(Yc) Step I: Data forwarding and local computation: Given dependency graph G and network graph N. Randomly select a representative (processor) in each clique of G. Clique members forward data to processor via SPR on N Step II: aggregating LLR over MST

φ(Yc1) + φ(Yc2) φ(Yc3) φ(Yc4)

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 28 / 59

slide-73
SLIDE 73

Data Fusion for Markov Random Field (DFMRF)

Log-likelihood Ratio LG(YV) =

c∈C

φ(Yc) Step I: Data forwarding and local computation: Given dependency graph G and network graph N. Randomly select a representative (processor) in each clique of G. Clique members forward data to processor via SPR on N Step II: aggregating LLR over MST

+

LG

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 28 / 59

slide-74
SLIDE 74

Data Fusion for Markov Random Field (DFMRF)

Log-likelihood Ratio LG(YV) =

c∈C

φ(Yc) Step I: Data forwarding and local computation: Given dependency graph G and network graph N. Randomly select a representative (processor) in each clique of G. Clique members forward data to processor via SPR on N Step II: aggregating LLR over MST

+

LG

Total energy consumption= Data Forwarding + MST Aggregation

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 28 / 59

slide-75
SLIDE 75

Effects of Dependency Graph Sparsity on Scalability

Sparsity of Dependency Graph

  • i∈V

φ(Yi)

  • c∈C

φ(Yc) φ(YV)

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 29 / 59

slide-76
SLIDE 76

Effects of Dependency Graph Sparsity on Scalability

Sparsity of Dependency Graph

  • i∈V

φ(Yi)

  • c∈C

φ(Yc) φ(YV)

Stabilizing graph (Penrose-Yukich)

Local graph structure not affected by far away points (k-NNG, Disk)

  • M. D. Penrose and J. E. Yukich, “Weak Laws Of Large Numbers In Geometric Probability,”

Annals of Applied probability, vol. 13, no. 1, pp. 277-303, 2003

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 29 / 59

slide-77
SLIDE 77

Effects of Network Graph Sparsity on Scalability

Sparsity of Network Graph

Single Hop u-Spanner Complete (Nn)

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 30 / 59

slide-78
SLIDE 78

Effects of Network Graph Sparsity on Scalability

Sparsity of Network Graph

Single Hop u-Spanner Complete (Nn)

u-Spanner

Given network graph Nn and its completion Nn, Nn is a u-spanner if max

Vi,Vj∈Vn

E(Vi → Vj; SP on Nn) E(Vi → Vj; SP on Nn) ≤ u

Gabriel: u = 1 for ν ≥ 2

  • Anima Anandkumar (Cornell)

Scaling Laws B-exam 05/26/09 30 / 59

slide-79
SLIDE 79

Effects of Network Graph Sparsity on Scalability

Sparsity of Network Graph

Single Hop u-Spanner Complete (Nn)

u-Spanner

Given network graph Nn and its completion Nn, Nn is a u-spanner if max

Vi,Vj∈Vn

E(Vi → Vj; SP on Nn) E(Vi → Vj; SP on Nn) ≤ u

Gabriel: u = 1 for ν ≥ 2

  • Longest edge O(√log n)

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 30 / 59

slide-80
SLIDE 80

Main Result: Scalability of DFMRF

u-Spanner Stabilizing DFMRF Network graph Dependency graph Fusion policy graph

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 31 / 59

slide-81
SLIDE 81

Main Result: Scalability of DFMRF

u-Spanner Stabilizing DFMRF Network graph Dependency graph Fusion policy graph

Scaling Constant for Scale-Invariant Graphs (k-NNG)

lim sup

n→∞

E(πDFMRF

n

) n ≤ λ− ν

2 [u ζ(ν; G)

  • data forward

+ ζ(ν; MST)

  • MST aggregation

]

  • Q1

κ(x)1− ν

2 dx,

ζ(ν; G)

∆

= E

  • (0,j)∈G(P1∪{0})

|0, j|ν

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 31 / 59

slide-82
SLIDE 82

Approximation Ratio for DFMRF

Recall FG

∆

={π : LG(YV) computable at the fusion center} E(π∗

n) = min π∈FG

  • i

Ei(πn)

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 32 / 59

slide-83
SLIDE 83

Approximation Ratio for DFMRF

Recall FG

∆

={π : LG(YV) computable at the fusion center} E(π∗

n) = min π∈FG

  • i

Ei(πn)

Lower and Upper Bounds For Optimal Fusion Policy

E(πMST

n

) ≤ E(π∗

n) ≤ E(πDFMRF n

)

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 32 / 59

slide-84
SLIDE 84

Approximation Ratio for DFMRF

Recall FG

∆

={π : LG(YV) computable at the fusion center} E(π∗

n) = min π∈FG

  • i

Ei(πn)

Lower and Upper Bounds For Optimal Fusion Policy

E(πMST

n

) ≤ E(π∗

n) ≤ E(πDFMRF n

)

Approximation Ratio of DFMRF for k-NNG Dependency

lim sup

n→∞ E(πDFMRF

n

) E(π∗

n)

≤

  • 1 + u

ζ(ν; G) ζ(ν; MST)

  • Constant factor approximation for DFMRF for large networks

Approximation ratio independent of node placement for k-NNG

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 32 / 59

slide-85
SLIDE 85

Simulation Results for k-NNG Dependency

  • Avg. Energy Under Uniform Placement

20 40 60 80 100 120 140 160 180 1 2 3 4 5 6 7 8 9 10

Number of nodes n

1-NNG: DFMRF 3-NNG: DFMRF 2-NNG: DFMRF No Fusion: SPR 0-NNG: MST

  • Approx. Ratio for DFMRF

20 40 60 80 100 120 140 160 180 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5

Number of nodes n

1-NNG dependency 3-NNG dependency 2-NNG dependency No correlation Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 33 / 59

slide-86
SLIDE 86

What Have We Done and Left Out....

Energy scaling laws

◮ Assumed stabilizing dependency graph and u-spanner network graph ◮ Defined a fusion policy πDFMRF

n

(DFMRF)

◮ Scalability analysis: lim sup

n→∞ 1 n

  • i Ei(πDFMRF

n

) ≤ ¯ EDFMRF

∞

α ≤ ¯ Eπ∗

∞ ≤ ¯

EDFMRF

∞

≤ β < ∞

◮ Asymptotic approximation ratio:

β α.

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 34 / 59

slide-87
SLIDE 87

What Have We Done and Left Out....

Energy scaling laws

◮ Assumed stabilizing dependency graph and u-spanner network graph ◮ Defined a fusion policy πDFMRF

n

(DFMRF)

◮ Scalability analysis: lim sup

n→∞ 1 n

  • i Ei(πDFMRF

n

) ≤ ¯ EDFMRF

∞

α ≤ ¯ Eπ∗

∞ ≤ ¯

EDFMRF

∞

≤ β < ∞

◮ Asymptotic approximation ratio:

β α.

Remarks

◮ Energy consumption is a key parameter for large sensor networks. ◮ Sensor location is a new source of randomness in distributed inference ◮ Asymptotic techniques are useful in overall network design. Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 34 / 59

slide-88
SLIDE 88

What Have We Done and Left Out....

Energy scaling laws

◮ Assumed stabilizing dependency graph and u-spanner network graph ◮ Defined a fusion policy πDFMRF

n

(DFMRF)

◮ Scalability analysis: lim sup

n→∞ 1 n

  • i Ei(πDFMRF

n

) ≤ ¯ EDFMRF

∞

α ≤ ¯ Eπ∗

∞ ≤ ¯

EDFMRF

∞

≤ β < ∞

◮ Asymptotic approximation ratio:

β α.

Remarks

◮ Energy consumption is a key parameter for large sensor networks. ◮ Sensor location is a new source of randomness in distributed inference ◮ Asymptotic techniques are useful in overall network design.

We have ignored several issues:

◮ one-shot inference ◮ quantization of measurements and link capacity constraints ◮ perfect transmission/reception and scheduling ◮ computation cost and overheads Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 34 / 59

slide-89
SLIDE 89

Outline

Models, assumptions, and problem formulations

◮ Propagation, network, and inference models

Insights from special cases Markov random fields Scalable data fusion for Markov random field Some related problems

◮ Error exponents on random graph ◮ Cost performance tradeoff ◮ Inference in finite networks

Conclusion and future work

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 35 / 59

slide-90
SLIDE 90

Design for Energy Constrained Inference

Error Exponent (IT ‘09, ISIT ‘09)

For MRF hypothesis with node density λ and distribution κ(x), − 1 n log P1→0(n)

?

− → Dλ,κ

P1→0(n) ∼ exp(−nDλ,κ)

n P1→0(n)

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 36 / 59

slide-91
SLIDE 91

Design for Energy Constrained Inference

Error Exponent (IT ‘09, ISIT ‘09)

For MRF hypothesis with node density λ and distribution κ(x), − 1 n log P1→0(n)

?

− → Dλ,κ

P1→0(n) ∼ exp(−nDλ,κ)

n P1→0(n)

Design for Energy Constrained Inference (SP ‘08)

max

λ,κ,π Dλ,κ

subject to ¯ Eπ

λ,κ ≤ ¯

Eo

(1) A. Anandkumar, L. Tong, A. Swami, “Detection of Gauss-Markov Random Fields with Nearest-Neighbor Dependency,” IEEE

  • Tran. on Information Theory, Feb. 2009

(2) A. Anandkumar, J.E. Yukich, L. Tong, A. Willsky, “Detection Error Exponent for Spatially Dependent Samples in Random Networks,” Proc. of IEEE ISIT, Jun. 2009 (3) A. Anandkumar, L. Tong, and A. Swami, “Optimal Node Density for Detection in Energy Constrained Random Networks,” IEEE Tran. Signal Proc., pp. 5232-5245, Oct. 2008. Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 36 / 59

slide-92
SLIDE 92

Inference In Finite Fusion Networks

We have so far considered

Random node placement Scaling as n → ∞

Harder problem

Arbitrary node placement Finite n

Results (INFOCOM ‘08 & ‘09)

Fusion scheme has a Steiner tree reduction Cost-performance tradeoff

Fusion Center

Yn = [Y1, . . . , Yn] (1) A. Anandkumar, L. Tong, A. Swami, and A. Ephremides, “Minimum Cost Data Aggregation with Localized Processing for Statistical Inference,” in Proc. of INFOCOM, April 2008 (2) A. Anandkumar, M. Wang, L. Tong, and A. Swami, “Prize-Collecting Data Fusion for Cost- Performance Tradeoff in Distributed Inference,” in Proc. of IEEE INFOCOM, April 2009. Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 37 / 59

slide-93
SLIDE 93

Medium Access Control

(SP ‘07, IT ‘08) With L. Tong, Cornell, & A. Swami, ARL

Constant BW Scaling

Fusion Center Realization . Type

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 38 / 59

slide-94
SLIDE 94

Medium Access Control

(SP ‘07, IT ‘08) With L. Tong, Cornell, & A. Swami, ARL

Constant BW Scaling

Fusion Center Realization . Type

Transaction Monitoring

(Sigmetrics ‘08) With C. Bisdikian & D. Agrawal, IBM Research

Decentralized Bipartite Matching

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 38 / 59

slide-95
SLIDE 95

Medium Access Control

(SP ‘07, IT ‘08) With L. Tong, Cornell, & A. Swami, ARL

Constant BW Scaling

Fusion Center Realization . Type

Transaction Monitoring

(Sigmetrics ‘08) With C. Bisdikian & D. Agrawal, IBM Research

Decentralized Bipartite Matching

Learning dependency models

(ISIT ‘09) With V. Tan, A. Willsky, MIT, & L. Tong, Cornell

SNR for learning

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 38 / 59

slide-96
SLIDE 96

Medium Access Control

(SP ‘07, IT ‘08) With L. Tong, Cornell, & A. Swami, ARL

Constant BW Scaling

Fusion Center Realization . Type

Transaction Monitoring

(Sigmetrics ‘08) With C. Bisdikian & D. Agrawal, IBM Research

Decentralized Bipartite Matching

Learning dependency models

(ISIT ‘09) With V. Tan, A. Willsky, MIT, & L. Tong, Cornell

SNR for learning

Competitive Learning

With A.K. Tang, Cornell Univ.

Regret-free under interference

Whitespace Spectrum Primary Secondary

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 38 / 59

slide-97
SLIDE 97

Holy Grail...

Networks

Seamless operation Efficient resource utilization Unified theory: feasibility of large networks under different applications

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 39 / 59

slide-98
SLIDE 98

Holy Grail...

Networks

Seamless operation Efficient resource utilization Unified theory: feasibility of large networks under different applications

Network Data

Data-centric paradigms Unifying computation and communication.

◮ e.g., inference

Fundamental limits and scalable algorithms

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 39 / 59

slide-99
SLIDE 99

Multidisciplinary Research

Detection/Estimation Theory Information Theory Asymptotics and Large Deviations Communication Theory Random Graphs Approximation Algorithms

Unified Network Theory

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 40 / 59

slide-100
SLIDE 100

http://acsp.ece.cornell.edu/members/anima.html

Thank You!

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 41 / 59

slide-101
SLIDE 101

Appendix

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 42 / 59

slide-102
SLIDE 102

Illustration of Stabilization: 1-NNG

For stabilizing dependency graphs, computation of clique potentials φc(Yc) does not require long distance communication

  • Dep. Edges

Forwarding Processor

Back Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 43 / 59

slide-103
SLIDE 103

Illustration of Stabilization: 1-NNG

For stabilizing dependency graphs, computation of clique potentials φc(Yc) does not require long distance communication

  • Dep. Edges

Forwarding Processor

men

Back Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 43 / 59

slide-104
SLIDE 104

Illustration of Stabilization: 1-NNG

For stabilizing dependency graphs, computation of clique potentials φc(Yc) does not require long distance communication

  • Dep. Edges

Forwarding Processor

men

Back Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 43 / 59

slide-105
SLIDE 105

Illustration of Stabilization: 1-NNG

For stabilizing dependency graphs, computation of clique potentials φc(Yc) does not require long distance communication

  • Dep. Edges

Forwarding Processor

men

Back Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 43 / 59

slide-106
SLIDE 106

Illustration of Stabilization: 1-NNG

For stabilizing dependency graphs, computation of clique potentials φc(Yc) does not require long distance communication

  • Dep. Edges

Forwarding Processor

men

Back Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 43 / 59

slide-107
SLIDE 107

Illustration of Stabilization: 1-NNG

For stabilizing dependency graphs, computation of clique potentials φc(Yc) does not require long distance communication

  • Dep. Edges

Forwarding Processor

men

Back Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 43 / 59

slide-108
SLIDE 108

Illustration of Stabilization: 1-NNG

For stabilizing dependency graphs, computation of clique potentials φc(Yc) does not require long distance communication

  • Dep. Edges

Forwarding Processor

men

Back Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 43 / 59

slide-109
SLIDE 109

Illustration of Stabilization: 1-NNG

For stabilizing dependency graphs, computation of clique potentials φc(Yc) does not require long distance communication

  • Dep. Edges

Forwarding Processor

men

Back Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 43 / 59

slide-110
SLIDE 110

Illustration of Stabilization: 1-NNG

For stabilizing dependency graphs, computation of clique potentials φc(Yc) does not require long distance communication

  • Dep. Edges

Forwarding Processor

men

Back Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 43 / 59

slide-111
SLIDE 111

Illustration of Stabilization: 1-NNG

For stabilizing dependency graphs, computation of clique potentials φc(Yc) does not require long distance communication

  • Dep. Edges

Forwarding Processor

men

Back Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 43 / 59

slide-112
SLIDE 112

Illustration of Stabilization: 1-NNG

For stabilizing dependency graphs, computation of clique potentials φc(Yc) does not require long distance communication

  • Dep. Edges

Forwarding Processor

men

Back Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 43 / 59

slide-113
SLIDE 113

Illustration of Stabilization: 1-NNG

For stabilizing dependency graphs, computation of clique potentials φc(Yc) does not require long distance communication

  • Dep. Edges

Forwarding Processor

men

Back Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 43 / 59

slide-114
SLIDE 114

Illustration of Stabilization: 1-NNG

For stabilizing dependency graphs, computation of clique potentials φc(Yc) does not require long distance communication

  • Dep. Edges

Forwarding Processor

men

Back Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 43 / 59

slide-115
SLIDE 115

Key ideas

Bound on Forwarding

E(Forward) =

  • c∈C(V)
  • i⊂c

SP(i, Proc(c)) ≤ u

  • c∈C(V)
  • i⊂c

|i, Proc(c)|ν

  • Direct Tx.

≤ u

  • e∈G

|e|ν

In Each Clique

  • Dep. Edges

Forwarding Processor

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 44 / 59

slide-116
SLIDE 116

Key ideas

Bound on Forwarding

E(Forward) =

  • c∈C(V)
  • i⊂c

SP(i, Proc(c)) ≤ u

  • c∈C(V)
  • i⊂c

|i, Proc(c)|ν

  • Direct Tx.

≤ u

  • e∈G

|e|ν

In Each Clique

  • Dep. Edges

Forwarding Processor

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 44 / 59

slide-117
SLIDE 117

Key ideas

Bound on Forwarding

E(Forward) =

  • c∈C(V)
  • i⊂c

SP(i, Proc(c)) ≤ u

  • c∈C(V)
  • i⊂c

|i, Proc(c)|ν

  • Direct Tx.

≤ u

  • e∈G

|e|ν

In Each Clique

  • Dep. Edges

Forwarding Processor

LLN for Normalized Sum of Edge Weights (Penrose-Yukich)

n → ∞ Origin

  • f origin of Poisson process

1 n

  • e∈G(Vn)

|e|ν

1 2E[

  • (0,j)∈G(P1∪0)

|0, j|ν]

  • Q1

κ(x)1− ν

2 dx

Back Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 44 / 59

slide-118
SLIDE 118

Scaling Constant via Poissonization

1 n

  • e∈MSTn

|e|ν

L2

→ ¯ E MST

∞

¯ E MST

∞ (κ)

= ζ(ν; MST)

  • Q1

κ(x)1− ν

2 dx,

ζ(ν; MST) = 1 2E   

  • (0,j)∈MST(P1∪0)

|0, j|ν   

n → ∞ Origin

Back IID Back MRF Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 45 / 59

slide-119
SLIDE 119

Optimal Fusion: Lower Bound

Recall FG

∆

={π : LG(YV) computable at the fusion center} E(π∗

n) = min π∈FG

  • i

Ei(πn)

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 46 / 59

slide-120
SLIDE 120

Optimal Fusion: Lower Bound

Recall FG

∆

={π : LG(YV) computable at the fusion center} E(π∗

n) = min π∈FG

  • i

Ei(πn)

Lower Bound

For any dependency graph G 1 nE(π∗

n) ≥ 1

nE(πMST

n

) L2 → ζ(ν; MST)

  • Q1

κ(x)1− ν

2 dx

Each node must transmit at least once. The fusion graph needs to be connected. Lower bound is tight (achieved for independent data).

Back Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 46 / 59

slide-121
SLIDE 121

Example: Gauss-Markov random field

Test on GMRF: H0 : XV ∼ N(0, σ2

0I)

H1 : XV ∼ N(0, Σ) Nearest neighbor graph.

Xj Xj

H0 H1

Xi Xi Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 47 / 59

slide-122
SLIDE 122

Example: Gauss-Markov random field

Test on GMRF: H0 : XV ∼ N(0, σ2

0I)

H1 : XV ∼ N(0, Σ) Nearest neighbor graph.

Xj Xj

H0 H1

Xi Xi

Tradeoff between exploiting signal strength and exploiting correlation: K = σ2

1

σ2 vs. g(Rij)∆ =Σ(i, j) σ2

1

where Σ[i, i] = σ2

1 and g(·) a decreasing function.

◮ Sparse deployment: independent samples, costly data fusion. ◮ Dense deployment: correlated samples, require less energy. Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 47 / 59

slide-123
SLIDE 123

Error exponent behavior

Closed-form error exponent − lim

n→∞ log PM(n)

= D(λ, K; g) = 1 2Eλ h

  • Zλ−0.5, K; g
  • + DIID(K)

The error exponent reverse its behavior at a threshold Kτ.

−10 −5 5 10 1 2 3 4 5 6

K in dB Error Exponent

λ → ∞ λ = 0 λ = √ 2 Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 48 / 59

slide-124
SLIDE 124

Design for Energy Constrained Inference

Energy constrained network for inference λ∗

∆

= arg max

λ>0 D(λ, K; g)

subject to ¯ E ≤ ¯ Emax Energy and performance scaling laws: K ∆ = σ2

1

σ2

Energy vs. λ

E

Density λ

Exponent vs. λ

Low var. independent correlated High var K < Kτ λ λ K > Kτ

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 49 / 59

slide-125
SLIDE 125

Design for Energy Constrained Inference

Optimal density

? Optimal density

Kτ =

g(0)2 log(1−g(0)2) 2 1−g(0)2

Kτ K′

τ

+∞

λE λ1 λ2

Variance ratio K

DFMRF Optimal fusion Lower bound Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 50 / 59

slide-126
SLIDE 126

Design for Energy Constrained Inference

Optimal density

? Optimal density

Kτ =

g(0)2 log(1−g(0)2) 2 1−g(0)2

Kτ K′

τ

+∞

λE λ1 λ2

Variance ratio K

DFMRF Optimal fusion Lower bound

Deployment implications

Back

λ∗ → 0 λ∗ → ∞

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 50 / 59

slide-127
SLIDE 127

Stages of LLR Computation: LG(Yn) =

c∈C

Φc(Yc)

Forwarding graph Dependency graph Aggregation graph Processor Fusion center

Recall FG

∆

={π : LG(YV) computable at the fusion center} E(π∗

n) = min π∈FG

  • i

Ei(πn)

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 51 / 59

slide-128
SLIDE 128

Stages of LLR Computation: LG(Yn) =

c∈C

Φc(Yc)

Forwarding graph Dependency graph Aggregation graph Processor Fusion center

Recall FG

∆

={π : LG(YV) computable at the fusion center} E(π∗

n) = min π∈FG

  • i

Ei(πn)

Local Computation of Clique Potentials: Processor is a Clique Member

Simplifies optimization problem Local knowledge of function parameters

c1

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 51 / 59

slide-129
SLIDE 129

Stages of LLR Computation: LG(Yn) =

c∈C

Φc(Yc)

Forwarding graph Dependency graph Aggregation graph Processor Fusion center

Recall FG

∆

={π : LG(YV) computable at the fusion center} E(π∗

n) = min π∈FG

  • i

Ei(πn)

Local Computation of Clique Potentials: Processor is a Clique Member

Simplifies optimization problem Local knowledge of function parameters

c1

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 51 / 59

slide-130
SLIDE 130

Stages of LLR Computation: LG(Yn) =

c∈C

Φc(Yc)

Forwarding graph Dependency graph Aggregation graph Processor Fusion center

Recall FG

∆

={π : LG(YV) computable at the fusion center} E(π∗

n) = min π∈FG

  • i

Ei(πn)

Local Computation of Clique Potentials: Processor is a Clique Member

Simplifies optimization problem Local knowledge of function parameters

c1

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 51 / 59

slide-131
SLIDE 131

Stages of LLR Computation: LG(Yn) =

c∈C

Φc(Yc)

Forwarding graph Dependency graph Aggregation graph Processor Fusion center

Recall FG

∆

={π : LG(YV) computable at the fusion center} E(π∗

n) = min π∈FG

  • i

Ei(πn)

Local Computation of Clique Potentials: Processor is a Clique Member

Simplifies optimization problem Local knowledge of function parameters

c1

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 51 / 59

slide-132
SLIDE 132

Steiner-Tree Reduction

Steiner Tree

Minimum cost tree containing a required set of nodes called terminals NP-hard problem, currently the best approximation is 1.55

Terminals

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 52 / 59

slide-133
SLIDE 133

Steiner-Tree Reduction

Steiner Tree

Minimum cost tree containing a required set of nodes called terminals NP-hard problem, currently the best approximation is 1.55

Terminals

Main result

Min cost fusion has approx. ratio preserving Steiner tree reduction

Implications

Any approximation for Steiner tree has same ratio for fusion Best approximation for min cost fusion: 1.55

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 52 / 59

slide-134
SLIDE 134

Example : Chain dependency graph

1 2 3 4

Fusion Center Graph transformation and building Steiner tree.

Back Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 53 / 59

slide-135
SLIDE 135

Example : Chain dependency graph

1 2 3 4

Fusion Center Graph transformation and building Steiner tree.

Back Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 53 / 59

slide-136
SLIDE 136

Example : Chain dependency graph

1 2 3 4

Fusion Center v12 v23 v34 Graph transformation and building Steiner tree.

Back Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 53 / 59

slide-137
SLIDE 137

Example : Chain dependency graph

1 2 3 4

Fusion Center v12 v23 v34 Graph transformation and building Steiner tree.

Back Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 53 / 59

slide-138
SLIDE 138

Example : Chain dependency graph

1 2 3 4

Fusion Center v12 v23 v34 Graph transformation and building Steiner tree.

Back Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 53 / 59

slide-139
SLIDE 139

Example : Chain dependency graph

1 2 3 4

Fusion Center v12 v23 v34 Graph transformation and building Steiner tree.

Back Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 53 / 59

slide-140
SLIDE 140

Example : Chain dependency graph

1 2 3 4

Fusion Center v12 v23 v34 Graph transformation and building Steiner tree.

Back Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 53 / 59

slide-141
SLIDE 141

Example : Chain dependency graph

1 2 3 4

Fusion Center v12 v23 v34 Graph transformation and building Steiner tree.

Back Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 53 / 59

slide-142
SLIDE 142

Example : Chain dependency graph

Proc(12) Proc(23) Proc(34)

1 2 3 4

v12 v23 v34 Graph transformation and building Steiner tree.

Back Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 53 / 59

slide-143
SLIDE 143

Example : Chain dependency graph

FG FG FG

1 2 3 4

v12 v23 v34 Graph transformation and building Steiner tree.

Back Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 53 / 59

slide-144
SLIDE 144

Example : Chain dependency graph

FG FG FG AG AG

1 2 3 4

Graph transformation and building Steiner tree.

Back Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 53 / 59

slide-145
SLIDE 145

Example : Chain dependency graph

1 2 3 4

Y1 Y2 Y3 Graph transformation and building Steiner tree.

Back Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 53 / 59

slide-146
SLIDE 146

Example : Chain dependency graph

1 2 3 4

Φ(Y1, Y2) Graph transformation and building Steiner tree.

Back Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 53 / 59

slide-147
SLIDE 147

Example : Chain dependency graph

1 2 3 4

Φ(Y1, Y2) + Φ(Y2, Y3) Graph transformation and building Steiner tree.

Back Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 53 / 59

slide-148
SLIDE 148

Example : Chain dependency graph

1 2 3 4

LLR

Graph transformation and building Steiner tree.

Back Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 53 / 59

slide-149
SLIDE 149

Optimal Cost-Performance Tradeoff

Problem Statement

Select Vs ⊂ V and design a fusion scheme Γ(Vs). Minimize the total routing costs C(Γ(Vs)) plus a penalty π based on the error prob. PM(Vs). π(V \Vs)∆ = log PM(Vs) PM(V ) > 0

  • Fusion policy graph

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 54 / 59

slide-150
SLIDE 150

Optimal Cost-Performance Tradeoff

Problem Statement

Select Vs ⊂ V and design a fusion scheme Γ(Vs). Minimize the total routing costs C(Γ(Vs)) plus a penalty π based on the error prob. PM(Vs). π(V \Vs)∆ = log PM(Vs) PM(V ) > 0

  • Fusion policy graph

min

Vs⊂V,Γ(Vs)

  • C(Γ(Vs)) + µπ(V \Vs)
  • , µ > 0

Prize-Collecting Data Fusion

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 54 / 59

slide-151
SLIDE 151

Main Results

min

Vs⊂V,Γ(Vs)

  • C(Γ(Vs))+µ log PM(Vs)

PM(V ) )

  • , µ > 0

IID measurements

2 − (|V | − 1)−1 approximation via Prize-Collecting Steiner Tree

  • PCST

Correlated data: component and clique selection heuristics

Provable approximation guarantee for special dependency graphs. Substantially better than no data fusion. Performance under different node placements.

Back Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 55 / 59

slide-152
SLIDE 152

PCDF: IID case

min

Vs⊂V,Γ(Vs)

  • C(Γ(Vs)) + µ log PM(Vs)

PM(V )

  • , µ > 0

Simplifications of IID measurements

Hk : YV ∼

i∈V

fk(Yi) LG(YVs) =

i∈Vs

log f(Yi;H0)

f(Yi;H1) = i∈Vs

LG(Yi) Error exponent D = D(f(Y ; H0)||f(Y ; H1))

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 56 / 59

slide-153
SLIDE 153

PCDF: IID case

min

Vs⊂V,Γ(Vs)

  • C(Γ(Vs)) + µ log PM(Vs)

PM(V )

  • , µ > 0

Simplifications of IID measurements

Hk : YV ∼

i∈V

fk(Yi) LG(YVs) =

i∈Vs

log f(Yi;H0)

f(Yi;H1) = i∈Vs

LG(Yi) Error exponent D = D(f(Y ; H0)||f(Y ; H1))

Modified cost-performance tradeoff for IID

min

Vs⊂V,Γ(Vs)

  • C(Γ(Vs)) + µ[|V | − |Vs|]D
  • Asymptotic convergence to the original problem.

The optimal solution is the Prize Collecting Steiner Tree.

Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 56 / 59

slide-154
SLIDE 154

Prize Collecting Steiner Tree (PCST)

Definition

Tree with minimum sum edge costs plus node penalties not spanned T∗ = arg min

T=(V ′,E′)

  • e∈E′

ce+

  • i/

∈V ′

πi

  • .

NP-hard, Goemans-Williamson algorithm has approx. ratio of 2 −

1 |V |−1

  • Approx. PCST

Back Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 57 / 59

slide-155
SLIDE 155

Prize Collecting Steiner Tree (PCST)

Definition

Tree with minimum sum edge costs plus node penalties not spanned T∗ = arg min

T=(V ′,E′)

  • e∈E′

ce+

  • i/

∈V ′

πi

  • .

NP-hard, Goemans-Williamson algorithm has approx. ratio of 2 −

1 |V |−1

  • Fusion of IID measurements

q1 = LG(Y1) q2 = LG(Y2) LG(YVs) =

  • i∈Vs

LG(Yi)

Back Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 57 / 59

slide-156
SLIDE 156

Prize Collecting Steiner Tree (PCST)

Definition

Tree with minimum sum edge costs plus node penalties not spanned T∗ = arg min

T=(V ′,E′)

  • e∈E′

ce+

  • i/

∈V ′

πi

  • .

NP-hard, Goemans-Williamson algorithm has approx. ratio of 2 −

1 |V |−1

  • Fusion of IID measurements

q3 = LG(Y3) +

2

  • i=1

qi LG(YVs) =

  • i∈Vs

LG(Yi)

Back Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 57 / 59

slide-157
SLIDE 157

Prize Collecting Steiner Tree (PCST)

Definition

Tree with minimum sum edge costs plus node penalties not spanned T∗ = arg min

T=(V ′,E′)

  • e∈E′

ce+

  • i/

∈V ′

πi

  • .

NP-hard, Goemans-Williamson algorithm has approx. ratio of 2 −

1 |V |−1

  • Fusion of IID measurements

q4 = LG(Y4) LG(YVs) =

  • i∈Vs

LG(Yi)

Back Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 57 / 59

slide-158
SLIDE 158

Medium Access Control (MAC) For Inference

Base Station (Fusion Center)

Design of MAC (Single Hop)

Back (1) A. Anandkumar and L. Tong,“Type-based Random Access for Distributed Detection over Multi-access Fading Channels,” IEEE Tran. on Signal Processing, vol.55, no.10, pp.5032-5043, Oct. 2007 (2008 IEEE SPS Young Author Best Paper Award) (2) A. Anandkumar, L. Tong and A. Swami,“ Distributed Estimation via Random Access,” in IEEE Tran. on Information Theory, vol. 54, pp. 3175-3181, July 2008. Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 58 / 59

slide-159
SLIDE 159

Medium Access Control (MAC) For Inference

Base Station (Fusion Center)

Design of MAC (Single Hop)

Back (1) A. Anandkumar and L. Tong,“Type-based Random Access for Distributed Detection over Multi-access Fading Channels,” IEEE Tran. on Signal Processing, vol.55, no.10, pp.5032-5043, Oct. 2007 (2008 IEEE SPS Young Author Best Paper Award) (2) A. Anandkumar, L. Tong and A. Swami,“ Distributed Estimation via Random Access,” in IEEE Tran. on Information Theory, vol. 54, pp. 3175-3181, July 2008. Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 58 / 59

slide-160
SLIDE 160

Medium Access Control (MAC) For Inference

Base Station (Fusion Center)

Design of MAC (Single Hop)

Classical Design

Orthogonal Division

Proposed Design

Type-Based Random Access Sensor encoding based on data level Optimal spatio-temporal allocation based on channel conditions

Back (1) A. Anandkumar and L. Tong,“Type-based Random Access for Distributed Detection over Multi-access Fading Channels,” IEEE Tran. on Signal Processing, vol.55, no.10, pp.5032-5043, Oct. 2007 (2008 IEEE SPS Young Author Best Paper Award) (2) A. Anandkumar, L. Tong and A. Swami,“ Distributed Estimation via Random Access,” in IEEE Tran. on Information Theory, vol. 54, pp. 3175-3181, July 2008. Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 58 / 59

slide-161
SLIDE 161

Inference of Transaction Paths in Distributed Systems

Transactions & Log Records

Back (1) A. Anandkumar, C. Bisdikian, and D. Agrawal, Tracking in a Spaghetti Bowl: Monitoring Transactions Using Footprints, in

  • Proc. ACM Intl. Conf. on Measurement & Modeling of Computer Systems (Sigmetrics), June 2008

(2) A. Anandkumar, C. Bisdikian, T. He, and D. Agrawal, Designing A Fine Tooth Comb Frugally: Selectively Retrofitting Monitoring in Distributed Systems. Workshop on Mathematical Performance Modeling and Analysis June, 2009. Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 59 / 59

slide-162
SLIDE 162

Inference of Transaction Paths in Distributed Systems

Transactions & Log Records State Transition Model

1 2 3 4 5 2 1 3 5 4 ({1,3},{2,4,5}) 2 3 ({2},{3})

Maximum Likelihood Tracking ≡ Series of Bipartite Matches

Back (1) A. Anandkumar, C. Bisdikian, and D. Agrawal, Tracking in a Spaghetti Bowl: Monitoring Transactions Using Footprints, in

  • Proc. ACM Intl. Conf. on Measurement & Modeling of Computer Systems (Sigmetrics), June 2008

(2) A. Anandkumar, C. Bisdikian, T. He, and D. Agrawal, Designing A Fine Tooth Comb Frugally: Selectively Retrofitting Monitoring in Distributed Systems. Workshop on Mathematical Performance Modeling and Analysis June, 2009. Anima Anandkumar (Cornell) Scaling Laws B-exam 05/26/09 59 / 59