Inference and Signal Processing for Networks ALFRED O. HERO III - - PowerPoint PPT Presentation

▶
inference and signal processing for networks
SMART_READER_LITE
LIVE PREVIEW

Inference and Signal Processing for Networks ALFRED O. HERO III - - PowerPoint PPT Presentation

Nov 15, 2023 449 likes •678 views

Inference and Signal Processing for Networks ALFRED O. HERO III Depts. EECS, BME, Statistics University of Michigan - Ann Arbor http://www.eecs.umich.edu/~hero Students : Clyde Shih , Jose Costa Neal Patwari, Derek Justice, David Barsic Eric

slide-1
SLIDE 1

WISP: Nov. 04

Inference and Signal Processing for Networks

ALFRED O. HERO III

  • Depts. EECS, BME, Statistics

University of Michigan - Ann Arbor http://www.eecs.umich.edu/~hero

Outline

1. Dealing with the data cube 2. Challenges in multi-site Internet data analysis 3. Dimension reduction approaches 4. Conclusion

Students: Clyde Shih, Jose Costa Neal Patwari, Derek Justice, David Barsic

Eric Cheung, Adam Pocholski, Panna Felsen

slide-2
SLIDE 2

WISP: Nov. 04

My Current Research Areas

  • Dimension reduction, manifold learning and clustering

– Information theoretic dimensionality reduction (Costa) – Information theoretic graph approaches to clustering and classification (Costa)

  • Ad hoc networks

– Distributed detection and node-localization in wireless sensor nets (Costa, Patwari) – Distributed optimization and distributed detection (Blatt, Patwari)

  • Administered networks

– Spatio-temporal Internet traffic analysis (Patwari) – Tomography (Shih) – Topology discovery (Shih, Justice)

  • Adaptive resource allocation and scheduling in networks

– Sensor management for tracking multiple targets (Kreucher) – Sensor management for acquiring smart targets (Blatt)

  • Inference on gene regulation networks

– Gene and gene pair filtering and ranking (Jing, Fleury) – Confident discovery of dependency networks (Zhu)

  • Imaging

– Image and volume registration (Neemuchwala) – Tomographic reconstruction from projections in medical imaging (Fessler) – Quantum imaging, computational microscopy and MRFM (Ting) – Multi-static radar imaging with adaptive waveform diversity (Raich, Rangajaran)

slide-3
SLIDE 3

WISP: Nov. 04

Applications

  • Characterization of face manifolds (Costa)

– The set of face images evolve on a lower dimensional imbedded manifold in 128x128 =16384 dimensions

  • Handwriting (Costa) - Pattern Matching(Neemuchwala)
slide-4
SLIDE 4

WISP: Nov. 04

Clustering and classification (Costa)

Case 141

Ultrasound Breast Registration (Neemuchwala)

x y

Adaptive scheduling of measurements (Kreucher)

Applications

Gene microarray analysis (Zhu)

slide-5
SLIDE 5

WISP: Nov. 04

  • 1. Dealing with the data cube

D e s t i n a t i

  • n

I P

Source IP Port

yt,l (pi,di,si)

Single measurement site (router) Ports, applications, protocols > dozens of dimensions

slide-6
SLIDE 6

WISP: Nov. 04

SNVA STTL LOSA KSCY HSTN DNVR CHIN IPLS ATLA WASH NYCM

Dealing with the data cube

Multiple measurement sites (Abilene)

slide-7
SLIDE 7

WISP: Nov. 04

Multisite Analysis GUI (Patwari, Felsen)

Source: Felsen, Pacholski

slide-8
SLIDE 8

WISP: Nov. 04

  • 2. Internet SP Challenges
  • What makes multisite Internet data analysis hard from a SP

point of view?

– Bandwidth is always limited – Sampling will never be adequate

  • Spatial sampling: cannot measure all link/node correlations from passive

measurements at only a few sites

  • Temporal sampling: full bit stream cannot be captured
  • Category sampling: only a subset of all field variables can be monitored at

a time

– Measurement data is inherently non-stationary – Standard modeling approaches are difficult or inapplicable for such massive data sets – Little ground truth data is available to validate models

  • General robust and principled approach is needed:

– Adopt hierarchical multiresolution modeling and analysis framework – Task-driven dimension reduction

slide-9
SLIDE 9

WISP: Nov. 04

Hierarchical Network Measurement Framework

query

Global Network Diagnoser

DAFM DAFM DAFM DAFM DAFM DAFM AS Router LAN

report

Level 1 Level 2 Level 3

Spatio-temporal models and systems

  • Feature extraction
  • Dimension reduction
  • Tomography
  • On-line traffic analysis

Event-driven models

  • Modular diagnosis
  • Active querying
  • Distributed detection

Data Measurement and Collection

Legend: DAFM - Data aggregation and filtering module AS – Autonomous System LAN – Local Area Network

slide-10
SLIDE 10

WISP: Nov. 04

Example: distributed anomaly detection

  • Detection performance can be

close to optimal [1]

– Even ρ = 0.01 sensors greatly improve performance

Environment

yi

( ) 3

yi

( ) 2

yi

( ) 1

> <

λ1

Do not send send

Local LRT

λ3

Do not send send

< >

λ2

Do not send send

Local LRT Local LRT

Sensor 3

yi

( ) 7

< >λ7

Decide H0 Decide H1

Global LRT

Sensor 7 < >

yi

( ) 3

yi

( ) 2

yi

( ) 1

> <

λ1

Do not send send

Local LRT

λ3

Do not send send

< >

λ2

send

Local LRT Local LRT

Sensor 3 < >

Do not send

∀ ρ = 1 ↔ centralized

  • 0 < ρ < 1 ↔ data fusion,

reduce data bottleneck at the root

  • Multi-hop is desirable for energy

efficiency, cost

  • Censored test can be iterated to

match arbitrary multi-hop ‘tree’ hierarchy

[1]

  • N. Patwari, A.O. Hero III, “Hierarchical Censoring for Distributed Detection in Wireless Sensor

Networks”, IEEE ICASSP ’03, April 2003.

slide-11
SLIDE 11

WISP: Nov. 04

Example: distributed anomaly detection

ρ1

– Parameter selected to constrain mean time btwn false alarms

3 6 4 5 1 2 7

Level 3 Level 2 Level 1

ρ2

slide-12
SLIDE 12

WISP: Nov. 04

Research Issues

  • Broad questions

– Anomaly detection, classification, and localization

  • Model-driven vs data-driven approaches
  • Partitioning of information and decisionmaking (Multiscale-

multiresolution decision trees)

  • Learning the “Baseline” and detecting deviations
  • Feature selection, updating, and validation

– Multi-site measurement and aggregation

  • Remote monitoring: tomography and topology discovery
  • Multi-site spatio-temporal correlation
  • Distributed optimization/computation

– Dynamic spatio-temporal measurement

  • Sensor management: scheduling measurements and communication
  • Passive sensing vs. active probing
  • Adaptive spatio-temporal resolution control

– Dimension reduction methods

  • Beyond linear PCA/ICA/MDS…
slide-13
SLIDE 13

WISP: Nov. 04

  • 3. Dimension Reduction
  • Manifold domain reconstruction from samples: “the data manifold”

– Linearity hypothesis: PCA, ICA, multidimensional scaling (MDS) – Smoothness hypothesis: ISOMAP, LLE, HLLE

  • Dimension estimation: infer degrees of freedom of data manifold
  • Infer entropy, relative entropy of sampling distribution on manifold

g(zi)g(zk)

zi

.. . .

zk zi zk

g(zi) g(zk)

slide-14
SLIDE 14

WISP: Nov. 04

Application: Internet Traffic Visualization

  • Spatio-temporal measurement vector:

day tempera ture

temperat ure day

day temperat ure

slide-15
SLIDE 15

WISP: Nov. 04

Key problem: dimension estimation

Residual fitting curves for 11x21 = 231 dimensional Abilene Netflow data set

2 4 6 8 10 12 14 16 18 20 0.005 0.01 0.015 R e s i d u a l v a r i a n c e Isomap dimensionality Residual variance vs dimentionality- Data Set 1

ISOMAP residual curve for 41+11=51 dimensional Abilene OD link data (Lakhina,Crovella, Diot)

slide-16
SLIDE 16

WISP: Nov. 04

GMST Rate of convergence=dimension, entropy

n=400 n=800

Rate of increase in length functional of MST should be related to the intrinsic dimension of data manifold

slide-17
SLIDE 17

WISP: Nov. 04

BHH Theorem

Extended BHH Theorem (Costa&Hero):

slide-18
SLIDE 18

WISP: Nov. 04

Application: ISOMAP Database

  • http://isomap.stanford.edu/datasets.html
  • Synthesized 3D face surface
  • Computer generated images representing

700 different angles and illuminations

  • Subsampled to 64 x 64 resolution (D=4096)
  • Disagreement over intrinsic dimensionality

– d=3 (Tenenbaum) vs d=4 (Kegl)

Resampling Histogram of d hat Mean GMST Length Function d=3 H=21.1 bits

slide-19
SLIDE 19

WISP: Nov. 04

Illustration: Abilene Netflow

  • 11 routers and 21 applications = each sample lives in 231

dimensions

  • 24 hour data block divided into 5 min intervals = 288 samples

d=5 H=98.12 bits

Mean GMST Length Function Resampling histogram of d hat

slide-20
SLIDE 20

WISP: Nov. 04

dwMDS embedding/visualization

Data: total packet flow over 5 minute intervals 10 june ’04 Isomap(Tennbaum): k=3, 2D projection, L2 distances DW MDS(Costa&Patwari&Hero): k=5, 2D projection, L2 distances Abilene Network Isomap (Centralized computation) Abilene Network DW MDS (Distributed computation)

slide-21
SLIDE 21

WISP: Nov. 04

dwMDS embedding/visualization

Data: total packet flow over 5 minute intervals 10 june ’04 MDS: 2D projection, L2 distances Abilene Network MDS (linear) (Centralized computation)

slide-22
SLIDE 22

WISP: Nov. 04

  • 4. Conclusions
  • Interface of SP, control, info theory, statistics and applied

math is fertile ground for network measurement/data analysis

  • SP will benefit from scalable hierarchical multiresolution

modeling and analysis framework

– Multiresolution modeling, communication, decisionmaking

  • Task-driven dimension reduction is necessary

– Go beyond linear methods (PCA/ICA)

  • What is goal? Estimation/Detection/Classification?
  • Subspace constraints (smoothness, anchors)?
  • Out-of-sample updates?
  • Mixed dimensions?
  • Validation is a critical problem: annotated classified data or

ground truth data is lacking.