Error Exponents for Composite Hypothesis Testing of Markov Forest - - PowerPoint PPT Presentation

Error Exponents for Composite Hypothesis Testing

• f Markov Forest Distributions

Vincent Tan, Anima Anandkumar, Alan S. Willsky

Stochastic Systems Group, Laboratory for Information and Decision Systems, Massachusetts Institute of Technology

ISIT (Jun 18, 2010)

1/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 1 / 17

Motivation

Continuation of line of work on error exponents for learning tree-structured graphical models:

Discrete Case: Tan, Anandkumar, Tong, Willsky, ISIT 2009.

2/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 2 / 17

Motivation

Continuation of line of work on error exponents for learning tree-structured graphical models:

Discrete Case: Tan, Anandkumar, Tong, Willsky, ISIT 2009. Gaussian Case: Tan, Anandkumar, Willsky, Trans. SP 2010.

2/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 2 / 17

Motivation

Continuation of line of work on error exponents for learning tree-structured graphical models:

Discrete Case: Tan, Anandkumar, Tong, Willsky, ISIT 2009. Gaussian Case: Tan, Anandkumar, Willsky, Trans. SP 2010.

Instead of learning, we instead focus on hypothesis testing.

2/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 2 / 17

Motivation

Continuation of line of work on error exponents for learning tree-structured graphical models:

Discrete Case: Tan, Anandkumar, Tong, Willsky, ISIT 2009. Gaussian Case: Tan, Anandkumar, Willsky, Trans. SP 2010.

Instead of learning, we instead focus on hypothesis testing. Provides intuition for which classes of graphical models are easy for learning in terms of the detection error exponent.

2/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 2 / 17

Motivation

Continuation of line of work on error exponents for learning tree-structured graphical models:

Discrete Case: Tan, Anandkumar, Tong, Willsky, ISIT 2009. Gaussian Case: Tan, Anandkumar, Willsky, Trans. SP 2010.

Instead of learning, we instead focus on hypothesis testing. Provides intuition for which classes of graphical models are easy for learning in terms of the detection error exponent. Is there a relation between the detection error exponent and the exponent associated to structure learning?

2/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 2 / 17

Background on Tree-Structured Graphical Models

Graphical model: family of multivariate probability distributions that factorize according to a given graph G = (V, E). Vertices in the set V = {1, . . . , d} correspond to variables and edges in E ⊂ V

2

• to conditional independences.

3/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 3 / 17

Background on Tree-Structured Graphical Models

Graphical model: family of multivariate probability distributions that factorize according to a given graph G = (V, E). Vertices in the set V = {1, . . . , d} correspond to variables and edges in E ⊂ V

2

• to conditional independences.

Example for tree-structured P(x) with d = 4.

3/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 3 / 17

Background on Tree-Structured Graphical Models

Graphical model: family of multivariate probability distributions that factorize according to a given graph G = (V, E). Vertices in the set V = {1, . . . , d} correspond to variables and edges in E ⊂ V

2

• to conditional independences.

Example for tree-structured P(x) with d = 4.

✉ ✉ ✉ ✉ ❅ ❅ ❅ ❅

• X4

X1 X3 X2

3/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 3 / 17

Background on Tree-Structured Graphical Models

Graphical model: family of multivariate probability distributions that factorize according to a given graph G = (V, E). Vertices in the set V = {1, . . . , d} correspond to variables and edges in E ⊂ V

2

• to conditional independences.

Example for tree-structured P(x) with d = 4.

✉ ✉ ✉ ✉ ❅ ❅ ❅ ❅

• X4

X1 X3 X2 P(x1, x2, x3, x4) = P1(x1) × P1,2(x1, x2) P1(x1) × P1,3(x1, x3) P1(x1) × P1,4(x1, x4) P1(x1) .

3/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 3 / 17

Learning vs Hypothesis Testing

Canonical Problem: Given x1, . . . , xn ∼ P, learn structure of P.

4/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 4 / 17

Learning vs Hypothesis Testing

Canonical Problem: Given x1, . . . , xn ∼ P, learn structure of P. If P is a tree, can use Chow and Liu (1968) as an efficient implementation of ML.

4/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 4 / 17

Learning vs Hypothesis Testing

Canonical Problem: Given x1, . . . , xn ∼ P, learn structure of P. If P is a tree, can use Chow and Liu (1968) as an efficient implementation of ML. Denote set of distributions Markov on a tree T0 ∈ T as D(T0). Set

• f distributions Markov on any tree is D(T ).

4/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 4 / 17

Learning vs Hypothesis Testing

Canonical Problem: Given x1, . . . , xn ∼ P, learn structure of P. If P is a tree, can use Chow and Liu (1968) as an efficient implementation of ML. Denote set of distributions Markov on a tree T0 ∈ T as D(T0). Set

• f distributions Markov on any tree is D(T ).

Composite hypothesis testing problem considered here: H0 : x1, . . . , xn ∼ Λ0 ⊂ D(T ) H1 : x1, . . . , xn ∼ Λ1 ⊂ D(T ) Λi closed and Λ0 ∩ Λ1 = ∅.

4/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 4 / 17

Definition of Worst-Case Type-II Error Exponent

Neyman-Pearson setup. Acceptance regions (An).

5/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 5 / 17

Definition of Worst-Case Type-II Error Exponent

Neyman-Pearson setup. Acceptance regions (An). Def: Type-II error exponent for a fixed Q ∈ Λ1 given (An): J(Λ0, Q; An) := lim inf

n→∞ −1

n log Qn(An)

5/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 5 / 17

Definition of Worst-Case Type-II Error Exponent

Neyman-Pearson setup. Acceptance regions (An). Def: Type-II error exponent for a fixed Q ∈ Λ1 given (An): J(Λ0, Q; An) := lim inf

n→∞ −1

n log Qn(An) Def: Optimal Type-II error exponent J∗(Λ0, Q) := sup

An:Pn(An)≤α,∀P∈Λ0

J(Λ0, Q; An)

5/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 5 / 17

Definition of Worst-Case Type-II Error Exponent

Neyman-Pearson setup. Acceptance regions (An). Def: Type-II error exponent for a fixed Q ∈ Λ1 given (An): J(Λ0, Q; An) := lim inf

n→∞ −1

n log Qn(An) Def: Optimal Type-II error exponent J∗(Λ0, Q) := sup

An:Pn(An)≤α,∀P∈Λ0

J(Λ0, Q; An) Def: Worst-Case Optimal Type-II error exponent J∗(Λ0, Λ1) := inf

Q∈Λ1 J∗(Λ0, Q)

5/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 5 / 17

Definition of Worst-Case Type-II Error Exponent

Neyman-Pearson setup. Acceptance regions (An). Def: Type-II error exponent for a fixed Q ∈ Λ1 given (An): J(Λ0, Q; An) := lim inf

n→∞ −1

n log Qn(An) Def: Optimal Type-II error exponent J∗(Λ0, Q) := sup

An:Pn(An)≤α,∀P∈Λ0

J(Λ0, Q; An) Def: Worst-Case Optimal Type-II error exponent J∗(Λ0, Λ1) := inf

Q∈Λ1 J∗(Λ0, Q)

Optimizing distribution Q∗ called the least favorable distribution.

5/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 5 / 17

Why Difficult?

6/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 6 / 17

Why Difficult?

Many trees: If there are d nodes, there are dd−2 trees! Searching for the dominant error event may be intractable.

6/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 6 / 17

Why Difficult?

Many trees: If there are d nodes, there are dd−2 trees! Searching for the dominant error event may be intractable. Natural Questions: Any closed-form expressions for the worst-case error exponent for special Λ0, Λ1? How does this depend on the true distribution?

6/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 6 / 17

Why Difficult?

Many trees: If there are d nodes, there are dd−2 trees! Searching for the dominant error event may be intractable. Natural Questions: Any closed-form expressions for the worst-case error exponent for special Λ0, Λ1? How does this depend on the true distribution? Connections to learning? Intuition and characterization of the least favorable distribution?

6/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 6 / 17

A Simplification

Assume that H0 is simple and P is Markov on T0 = (V, E0). H0 : x1, . . . , xn ∼ {P} H1 : x1, . . . , xn ∼ Λ1 = D(T ) \ D(T0)

7/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 7 / 17

A Simplification

Assume that H0 is simple and P is Markov on T0 = (V, E0). H0 : x1, . . . , xn ∼ {P} H1 : x1, . . . , xn ∼ Λ1 = D(T ) \ D(T0)

✂ ✂ ✂

Λ1 = D(T ) \ D(T0) D(T0) P

✉

J∗(P) Q∗

✉

J∗(P) := J∗({P}, D(T ) \ D(T0))

7/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 7 / 17

Setup for Main Result

For a non-edge e′ = (i, j), let Path(e′) be the unique path joining i and j. Let L(i, j) be the number of hops between i and j.

8/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 8 / 17

Setup for Main Result

For a non-edge e′ = (i, j), let Path(e′) be the unique path joining i and j. Let L(i, j) be the number of hops between i and j.

✇ ✇ ✇ ✇ ✇

• ❅

❅ ❅ ❅ ❅

i j k e′ =(i, j) e1 =(i, k) e2 =(k, j) l

8/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 8 / 17

Setup for Main Result

For a non-edge e′ = (i, j), let Path(e′) be the unique path joining i and j. Let L(i, j) be the number of hops between i and j.

✇ ✇ ✇ ✇ ✇

• ❅

❅ ❅ ❅ ❅

i j k e′ =(i, j) e1 =(i, k) e2 =(k, j) l

• ❅

❅ ❅ ❅ ❅ Figure: Path(e′) = {(i, k), (k, j)}

8/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 8 / 17

Setup for Main Result

For a non-edge e′ = (i, j), let Path(e′) be the unique path joining i and j. Let L(i, j) be the number of hops between i and j.

✇ ✇ ✇ ✇ ✇

• ❅

❅ ❅ ❅ ❅

i j k e′ =(i, j) e1 =(i, k) e2 =(k, j) l

• ❅

❅ ❅ ❅ ❅ Figure: Path(e′) = {(i, k), (k, j)}

Mutual information of joint distribution Pe = Pi,j denoted as I(Pe).

8/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 8 / 17

Main Result

Proposition J∗(P) = min

e′=(i,j)/ ∈E0 L(i,j)=2

min

e∈Path(e′) {I(Pe) − I(Pe′)},

9/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 9 / 17

Main Result

Proposition J∗(P) = min

e′=(i,j)/ ∈E0 L(i,j)=2

min

e∈Path(e′) {I(Pe) − I(Pe′)},

Illustration:

✇ ✇ ✇ ✇

• ❅

❅ ❅ ❅ ❅ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣♣♣♣♣♣♣♣♣♣♣♣♣♣♣♣♣♣♣♣♣♣♣♣♣♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣

X4 X1 X3 X2

5 6 4 1.5 3.9 2

9/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 9 / 17

Main Result

Proposition J∗(P) = min

e′=(i,j)/ ∈E0 L(i,j)=2

min

e∈Path(e′) {I(Pe) − I(Pe′)},

Illustration:

✇ ✇ ✇ ✇

• ❅

❅ ❅ ❅ ❅ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣♣♣♣♣♣♣♣♣♣♣♣♣♣♣♣♣♣♣♣♣♣♣♣♣♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣

X4 X1 X3 X2

5 6 4 1.5 3.9 2

❆ ❆ ❆ ❆ ❆ ❆ ❆ ❆ ❆ ❆

3.9 4

9/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 9 / 17

Main Result

Proposition J∗(P) = min

e′=(i,j)/ ∈E0 L(i,j)=2

min

e∈Path(e′) {I(Pe) − I(Pe′)},

Illustration:

✇ ✇ ✇ ✇

• ❅

❅ ❅ ❅ ❅ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣♣♣♣♣♣♣♣♣♣♣♣♣♣♣♣♣♣♣♣♣♣♣♣♣♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣ ♣

X4 X1 X3 X2

5 6 4 1.5 3.9 2

❆ ❆ ❆ ❆ ❆ ❆ ❆ ❆ ❆ ❆

3.9 4

J∗(P) = 4 − 3.9 = 0.1

9/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 9 / 17

Least Favorable Distribution

The least favorable distribution Q∗ is characterized by EQ∗ = argmax

E=E0, E acyclic

• e∈E

I(Pe) a second-best max-weight spanning tree problem,

10/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 10 / 17

Least Favorable Distribution

The least favorable distribution Q∗ is characterized by EQ∗ = argmax

E=E0, E acyclic

• e∈E

I(Pe) a second-best max-weight spanning tree problem, and Q∗

i (xi) = Pi(xi),

∀ i ∈ V Q∗

i,j(xi, xj) = Pi,j(xi, xj),

∀ (i, j) ∈ EQ∗

10/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 10 / 17

Proof Outline

Optimization for worst-case exponent is inf

Q∈D(T )\D({T0}) D(Q || P)

11/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 11 / 17

Proof Outline

Optimization for worst-case exponent is inf

Q∈D(T )\D({T0}) D(Q || P) =

min

T∈T \{T0}

• inf

Q∈D(T) D(Q || P)

• 11/17

Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 11 / 17

Proof Outline

Optimization for worst-case exponent is inf

Q∈D(T )\D({T0}) D(Q || P) =

min

T∈T \{T0}

• inf

Q∈D(T) D(Q || P)

• Use tree decomposition (junction tree theorem)

Q(x) =

• i∈V(T)

Qi(xi)

• (i,j)∈E(T)

Qi,j(xi, xj) Qi(xi)Qj(xj)

11/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 11 / 17

Proof Outline

Optimization for worst-case exponent is inf

Q∈D(T )\D({T0}) D(Q || P) =

min

T∈T \{T0}

• inf

Q∈D(T) D(Q || P)

• Use tree decomposition (junction tree theorem)

Q(x) =

• i∈V(T)

Qi(xi)

• (i,j)∈E(T)

Qi,j(xi, xj) Qi(xi)Qj(xj) Emulate Chow and Liu (1968).

11/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 11 / 17

Proof Outline

Optimization for worst-case exponent is inf

Q∈D(T )\D({T0}) D(Q || P) =

min

T∈T \{T0}

• inf

Q∈D(T) D(Q || P)

• Use tree decomposition (junction tree theorem)

Q(x) =

• i∈V(T)

Qi(xi)

• (i,j)∈E(T)

Qi,j(xi, xj) Qi(xi)Qj(xj) Emulate Chow and Liu (1968). Second-best max-weight spanning tree differs from best one by a single edge [Cormen et al. 2003].

11/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 11 / 17

Proof Outline

Optimization for worst-case exponent is inf

Q∈D(T )\D({T0}) D(Q || P) =

min

T∈T \{T0}

• inf

Q∈D(T) D(Q || P)

• Use tree decomposition (junction tree theorem)

Q(x) =

• i∈V(T)

Qi(xi)

• (i,j)∈E(T)

Qi,j(xi, xj) Qi(xi)Qj(xj) Emulate Chow and Liu (1968). Second-best max-weight spanning tree differs from best one by a single edge [Cormen et al. 2003]. Data processing inequality.

11/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 11 / 17

Intuition

J∗(P) = min

e′=(i,j)/ ∈E0 L(i,j)=2

min

e∈Path(e′) {I(Pe) − I(Pe′)},

Smaller the difference between MI on true edge and MI on non-edge (along path), smaller the detection error exponent.

12/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 12 / 17

Intuition

J∗(P) = min

e′=(i,j)/ ∈E0 L(i,j)=2

min

e∈Path(e′) {I(Pe) − I(Pe′)},

Smaller the difference between MI on true edge and MI on non-edge (along path), smaller the detection error exponent. Detection error exponent depends only on bottleneck edges.

12/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 12 / 17

Comparison to Existing Results

J∗(P) = min

e′=(i,j)/ ∈E0 L(i,j)=2

min

e∈Path(e′) {I(Pe) − I(Pe′)},

Intuitive in light of the Chow-Liu algorithm for learning trees. ˆ EML := argmax

E=E0, E acyclic

• e∈E

I(ˆ µe) where ˆ µe is the pairwise type on edge e.

13/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 13 / 17

Comparison to Existing Results

J∗(P) = min

e′=(i,j)/ ∈E0 L(i,j)=2

min

e∈Path(e′) {I(Pe) − I(Pe′)},

Intuitive in light of the Chow-Liu algorithm for learning trees. ˆ EML := argmax

E=E0, E acyclic

• e∈E

I(ˆ µe) where ˆ µe is the pairwise type on edge e. Learning error exponent in very-noisy regime

• K(P) := min

e′ / ∈E0

min

e∈Path(e′)

(I(Pe) − I(Pe′))2 2Var(Se − Se′)

13/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 13 / 17

Comparison to Existing Results

J∗(P) = min

e′=(i,j)/ ∈E0 L(i,j)=2

min

e∈Path(e′) {I(Pe) − I(Pe′)},

Intuitive in light of the Chow-Liu algorithm for learning trees. ˆ EML := argmax

E=E0, E acyclic

• e∈E

I(ˆ µe) where ˆ µe is the pairwise type on edge e. Learning error exponent in very-noisy regime

• K(P) := min

e′ / ∈E0

min

e∈Path(e′)

(I(Pe) − I(Pe′))2 2Var(Se − Se′) J∗(P) and K(P) depend on the difference of mutual informations.

13/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 13 / 17

Performing the Hypothesis Test

Known that the worst-case error exponent is achieved by the Hoeffding Test. But hard to implement for tree distributions.

14/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 14 / 17

Performing the Hypothesis Test

Known that the worst-case error exponent is achieved by the Hoeffding Test. But hard to implement for tree distributions. The generalized likelihood ratio test (GLRT) has acceptance regions An :=

• xn : 1

n log maxQ∈Λ1 Qn(xn) maxP∈Λ0 Pn(xn) ≥ γ

• 14/17

Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 14 / 17

Performing the Hypothesis Test

Known that the worst-case error exponent is achieved by the Hoeffding Test. But hard to implement for tree distributions. The generalized likelihood ratio test (GLRT) has acceptance regions An :=

• xn : 1

n log maxQ∈Λ1 Qn(xn) maxP∈Λ0 Pn(xn) ≥ γ

• When the null hypothesis is simple, the GLRT also simplifies.

14/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 14 / 17

Performing the Hypothesis Test

Known that the worst-case error exponent is achieved by the Hoeffding Test. But hard to implement for tree distributions. The generalized likelihood ratio test (GLRT) has acceptance regions An :=

• xn : 1

n log maxQ∈Λ1 Qn(xn) maxP∈Λ0 Pn(xn) ≥ γ

• When the null hypothesis is simple, the GLRT also simplifies.

H0 : x1, . . . , xn ∼ {P} H1 : x1, . . . , xn ∼ Λ1 = D(T ) \ D(T0)

14/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 14 / 17

The Generalized Likelihood Ratio Test

Denote the joint type of xn as ˆ µ := ˆ µ(· ; xn). Denote the pairwise type on e as ˆ µe. True set of edges: E0. Proposition The GLRT simplifies as An =

• xn :
• e∈E∗

I(ˆ µe) −

• e∈E0

I(ˆ µe) ≥ γ

• where the “dominating edge set” is

E∗ = argmax

E=E0, E acyclic

• e∈E

I(ˆ µe)

15/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 15 / 17

Interpretation and Extensions

Easy to implement the GLRT for testing between trees.

1VTan, A. Anandkumar, A. Willsky “Learning High-Dimensional Markov Forest Distributions:

Analysis of Error Rates”, Submitted to JMLR, May 2010.

16/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 16 / 17

Interpretation and Extensions

Easy to implement the GLRT for testing between trees. Can find the tree structure E∗ efficiently once pairwise types ˆ µe have been computed.

1VTan, A. Anandkumar, A. Willsky “Learning High-Dimensional Markov Forest Distributions:

Analysis of Error Rates”, Submitted to JMLR, May 2010.

16/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 16 / 17

Interpretation and Extensions

Easy to implement the GLRT for testing between trees. Can find the tree structure E∗ efficiently once pairwise types ˆ µe have been computed. Extensions to forest-structured distributions for error exponent and GLRT are straightforward.

1VTan, A. Anandkumar, A. Willsky “Learning High-Dimensional Markov Forest Distributions:

Analysis of Error Rates”, Submitted to JMLR, May 2010.

16/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 16 / 17

Interpretation and Extensions

Easy to implement the GLRT for testing between trees. Can find the tree structure E∗ efficiently once pairwise types ˆ µe have been computed. Extensions to forest-structured distributions for error exponent and GLRT are straightforward. Recent work on high-dimensional learning of forest-structured

• distributions. 1

1VTan, A. Anandkumar, A. Willsky “Learning High-Dimensional Markov Forest Distributions:

Analysis of Error Rates”, Submitted to JMLR, May 2010.

16/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 16 / 17

Concluding Remarks

Analyzed the worst-case type-II error exponent for composite hypothesis testing of Markov forest distributions. Close relations to learning.

17/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 17 / 17

Concluding Remarks

Analyzed the worst-case type-II error exponent for composite hypothesis testing of Markov forest distributions. Close relations to learning. Possible extension 1: Bayesian formulation (Chernoff Information).

17/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 17 / 17

Concluding Remarks

Analyzed the worst-case type-II error exponent for composite hypothesis testing of Markov forest distributions. Close relations to learning. Possible extension 1: Bayesian formulation (Chernoff Information). Possible extension 2: Decomposable graphical models.

17/17 Vincent Tan (SSG, LIDS, MIT) Testing Markov Forest Distributions ISIT 2010 17 / 17