Markov Bases for Logistic Regression Models: A Preliminary Report - - PowerPoint PPT Presentation

Markov Bases for Logistic Regression Models: A Preliminary Report

Dane Wilburne

Illinois Institute of Technology Joint work with: Hisayuki Hara (Niigata University) Loyola University AMS Fall Central Section Meeting Special Session on Algebraic Statistics and its Interactions with Combinatorics, Computation, and Network Science

Chicago, IL October 4, 2015

MB for LR October 4, 2015 0 / 9

Poisson Regression 2 × 2 × L tables

Poisson Regression

xjkl ∼ Po(λjkl), j = 0, 1, k = 0, 1, l = 1, . . . , L log(λjkl) = α + βl + γdj

1 + δdk 1

dj

1 =

• 1,

j = 1 0,

• therwise ,

dk

1 =

• 1,

k = 1 0,

• therwise

l (j, k) 1 2 . . . L (0, 0) ∗ ∗ . . . ∗ (0, 1) ∗ ∗ . . . ∗ (1, 0) ∗ ∗ . . . ∗ (1, 1) ∗ ∗ . . . ∗ Example You own a fishing boat, and you would like to model how many fish are caught with l fisherman in fair/poor weather (dj

1) on a

weekend/weekday (dk

1 ). Record data in a

table such as the one on the left.

MB for LR October 4, 2015 1 / 9

Poisson Regression 2 × 2 × L tables

Poisson Regression

xjkl ∼ Po(λjkl), j = 0, 1, k = 0, 1, l = 1, . . . , L log(λjkl) = α + βl + γdj

1 + δdk 1

dj

1 =

• 1,

j = 1 0,

• therwise ,

dk

1 =

• 1,

k = 1 0,

• therwise

Sufficient statistics: x+++ x1++ x+1+ L

l=1(l · x++l)

Configuration matrix:

B =     1 . . . 1 1 . . . 1 1 . . . 1 1 . . . 1 . . . . . . 1 . . . 1 1 . . . 1 . . . 1 . . . 1 . . . 1 . . . 1 1 . . . L 1 . . . L 1 . . . L 1 . . . L     MB for LR October 4, 2015 1 / 9

Poisson Regression 2 × 2 × . . . × 2 × L tables

Markov bases for Poisson Regression

Proposition (Hara, W., 2015) There exists a Markov basis for the Poisson regression model for 2 × 2 × L tables containing only degree two moves. Example (The case L = 3) The minimal Markov basis for the Poisson regression model for 2 × 2 × 3 tables consists of 33 degree moves of degree two. l (j, k) 1 2 3 (0, 0) 1 (0, 1) −1 (1, 0) −1 (1, 1) 1 Generalization: One way to generalize this model is to increase the number of dummy

• variables. In other words, we now consider the Possion regression model

for 2 × 2 × . . . × 2

• I times

×L tables.

MB for LR October 4, 2015 2 / 9

Poisson Regression 2 × 2 × . . . × 2 × L tables

Markov bases for Poisson Regression

Proposition (Hara, W., 2015) There exists a Markov basis for the Poisson regression model for 2 × 2 × L tables containing only degree two moves. Example (The case L = 3) The minimal Markov basis for the Poisson regression model for 2 × 2 × 3 tables consists of 33 degree moves of degree two. l (j, k) 1 2 3 (0, 0) 1 (0, 1) −1 (1, 0) −1 (1, 1) 1 Proposition (Hara, W. 2015) There exists a Markov basis for the Poisson regression model for 2 × 2 × . . . × 2

• I times

×L tables containing only degree two moves.

MB for LR October 4, 2015 2 / 9

Partition Identities PPIs

Primitive Partition Identities

Definition Fix a positive integer n. A partition identity is an identity of the form: a1 + a2 + . . . + ak = b1 + b2 + . . . + bl, (1) where 0 < ai, bj ≤ n, ai, bj ∈ Z. The quantity k + l is called the degree of the partition identity. We call the partition identity in (1) primitive if there is no proper subidentity ai1 + ai2 + . . . + air = bj1 + bj2 + . . . + bjs, (2) where 1 ≤ r + s ≤ k + l − 1 and we call it homogeneous if k = l. Theorem (Diaconis-Graham-Sturmfels) The degree of any primitive partition identity satisfies k + l ≤ 2n − 1.

MB for LR October 4, 2015 3 / 9

Partition Identities PPIs

Primitive Partition Identities

Example 1 + 2 + 3 = 3 + 3 is a non-primitive partition identity with largest part n = 3 and degree 3 + 2 = 5; 1 + 3 + 3 + 3 = 5 + 5 is a primitive partition identity with largest part n = 5 and degree 4 + 2 = 6. IA: toric ideal assoc. to A ∈ Zd×n Gr(A): Graver basis of A; Gr(A) = IA Observation Let A be the 1 × n integer matrix A = (1 2 . . . n). Then, the binomial xa1xa2 · · · xak − xb1xb2 · · · xbl is a primitive element of IA iff a1 + a2 + . . . + ak = b1 + b2 + . . . + bl is a ppi, i.e. there is a correspondence between ppi’s and elements of Gr(A).

MB for LR October 4, 2015 4 / 9

Partition Identities CHPPIs

Color-Homogeneous PPIs

Definition A colored partition identity in the colors 1, . . . , c is an identity of the form: a1,1 + . . . + a1,k1 + a21 + . . . + a2,k2 + ac,1 + . . . + ac,kc = b1,1 + . . . + b1,l1 + b21 + . . . + b2,l2 + bc,1 + . . . + bc,lc where 1 ≤ ap,j, bp,j ≤ np are positive integers for all j, 1 ≤ p ≤ c and some positive integers n1, . . . , nc. If kj = lj for all 1 ≤ j ≤ c, then it is called color-homogeneous. A chpi is primitive if there is no proper color-homogeneous subidentity a−,i1 + . . . + a−,ir = b−,j1 + . . . + b−,js with 1 ≤ r + s ≤ k1 + . . . + kc + l1 + . . . + lc . The degree is the number

• f summands k1 + . . . + kc + l1 + . . . + lc.

MB for LR October 4, 2015 5 / 9

Partition Identities CHPPIs

Color-Homogeneous PPIs

Example Let c = 2 and let n1 = n2 = 3. Then, the following are examples of color-homogeneous partition identities in c = 2 colors with n1 = n2 = 3: 1 + 3 = 2 + 2 1 + 3 = 3 + 1 2 + 2 + 3 = 3 + 3 + 1 Theorem (Petrovi´ c) Let nP = max{ni : 1 ≤ i ≤ c} and let nQ = max{nj : 1 ≤ j ≤ c}, j = P}. Then, any primitive color-homogeneous partition identity satisfies k1 + . . . + kc + l1 + . . . + lc ≤ nP + nQ − 2. Remark Again, there is a correspondence between primitive color-homogeneous partition identitites and Gr(A) for some integer matrix A.

MB for LR October 4, 2015 6 / 9

Partition Identities CHPPIs

Moves of the Poisson model

l (j, k) 1 2 3 (0, 0) 2 (0, 1) −3 1 (1, 0) −2 (1, 1) 2 We record an entry in the (l, (j, k)) cell as ljk, putting positive entries on the left and negative entries on the right, which gives: 200 + 200 + 301 + 111 + 111 = 101 + 101 + 101 + 310 + 310 Color according to j: 200 + 200 + 301 + 111 + 111 = 101 + 101 + 101 + 310 + 310 Color according to k: 200 + 200 + 301 + 111 + 111 = 101 + 101 + 101 + 310 + 310

MB for LR October 4, 2015 7 / 9

Logistic regression 2 × 2 × 2 × L tables

Logistic regression

log

• p1jkl

1 − p1jkl

• = α + βl + γdj

1 + δdk 1

j = 0, 1; k = 0, 1 l = 1, . . . , L dj

1 =

• 1,

j = 1 0,

• therwise ,

dk

1 =

• 1,

k = 1 0,

• therwise

Configuration matrix: Λ(B) =

• B

In×n In×n

• ,

where B is the Poisson configuration (this is called the Lawrence lifting

• f B).

Example In the fishing example from earlier, we now think of modeling success/failure outcomes instead of counts (i.e. did you meet your quota for the day, or not?) where the variables have the same interpretation as before (# of fisherman, weather, weekend/weekday).

MB for LR October 4, 2015 8 / 9

Logistic regression 2 × 2 × 2 × L tables

Markov bases for Logistic Regression

The idea is to exploit the connection to generalized CHPIs to prove bounds on the elements of the Gr(A), since this corresponds to a Markov basis for IΛ(A). Conjecture (Hara, W. 2015) There exists a Markov basis for the logistic regression model for 2 × 2 × 2 × L tables with maximum degree d ≤

• 6

if L = 2 6L − 8 if L ≥ 3 . Remark This conjecture has been verified computationally for L ≤ 6.

MB for LR October 4, 2015 9 / 9