High-dimensional Ising model and Monte Carlo methods Wojciech - - PowerPoint PPT Presentation

▶
high dimensional ising model and monte carlo methods
SMART_READER_LITE
LIVE PREVIEW

High-dimensional Ising model and Monte Carlo methods Wojciech - - PowerPoint PPT Presentation

Mar 12, 2023 162 likes •1.01k views

Introduction Model selection consistency Experiments References High-dimensional Ising model and Monte Carlo methods Wojciech Rejchel Nicolaus Copernicus University in Toru Joint work with Baej Miasojedow Wojciech Rejchel

slide-1
SLIDE 1

Introduction Model selection consistency Experiments References

High-dimensional Ising model and Monte Carlo methods

Wojciech Rejchel Nicolaus Copernicus University in Toruń Joint work with Błażej Miasojedow

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-2
SLIDE 2

Introduction Model selection consistency Experiments References

Markov random field

Undirected graph (V , E)

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-3
SLIDE 3

Introduction Model selection consistency Experiments References

Markov random field

Undirected graph (V , E) V = {1, . . . , d} - set of vertices

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-4
SLIDE 4

Introduction Model selection consistency Experiments References

Markov random field

Undirected graph (V , E) V = {1, . . . , d} - set of vertices E ⊂ V × V - set of edges

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-5
SLIDE 5

Introduction Model selection consistency Experiments References

Markov random field

Undirected graph (V , E) V = {1, . . . , d} - set of vertices E ⊂ V × V - set of edges Y = (Y (1), . . . , Y (d)) - random vector

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-6
SLIDE 6

Introduction Model selection consistency Experiments References

Markov random field

Undirected graph (V , E) V = {1, . . . , d} - set of vertices E ⊂ V × V - set of edges Y = (Y (1), . . . , Y (d)) - random vector Y (s) is associated with vertex s ∈ V

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-7
SLIDE 7

Introduction Model selection consistency Experiments References

Ising model

Y (s) ∈ {−1, 1}

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-8
SLIDE 8

Introduction Model selection consistency Experiments References

Ising model

Y (s) ∈ {−1, 1} Joint distribution of Y is given by p(y|θ⋆) = 1 C(θ⋆) exp

  • r<s

θ⋆

rsy(r)y(s)

  • Wojciech Rejchel

High-dimensional Ising model and Monte Carlo methods

slide-9
SLIDE 9

Introduction Model selection consistency Experiments References

Ising model

Y (s) ∈ {−1, 1} Joint distribution of Y is given by p(y|θ⋆) = 1 C(θ⋆) exp

  • r<s

θ⋆

rsy(r)y(s)

  • θ⋆ ∈ R

d(d−1) 2

  • true parameter

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-10
SLIDE 10

Introduction Model selection consistency Experiments References

Ising model

Y (s) ∈ {−1, 1} Joint distribution of Y is given by p(y|θ⋆) = 1 C(θ⋆) exp

  • r<s

θ⋆

rsy(r)y(s)

  • θ⋆ ∈ R

d(d−1) 2

  • true parameter

Intractable norming constant C(θ⋆) =

  • y∈{0,1}d

exp

  • r<s

θ⋆

rsy(r)y(s)

  • Wojciech Rejchel

High-dimensional Ising model and Monte Carlo methods

slide-11
SLIDE 11

Introduction Model selection consistency Experiments References

Ising model

Y (s) ∈ {−1, 1} Joint distribution of Y is given by p(y|θ⋆) = 1 C(θ⋆) exp

  • r<s

θ⋆

rsy(r)y(s)

  • θ⋆ ∈ R

d(d−1) 2

  • true parameter

Intractable norming constant C(θ⋆) =

  • y∈{0,1}d

exp

  • r<s

θ⋆

rsy(r)y(s)

  • J(y) = (y(r)y(s))r<s

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-12
SLIDE 12

Introduction Model selection consistency Experiments References

Ising model

Y (s) ∈ {−1, 1} Joint distribution of Y is given by p(y|θ⋆) = 1 C(θ⋆) exp

  • r<s

θ⋆

rsy(r)y(s)

  • θ⋆ ∈ R

d(d−1) 2

  • true parameter

Intractable norming constant C(θ⋆) =

  • y∈{0,1}d

exp

  • r<s

θ⋆

rsy(r)y(s)

  • J(y) = (y(r)y(s))r<s

p(y|θ⋆) = 1 C(θ⋆) exp

(θ⋆)′J(y)

  • Wojciech Rejchel

High-dimensional Ising model and Monte Carlo methods

slide-13
SLIDE 13

Introduction Model selection consistency Experiments References

Ising model

θ⋆

rs = 0

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-14
SLIDE 14

Introduction Model selection consistency Experiments References

Ising model

θ⋆

rs = 0 means that Y (r) and Y (s) are conditionally

independent

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-15
SLIDE 15

Introduction Model selection consistency Experiments References

Ising model

θ⋆

rs = 0 means that Y (r) and Y (s) are conditionally

independent Finding conditional independence

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-16
SLIDE 16

Introduction Model selection consistency Experiments References

Ising model

θ⋆

rs = 0 means that Y (r) and Y (s) are conditionally

independent Finding conditional independence ⇔ recognizing structure of graph

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-17
SLIDE 17

Introduction Model selection consistency Experiments References

Ising model

θ⋆

rs = 0 means that Y (r) and Y (s) are conditionally

independent Finding conditional independence ⇔ recognizing structure of graph ⇔ estimation of θ⋆

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-18
SLIDE 18

Introduction Model selection consistency Experiments References

Likelihood estimation

Y1, . . . , Yn - independent random vectors from p(·|θ⋆)

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-19
SLIDE 19

Introduction Model selection consistency Experiments References

Likelihood estimation

Y1, . . . , Yn - independent random vectors from p(·|θ⋆) Negative log-likelihood ℓn(θ) = −1 n

n

  • i=1

θ′J(Yi) + log C(θ)

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-20
SLIDE 20

Introduction Model selection consistency Experiments References

Likelihood estimation

Y1, . . . , Yn - independent random vectors from p(·|θ⋆) Negative log-likelihood ℓn(θ) = −1 n

n

  • i=1

θ′J(Yi) + log C(θ) Pseudolikelihood approximation

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-21
SLIDE 21

Introduction Model selection consistency Experiments References

Likelihood estimation

Y1, . . . , Yn - independent random vectors from p(·|θ⋆) Negative log-likelihood ℓn(θ) = −1 n

n

  • i=1

θ′J(Yi) + log C(θ) Pseudolikelihood approximation Monte Carlo (MC) approximation

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-22
SLIDE 22

Introduction Model selection consistency Experiments References

Pseudolikelihood approximation

p(y|θ) =

d

  • s=1

p(y(s)|y(s − 1), . . . , y(1), θ)

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-23
SLIDE 23

Introduction Model selection consistency Experiments References

Pseudolikelihood approximation

p(y|θ) =

d

  • s=1

p(y(s)|y(s − 1), . . . , y(1), θ) ≈

d

  • s=1

p(y(s)|y(−s), θ)

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-24
SLIDE 24

Introduction Model selection consistency Experiments References

Pseudolikelihood approximation

p(y|θ) =

d

  • s=1

p(y(s)|y(s − 1), . . . , y(1), θ) ≈

d

  • s=1

p(y(s)|y(−s), θ) y(−s) = (y(1), . . . , y(s − 1), y(s + 1), . . . , y(d))

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-25
SLIDE 25

Introduction Model selection consistency Experiments References

MC approximation

h(y) - importance sampling distribution

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-26
SLIDE 26

Introduction Model selection consistency Experiments References

MC approximation

h(y) - importance sampling distribution Norming constant C(θ) =

  • y∈{0,1}d

exp

θ′J(y)

  • Wojciech Rejchel

High-dimensional Ising model and Monte Carlo methods

slide-27
SLIDE 27

Introduction Model selection consistency Experiments References

MC approximation

h(y) - importance sampling distribution Norming constant C(θ) =

  • y∈{0,1}d

exp

θ′J(y) =

  • y∈{0,1}d

exp [θ′J(y)] h(y) h(y)

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-28
SLIDE 28

Introduction Model selection consistency Experiments References

MC approximation

h(y) - importance sampling distribution Norming constant C(θ) =

  • y∈{0,1}d

exp

θ′J(y) =

  • y∈{0,1}d

exp [θ′J(y)] h(y) h(y) = EY ∼h exp [θ′J(Y )] h(Y )

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-29
SLIDE 29

Introduction Model selection consistency Experiments References

MC approximation

h(y) - importance sampling distribution Norming constant C(θ) =

  • y∈{0,1}d

exp

θ′J(y) =

  • y∈{0,1}d

exp [θ′J(y)] h(y) h(y) = EY ∼h exp [θ′J(Y )] h(Y ) Norming constant approximation 1 m

m

  • k=1

exp

  • θ′J(Y k)
  • h(Y k)

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-30
SLIDE 30

Introduction Model selection consistency Experiments References

MC approximation

h(y) - importance sampling distribution Norming constant C(θ) =

  • y∈{0,1}d

exp

θ′J(y) =

  • y∈{0,1}d

exp [θ′J(y)] h(y) h(y) = EY ∼h exp [θ′J(Y )] h(Y ) Norming constant approximation 1 m

m

  • k=1

exp

  • θ′J(Y k)
  • h(Y k)

Y 1, . . . , Y m - Markov chain with stationary distribution h

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-31
SLIDE 31

Introduction Model selection consistency Experiments References

MCMC approximation

ℓm

n (θ) = −1

n

n

  • i=1

θ′J(Yi) + log

  1

m

m

  • k=1

exp

  • θ′J(Y k)
  • h(Y k)

 

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-32
SLIDE 32

Introduction Model selection consistency Experiments References

MCMC approximation

ℓm

n (θ) = −1

n

n

  • i=1

θ′J(Yi) + log

  1

m

m

  • k=1

exp

  • θ′J(Y k)
  • h(Y k)

 

Importance sampling distribution h(y) = p(y|ψ)

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-33
SLIDE 33

Introduction Model selection consistency Experiments References

MCMC approximation

ℓm

n (θ) = −1

n

n

  • i=1

θ′J(Yi) + log

  1

m

m

  • k=1

exp

  • θ′J(Y k)
  • h(Y k)

 

Importance sampling distribution h(y) = p(y|ψ) −1 n

n

  • i=1

θ′J(Yi)+log

  • 1

m

m

  • k=1

exp

  • (θ − ψ)′J(Y k)
  • −log(C(ψ))

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-34
SLIDE 34

Introduction Model selection consistency Experiments References

High-dimensional setting

d = dn >> n

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-35
SLIDE 35

Introduction Model selection consistency Experiments References

High-dimensional setting

d = dn >> n Number of parameters = d(d−1)

2

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-36
SLIDE 36

Introduction Model selection consistency Experiments References

High-dimensional setting

d = dn >> n Number of parameters = d(d−1)

2

Penalized empirical risk minimization ℓm

n (θ) + λm n |θ|1

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-37
SLIDE 37

Introduction Model selection consistency Experiments References

High-dimensional setting

d = dn >> n Number of parameters = d(d−1)

2

Penalized empirical risk minimization ℓm

n (θ) + λm n |θ|1

|θ|1 =

r<s |θrs|

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-38
SLIDE 38

Introduction Model selection consistency Experiments References

High-dimensional setting

d = dn >> n Number of parameters = d(d−1)

2

Penalized empirical risk minimization ℓm

n (θ) + λm n |θ|1

|θ|1 =

r<s |θrs|

ˆ θm

n = arg minθ ℓm n (θ) + λm n |θ|1

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-39
SLIDE 39

Introduction Model selection consistency Experiments References

Related papers

Ravikumar, P., Wainwright, M. J., Lafferty, J. - Ann. Statist. (2010)

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-40
SLIDE 40

Introduction Model selection consistency Experiments References

Related papers

Ravikumar, P., Wainwright, M. J., Lafferty, J. - Ann. Statist. (2010) H¨

  • fling, H., Tibshirani, R. - JMLR (2009)

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-41
SLIDE 41

Introduction Model selection consistency Experiments References

Related papers

Ravikumar, P., Wainwright, M. J., Lafferty, J. - Ann. Statist. (2010) H¨

  • fling, H., Tibshirani, R. - JMLR (2009)

Guo J., Levina E., Michailidis G., Zhu J. (2010)

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-42
SLIDE 42

Introduction Model selection consistency Experiments References

Related papers

Ravikumar, P., Wainwright, M. J., Lafferty, J. - Ann. Statist. (2010) H¨

  • fling, H., Tibshirani, R. - JMLR (2009)

Guo J., Levina E., Michailidis G., Zhu J. (2010) Jalali, A., Johnson, C. C., Ravikumar, P. K. - NIPS (2011)

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-43
SLIDE 43

Introduction Model selection consistency Experiments References

Related papers

Ravikumar, P., Wainwright, M. J., Lafferty, J. - Ann. Statist. (2010) H¨

  • fling, H., Tibshirani, R. - JMLR (2009)

Guo J., Levina E., Michailidis G., Zhu J. (2010) Jalali, A., Johnson, C. C., Ravikumar, P. K. - NIPS (2011) Xue, L., Zou, H., Cai, T. - Ann. Statist. (2012)

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-44
SLIDE 44

Introduction Model selection consistency Experiments References

Notations

ˆ θ = ˆ θm

n

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-45
SLIDE 45

Introduction Model selection consistency Experiments References

Notations

ˆ θ = ˆ θm

n

λ = λm

n

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-46
SLIDE 46

Introduction Model selection consistency Experiments References

Notations

ˆ θ = ˆ θm

n

λ = λm

n

¯ d = d(d − 1)/2

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-47
SLIDE 47

Introduction Model selection consistency Experiments References

Notations

ˆ θ = ˆ θm

n

λ = λm

n

¯ d = d(d − 1)/2 T = {(r, s) : θ⋆

rs = 0}

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-48
SLIDE 48

Introduction Model selection consistency Experiments References

Notations

ˆ θ = ˆ θm

n

λ = λm

n

¯ d = d(d − 1)/2 T = {(r, s) : θ⋆

rs = 0}

¯ d0 = |T|

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-49
SLIDE 49

Introduction Model selection consistency Experiments References

Notations

ˆ θ = ˆ θm

n

λ = λm

n

¯ d = d(d − 1)/2 T = {(r, s) : θ⋆

rs = 0}

¯ d0 = |T| Y 1, . . . , Y m - Gibbs sampler on {−1, 1}d with stationary distribution h

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-50
SLIDE 50

Introduction Model selection consistency Experiments References

Main results

Theorem Let ε > 0. If

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-51
SLIDE 51

Introduction Model selection consistency Experiments References

Main results

Theorem Let ε > 0. If

1 cone invertibility condition is satisfied Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-52
SLIDE 52

Introduction Model selection consistency Experiments References

Main results

Theorem Let ε > 0. If

1 cone invertibility condition is satisfied 2 n C1 ¯

d2

0 log( ¯

d/ε)

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-53
SLIDE 53

Introduction Model selection consistency Experiments References

Main results

Theorem Let ε > 0. If

1 cone invertibility condition is satisfied 2 n C1 ¯

d2

0 log( ¯

d/ε)

3 m C2

¯ d2

0 M2 log(β1 ¯

d/ε) β2

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-54
SLIDE 54

Introduction Model selection consistency Experiments References

Main results

Theorem Let ε > 0. If

1 cone invertibility condition is satisfied 2 n C1 ¯

d2

0 log( ¯

d/ε)

3 m C2

¯ d2

0 M2 log(β1 ¯

d/ε) β2

then with probability at least 1 − 4ε

  • ˆ

θ − θ⋆

  • ∞ C3λ,

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-55
SLIDE 55

Introduction Model selection consistency Experiments References

Main results

Theorem Let ε > 0. If

1 cone invertibility condition is satisfied 2 n C1 ¯

d2

0 log( ¯

d/ε)

3 m C2

¯ d2

0 M2 log(β1 ¯

d/ε) β2

then with probability at least 1 − 4ε

  • ˆ

θ − θ⋆

  • ∞ C3λ,

where λ = max

 

  • log( ¯

d/ε) n , M

  • log(β1 ¯

d/ε) β2m

 

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-56
SLIDE 56

Introduction Model selection consistency Experiments References

Main results

d ∼ O(exp(na)), ¯ d0 ∼ O(nb), if a + 2b < 1

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-57
SLIDE 57

Introduction Model selection consistency Experiments References

Main results

d ∼ O(exp(na)), ¯ d0 ∼ O(nb), if a + 2b < 1 Lasso estimator with threshold δ ˜ θrs =

ˆ

θrs if |ˆ θrs| > δ if |ˆ θrs| δ

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-58
SLIDE 58

Introduction Model selection consistency Experiments References

Main results

d ∼ O(exp(na)), ¯ d0 ∼ O(nb), if a + 2b < 1 Lasso estimator with threshold δ ˜ θrs =

ˆ

θrs if |ˆ θrs| > δ if |ˆ θrs| δ θ⋆

min = min r<s |θ⋆ rs|

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-59
SLIDE 59

Introduction Model selection consistency Experiments References

Main results

Corollary Let ε > 0. If conditions (1)-(3) are satisfied and θ⋆

min/2 δ C3λ,

then P

˜

T = T

  • 1 − 4ε.

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-60
SLIDE 60

Introduction Model selection consistency Experiments References

Simulated data sets

d = 20, 50

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-61
SLIDE 61

Introduction Model selection consistency Experiments References

Simulated data sets

d = 20, 50 n = 50, 100, 200, 500, 1000

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-62
SLIDE 62

Introduction Model selection consistency Experiments References

Simulated data sets

d = 20, 50 n = 50, 100, 200, 500, 1000 m = 105

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-63
SLIDE 63

Introduction Model selection consistency Experiments References

Model 1

first 10 vertices has the “chain structure”: for r 10 θ⋆

r−1,r = ±1

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-64
SLIDE 64

Introduction Model selection consistency Experiments References

Model 1

first 10 vertices has the “chain structure”: for r 10 θ⋆

r−1,r = ±1

  • thers are independent: θ⋆

rs = 0

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-65
SLIDE 65

Introduction Model selection consistency Experiments References

Model 1

first 10 vertices has the “chain structure”: for r 10 θ⋆

r−1,r = ±1

  • thers are independent: θ⋆

rs = 0

signs are chosen randomly

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-66
SLIDE 66

Introduction Model selection consistency Experiments References

Model 1

first 10 vertices has the “chain structure”: for r 10 θ⋆

r−1,r = ±1

  • thers are independent: θ⋆

rs = 0

signs are chosen randomly model dimension= 10

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-67
SLIDE 67

Introduction Model selection consistency Experiments References

Model 2

first 5 vertices are correlated: θ⋆

rs = ±ϑ for r < s and

s = 2, 3, 4, 5

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-68
SLIDE 68

Introduction Model selection consistency Experiments References

Model 2

first 5 vertices are correlated: θ⋆

rs = ±ϑ for r < s and

s = 2, 3, 4, 5 ϑ = 1, 2

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-69
SLIDE 69

Introduction Model selection consistency Experiments References

Model 2

first 5 vertices are correlated: θ⋆

rs = ±ϑ for r < s and

s = 2, 3, 4, 5 ϑ = 1, 2

  • thers are independent: θ⋆

rs = 0

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-70
SLIDE 70

Introduction Model selection consistency Experiments References

Model 2

first 5 vertices are correlated: θ⋆

rs = ±ϑ for r < s and

s = 2, 3, 4, 5 ϑ = 1, 2

  • thers are independent: θ⋆

rs = 0

signs are chosen randomly

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-71
SLIDE 71

Introduction Model selection consistency Experiments References

Model 2

first 5 vertices are correlated: θ⋆

rs = ±ϑ for r < s and

s = 2, 3, 4, 5 ϑ = 1, 2

  • thers are independent: θ⋆

rs = 0

signs are chosen randomly model dimension= 10

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-72
SLIDE 72

Introduction Model selection consistency Experiments References

Model 3

two blocks of correlated vertices: θ⋆

rs = ±2 for r < s,

s = 2, 3, 4 and for 5 r < s, s = 6, 7, 8

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-73
SLIDE 73

Introduction Model selection consistency Experiments References

Model 3

two blocks of correlated vertices: θ⋆

rs = ±2 for r < s,

s = 2, 3, 4 and for 5 r < s, s = 6, 7, 8

  • thers are independent: θ⋆

rs = 0

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-74
SLIDE 74

Introduction Model selection consistency Experiments References

Model 3

two blocks of correlated vertices: θ⋆

rs = ±2 for r < s,

s = 2, 3, 4 and for 5 r < s, s = 6, 7, 8

  • thers are independent: θ⋆

rs = 0

signs are chosen randomly

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-75
SLIDE 75

Introduction Model selection consistency Experiments References

Model 3

two blocks of correlated vertices: θ⋆

rs = ±2 for r < s,

s = 2, 3, 4 and for 5 r < s, s = 6, 7, 8

  • thers are independent: θ⋆

rs = 0

signs are chosen randomly model dimension= 12

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-76
SLIDE 76

Introduction Model selection consistency Experiments References

Simulated data sets

We draw 20 configuration of signs

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-77
SLIDE 77

Introduction Model selection consistency Experiments References

Simulated data sets

We draw 20 configuration of signs We draw 20 replications of data set

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-78
SLIDE 78

Introduction Model selection consistency Experiments References

Simulated data sets

We draw 20 configuration of signs We draw 20 replications of data set λ = c1 ∗

  • log ¯

d/n

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-79
SLIDE 79

Introduction Model selection consistency Experiments References

Simulated data sets

We draw 20 configuration of signs We draw 20 replications of data set λ = c1 ∗

  • log ¯

d/n δ = c2 ∗

  • log ¯

d/n

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-80
SLIDE 80

Introduction Model selection consistency Experiments References

Model 1

Pseudo MCMC d n Lasso TL Lasso TL 20 50 0.23 0.37 0.02 0.18 100 0.74 0.91 0.10 0.73 200 0.78 1.00 0.44 0.97 500 0.97 1.00 0.92 1.00 1000 1.00 1.00 1.00 1.00 50 50 0.20 0.20 0.03 0.12 100 0.70 0.83 0.07 0.61 200 0.88 1.00 0.33 0.93 500 0.99 1.00 0.73 1.00 1000 1.00 1.00 0.97 1.00

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-81
SLIDE 81

Introduction Model selection consistency Experiments References

Model 2, ϑ = 2

Pseudo MCMC d n Lasso TL Lasso TL 20 50 0.51 0.51 0.54 0.54 100 0.50 0.50 0.55 0.55 200 0.46 0.52 0.55 0.55 500 0.52 0.53 0.55 0.58 1000 0.57 0.57 0.58 0.65 50 50 0.51 0.51 0.54 0.54 100 0.48 0.48 0.55 0.55 200 0.46 0.46 0.55 0.55 500 0.51 0.53 0.56 0.56 1000 0.53 0.53 0.55 0.64

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-82
SLIDE 82

Introduction Model selection consistency Experiments References

Model 2, ϑ = 1

Pseudo MCMC d n Lasso TL Lasso TL 20 50 0.15 0.15 0.45 0.45 100 0.14 0.14 0.51 0.51 200 0.14 0.18 0.54 0.54 500 0.19 0.23 0.56 0.56 1000 0.25 0.26 0.55 0.55 50 50 0.15 0.15 0.46 0.46 100 0.14 0.14 0.50 0.50 200 0.15 0.15 0.53 0.53 500 0.16 0.25 0.55 0.55 1000 0.23 0.25 0.54 0.54

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-83
SLIDE 83

Introduction Model selection consistency Experiments References

Model 3

Pseudo MCMC d n Lasso TL Lasso TL 20 50 0.67 0.69 0.75 0.75 100 0.74 0.89 0.75 0.75 200 0.98 1.00 0.88 0.98 500 1.00 1.00 1.00 1.00 1000 1.00 1.00 1.00 1.00 50 50 0.70 0.70 0.74 0.75 100 0.70 0.74 0.75 0.75 200 0.90 1.00 0.75 0.84 500 1.00 1.00 0.97 1.00 1000 1.00 1.00 0.99 1.00

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods

slide-84
SLIDE 84

Introduction Model selection consistency Experiments References

References

Besag J. (1974). Spatial interaction and the statistical analysis of lattice systems. J.

  • R. Statist. Soc. B, 36, 192–236.

Guo J., Levina E., Michailidis G. and Zhu J. (2010). Joint structure estimation for categorical Markov networks, Technical report. H¨

  • fling, H. and Tibshirani, R. (2009). Estimation of sparse binary pairwise Markov

networks using pseudo-likelihoods. J. Mach. Learn. Res., 10, 883–906. Ising, E. (1925). Beitrag zur theorie des ferromagnetismus. Z. Physik, 31, 53–258. Jalali, A., Johnson, C. C. and Ravikumar, P. K. (2011). On learning discrete graphical models using greedy methods, Proceedings of NIPS. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria, 2013. Ravikumar, P., Wainwright, M. J. and Lafferty, J. (2010). High-dimensional Ising model selection using l1-regularized logistic regression. Ann. Statist., 38 1287–1319. Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. J. R. Statist.

  • Soc. B, 58, 267–288.

Xue, L., Zou, H. and Cai, T. (2012). Nonconcave penalized composite conditional likelihood estimation of sparse Ising models. Ann. Stat., 40, 1403–1429.

Wojciech Rejchel High-dimensional Ising model and Monte Carlo methods