The cavity method for matchings Marc Lelarge INRIA & ENS - - PowerPoint PPT Presentation

The cavity method for matchings

Marc Lelarge

INRIA & ENS

Cargese 2014

1

MAXIMUM MATCHINGS

Let G = (V, E) be a simple finite graph. A matching of G is a subset of edges with no common end-vertex. Notation: B = (Be)e∈E ∈ {0, 1}E, with

e∈∂v Be ≤ 1 for all v ∈ V .

The size of the matching is

e∈E Be.

A maximum matching is a matching with maximum size denoted ν(G) = maxB

• e Be.

MOTIVATION

• Optimization is easy: flow problem
• Counting is hard: #P- complete
• called monomer-dimer model in stat. phys. with a correspondance with Ising model

Heilmann, Lieb 72

• link with XORSAT

XORSAT

Does there exist a solution to the linear system Hx = b, i.e. does b belongs to the image

• f H?

CONNECTION WITH MATCHINGS

For any k × k submatrices A of H, we have in GF(2),

det(A) =

• σ∈Sk

k

• i=1

Ai,σ(i)

Lemma 1. Let G be the bipartite graph associated to H, then we have rk2(H) ≤ ν(G). For a given H ∈ {0, 1}n×m and a random b ∈ {0, 1}m, we then have

P ( XORSAT(H, b) has a solution) = 2rk 2(H)−m ≤ 2ν(G)−m

GIBBS MEASURE FOR MATCHINGS

Introduce the Gibbs measure on matchings:

µz

G(B) = z

• e Be

PG(z)

where PG(z) is the partition function:

PG(z) =

• B

z

• e Be

v∈V

1 I(

• e∈∂v

Be ≤ 1) =

ν(G)

• k=0

mk(G)zk,

where mk(G) is the number of matchings of size k.

CAVITY METHOD ON TREES

If G is a tree then

µz

G(Be = 1) =

Y−

→ e (z)Y−− → e (z)

z + Y−

→ e (z)Y−− → e (z) = xe(z) ∈ (0, 1)

where

Yu→v(z) = z 1 +

w∈∂u\v Yw→u(z)

Notation:

Y(z) = zRG(Y(z))

Zdeborov´ a, M´ ezard 06

BELIEF PROPAGATION ON GENERAL GRAPHS

The map (x1, . . . , xk) →

z 1+k

i=1 xi is non-increasing, hence iterating the map

zRG(.), we build a sequence Yt(z) such that 0 ≤ Y2t(z) ≤ Y−(z) ≤ Y(z) ≤ Y+(z) ≤ Y2t+1(z) ≤ z,

and

Y−(z) = zRG(Y+(z)) Y+(z) = zRG(Y−(z)).

In particular, we have

Y −

− → e R−− → e (Y−) = Y + −− → e R− → e (Y+).

Y+(z) = Y−(z): PROOF

Recall:

Y −

− → e R−− → e (Y−) = Y + −− → e R− → e (Y+).

We define

Dv(Y) =

• −

→ e ∈∂v

Y−

→ e R−− → e (Y)

1 + Y−

→ e R−− → e (Y)

=

• −

→ e ∈∂v Y− → e

1 +

− → e ∈∂v Y− → e

,

which is an increasing function of Y. Also Dv(Y+(z)) = Dv(Y−(z)), when summing over all vertices, we have

• v∈V

Dv(Y+(z)) =

• v∈V

Dv(Y−(z)).

LIMIT OF BELIEF PROPAGATION

Bethe internal energy

U B(x) = −

• e∈E

xe

Bethe entropy

SB(x) = 1 2

• v∈V
• e∈∂v

−xe ln xe + (1 − xe) ln(1 − xe) −2

• 1 −
• e∈∂v

xe

• ln
• 1 −
• e∈∂v

xe

• The Bethe entropy is concave on FM(G) = {x ∈ RE, xe ≥ 0,

e∈∂v xe ≤ 1}.

Vontobel 13

LIMIT OF BELIEF PROPAGATION

Belief Propagation maximizes the Bethe free entropy

ΦB(x, z) = −U B(x) ln z + SB(x)

From the messages Y(z), compute

xe(z) = Y−

→ e (z)Y−− → e (z)

z + Y−

→ e (z)Y−− → e (z) ∈ (0, 1)

Then, we have

ΦB(x(z), z) = max

x∈FM(G) ΦB(x, z).

LIMIT z → ∞

Since SB(x) ≤ |E|, for z sufficiently large, x(z) is on the optimal face of FM(G), so that

lim

z→∞

• e

xe(z) = ν∗(G)

where ν∗(G) is the fractional matching number. Chertkov 08 Relation between LP relaxation and BP: Bayati, Shah, Sharma 08; Sanghavi, Shah, Willsky 09; Sanghavi, Malioutov, Willsky 11; Bayati, Borgs, Chayes, Zecchina 11

SIMPLIFICATION OF BP WHEN z → ∞

The only relevant quantity is IY

− → e = 1

I(limz→∞ Y−

→ e (z) = ∞).

For {0, 1}-valued messages, define J = PG(I) by

Ju→v = 1 I  

w∈∂u\v

Iw→u = 0  

Then we have IY = PG ◦ PG(IY) and 2ν∗(G) =

v Fv(IY), with

Fv(I) = 1 ∧

u∈∂v

Iu→v

• +
• 1 −
• u∈∂v

Iv→u +

Indeed, IY minimizes

v Fv(I) under the constraint I = PG ◦ PG(I).

IMPLICATION FOR VERTEX COVER

The vertex cover number is the solution of the following ILP:

τ(G) = min

• v∈V

yv

s.t.

yu + yv ≥ 1, ∀(uv) ∈ E; yv ∈ {0, 1},

Then (Fv(IY )/2, v ∈ V ) is a minimum half-integral vertex cover. In particular,

(Fv(IY ), v ∈ V ) is a 2-approximate solution to vertex cover on G.

For bipartite graphs, a slight extension of these results gives a minimum vertex cover. Lelarge 14

RANDOM GRAPHS

We are interested in a sequence Gn of random diluted graphs : deg(v; Gn) = O(1) as the number of vertices n tends to infinity. Important examples of random graphs on {1, · · · , n},

• Erd˝
• s-R´

enyi graphs with parameter p = λ/n.

• Uniform measure on k-regular graphs.
• Graphs with prescribed degree distribution F∗ : independently for each vertex, we

draw a random number of half-edges with distribution F∗. If the total number of half-edge is even, we match them uniformly.

CAVITY METHOD ON LARGE GRAPHS

• Introduce the Gibbs measure on matchings:

µz

G(B) = z

• e Be

PG(z)

so that the size of a maximum matching of the graph G = (V, E) is given by

1 2 lim

z→∞

• v∈V
• e∈∂v

µz

G(Be = 1).

• Show that on trees, the marginal µz

G(Be = 1) can be computed by a message

passing algorithm with a unique fixed point.

• Show that on trees, when z → ∞, this message passing algorithm reduces to the

previously described 0 − 1 valued message passing algorithm and that the limit of

µz

G(Be = 1) can be computed from the minimal fixed point solution.

• Using a convexity argument, invert the limits in n and z.

? CONVERGENCES ?

Do we have convergence of the (normalized) matching number ?

lim

n→∞

1 2n

• v∈Vn

lim

z→∞

• e∈∂v

µz

Gn(Be = 1)

YES in the following cases:

• Karp & Sipser 81 for Erd˝
• s-R´

enyi graph.

• Bohman & Frieze 09 with a ’log-concave’ condition on the degree distribution.

For the random assignment problem: it converges at zero temperature to ζ(2), Aldous 01 and at very high temperature, Talagrand 03.

CAVITY METHOD ON LARGE GRAPHS

• Introduce the Gibbs measure on matchings:

µz

G(B) = z

• e Be

PG(z)

so that the size of a maximum matching of the graph G = (V, E) is given by

1 2 lim

z→∞

• v∈V
• e∈∂v

µz

G(Be = 1).

• Show that on trees, the marginal µz

G(Be = 1) can be computed by a message

passing algorithm with a unique fixed point. Unimodularity.

• Show that on trees, when z → ∞, this message passing algorithm reduces to the

previously described 0 − 1 valued message passing algorithm and that the limit of

µz

G(Be = 1) can be computed from the minimal fixed point solution.

• Using a convexity argument, invert the limits in n and z.

CAVITY METHOD ON LARGE GRAPHS

• Introduce the Gibbs measure on matchings:

µz

G(B) = z

• e Be

PG(z)

so that the size of a maximum matching of the graph G = (V, E) is given by

1 2 lim

z→∞

• v∈V
• e∈∂v

µz

G(Be = 1).

• Show that on trees, the marginal µz

G(Be = 1) can be computed by a message

passing algorithm with a unique fixed point. Unimodularity.

• Show that on trees, when z → ∞, this message passing algorithm reduces to the

previously described 0 − 1 valued message passing algorithm and that the limit of

µz

G(Be = 1) can be computed from the minimal fixed point solution.

• Using a convexity argument, invert the limits in n and z.

CAVITY METHOD ON LARGE GRAPHS

• Introduce the Gibbs measure on matchings:

µz

G(B) = z

• e Be

PG(z)

so that the size of a maximum matching of the graph G = (V, E) is given by

1 2 lim

z→∞

• v∈V
• e∈∂v

µz

G(Be = 1).

• Show that on trees, the marginal µz

G(Be = 1) can be computed by a message

passing algorithm with a unique fixed point. Unimodularity.

• Show that on trees, when z → ∞, this message passing algorithm reduces to the

previously described 0 − 1 valued message passing algorithm and that the limit of

µz

G(Be = 1) can be computed from the minimal fixed point solution.

• Using a convexity argument, invert the limits in n and z.

RESULT FOR RANDOM BIPARTITE GRAPHS

We consider standard bipartite graphs between variable and function nodes

GN = G(N, Λ, Γ), where N is the number of variable nodes, Λ(x) =

d≥0 Λdxd is

the variable-node degree distribution and Γ(x) =

d≥0 Γdxd is the function-node

degree distribution. Proposition 1. Bordenave, Lelarge, Salez 12; Salez 12; Lelarge 12; Leconte, Lelarge, Massouli´ e 13. For a sequence of graphs GN = G(N, Λ, Γ), where Λ and Γ are fixed, we have

1 M ν (GN) → 1 − max

x∈[0,1] H(x),

where

H(x) = Γ

• 1 − Λ′(1 − x)

Λ′(1)

• −Γ′(1)

Λ′(1)

• 1 − Λ(1 − x) − xΛ′(1 − x)
• .

SOME CONSEQUENCES

Erd˝

• s-R´

enyi graph: function H for λ = 2, λ = e and λ = 3.

0.7 1.0 0.9 0.8 0.6 0.5 0.4 0.13 0.14 0.15 0.16 0.17 0.18 0.19 0.20 0.21 0.22 0.0 0.1 0.2 0.3 0.6 0.4 0.5 1.0 0.3 0.065 0.070 0.075 0.080 0.085 0.090 0.095 0.0 0.100 0.1 0.105 0.2 0.9 0.8 0.7 0.8 0.7 0.050 0.055 0.060 0.065 0.070 0.075 0.6 0.0 0.5 0.1 0.4 1.0 0.3 0.2 0.9

This result disproves a conjecture of Wormald 99: there are graphs with minimal degree ≥ 3 and with no perfect matching.

IMPLICATIONS FOR XORSAT

Recall:

P ( XORSAT(H, b) has a solution) = 2rk 2(H)−m ≤ 2ν(G)−m

Corollary 1. Consider a random XORSAT instance (H(N, Λ, Γ), b) then the probability for this instance to be satisfiable goes to zero as N tends to infinity as soon as

maxx∈[0,1] H(x) > 0, where H(x) = Γ

• 1 − Λ′(1 − x)

Λ′(1)

• −Γ′(1)

Λ′(1)

• 1 − Λ(1 − x) − xΛ′(1 − x)
• Conjecture 1. We have as N tends to infinity

rk2(H(N, Λ, Γ)) − ν(G(N, Λ, Γ)) = o(N).

IRREGULAR XORSAT

Example: variable-node degree distribution is Λ(x) = 4

5x3 + 1 5x15 and the

function-node degree distribution is Γ(x) = b(α)x3 + (1 − b(α))x15 where

α = M/N is the ratio of the number of clauses to the number of variables.

Naive claim: αS = 1

IRREGULAR XORSAT

Example: variable-node degree distribution is Λ(x) = 4

5x3 + 1 5x15 and the

function-node degree distribution is Γ(x) = b(α)x3 + (1 − b(α))x15 where

α = M/N is the ratio of the number of clauses to the number of variables.

Naive claim cannot hold. Applying Corollary 1, we find that αS ≤ α∗, with

0.963025298 < α∗ < 0.963025299.

CONCLUSION

• Loopy BP algorithm for vertex cover and matching problems.
• General method to compute law of large numbers for combinatorial structures on

sparse (random) graphs. (a) to compute the ground state, add a (small) noise parameter. (b) crucially use monotonicity of the recursions

• Our method works for matchings, spanning subgraphs with degree constraints and

b-matchings but not (directly) for XORSAT.

• In the case of matchings, the cavity method works on general infinite graphs by

looking at the tree of self-avoiding walks Godsil 81.

THANK YOU!

more details at:

http://www.di.ens.fr/˜lelarge/