SauerShelahPerles Lemma for Lattices Joint work with Stijn C a - PowerPoint PPT Presentation

Sauer–Shelah–Perles Lemma for Lattices Joint work with Stijn C a mbie, Bogd a n Chornom a z, Zeev Dvir a nd Sh a y Mor a n Yuv a l Filmus, 24 November 2020

VC dimension ℱ ⊆ {0,1} X The VC dimension of a family is the maximal size of a shattered set. X { { 1 1 0 0 0 0 1 1 0 0 ℱ 0 0 1 1 0 0 0 0 1 1 Shattered VC dimension = 2

VC dimension Relation to learning: Hypothesis class is PAC-learnable i ff it has fi nite VC dimension. Sauer–Shelah–Perles lemma: | ℱ | ≤ ( ≤ d ) | X | ℱ ⊆ {0,1} X d If has VC dimension then . Dichotomy theorem: ℱ ⊆ {0,1} X Let , where is in fi nite. X If VC( ℱ ) < ∞ then | proj( ℱ , S ) | ≤ poly ( | S | ) for all S ⊆ X . | proj( ℱ , S ) | = 2 | S | If VC( ℱ ) = ∞ then for in fi nitely many . S

q -analog of VC dimension 𝔾 Can we de fi ne VC dimension for families of subspaces over some fi nite fi eld ? Alternative de fi nition of VC dimension for sets: ℱ ⊆ 2 X The VC dimension of family is the maximum size of a shattered set. ℱ ⊆ 2 X A family shatters a set S ⊆ X if S ∩ ℱ consists of all subsets of . S 1 2 3 4 5 Intersection with {2,3} {1,2} 1 {2} 1 0 0 0 {2,3} 0 1 1 0 0 {2,3} {3,4} 0 0 1 1 0 {3} {4,5} 0 0 0 1 1 ∅

q -analog of VC dimension Alternative de fi nition of VC dimension for sets: ℱ ⊆ 2 X The VC dimension of family is the maximum size of a shattered set. ℱ ⊆ 2 X A family shatters a set S ⊆ X if for S ∩ ℱ consists of all subsets of . S De fi nition of VC dimension for vector spaces 𝔾 n The VC dimension of family ℱ of subspaces of is the maximum dimension of a shattered subspace. 𝔾 n A family ℱ shatters a subspace of S if S ∩ ℱ consists of all subspaces of . S Sauer–Shelah–Perles lemma [Babai–Frankl]: | ℱ | ≤ [ ≤ d ] | 𝔾 | n 𝔾 n If ℱ is a family of subspaces of that has VC dimension then d .

Proving the Sauer–Shelah–Perles lemma Sauer–Shelah–Perles lemma: | ℱ | ≤ ( ≤ d ) | X | ℱ ⊆ {0,1} X If has VC dimension then d . Pajor’s strengthening: ℱ ⊆ {0,1} X | ℱ | If then ℱ shatters at least many sets. | X | Method 1: Induction on . ℱ = { S ∈ ℱ : x ∈ S } ∪ { S ∈ ℱ : x ∉ S } x ∈ X Decompose for an arbitrary . Method 2: Monotonization. Lemma trivial for downward-closed families. Monotonization increases number of shattered sets. Method 3: Polynomial / linear algebra method.

Linear algebra proof Pajor’s strengthening: ℱ ⊆ {0,1} X | ℱ | If then ℱ shatters at least many sets. Proof idea: Every function ℱ → ℝ can be expressed as linear combination of monomials corresponding to shattered sets. Key observation: If ℱ does not shatter then S x S is expressible as linear combination of smaller monomials for inputs in ℱ . Proof by example: • If {1,2} ∉ ℱ ∩ {1,2} then x 1 x 2 = 0 . Extends to vector spaces! • If {1} ∉ ℱ ∩ {1,2} then x 1 x 2 = x 1 . • If ∅ ∉ ℱ ∩ {1,2} then x 1 x 2 = x 1 + x 2 − 1 .

Sauer–Shelah–Perles lemma for lattices Proof works for any lattice of fl ats in a matroid ( geometric lattice ). • Complete uniform matroid: usual SSP lemma. • Complete linear matroid: SSP lemma for vector spaces. • Complete graphical matroid: SSP lemma for partitions. More generally, proof holds whenever the Möbius function doesn’t vanish. • If {1,2} ∉ ℱ ∩ {1,2} then x 1 x 2 = 0 . • If {1} ∉ ℱ ∩ {1,2} then x 1 x 2 = 1 ⋅ x 1 . • If ∅ ∉ ℱ ∩ {1,2} then x 1 x 2 = 1 ⋅ x 1 + 1 ⋅ x 2 − 1 . Negated Möbius function

When does Sauer–Shelah–Perles lemma hold? ℒ Sauer–Shelah–Perles lemma for lattice : If ℱ ⊆ ℒ then ℱ shatters at least | ℱ | many elements of ℒ . Babai–Frankl: SSP holds for ℒ if μ ( x , y ) ≠ 0 for all x ≤ y . SSP doesn’t hold: {1,2} 0 only shatters 2 2 ∧ {1,2} = {1,2} μ (1,2) = − 1 1 1 ∧ {1,2} = {1} μ (0,2) = 0 μ (0,1) = − 1 0 0 ∧ {1,2} = {0}

When does Sauer–Shelah–Perles lemma hold? Babai–Frankl: SSP holds for ℒ if μ ( x , y ) ≠ 0 for all x ≤ y . SSP holds for some lattices with vanishing Möbius function: 2 μ (1,2) = − 1 1 Doesn’t hold for 3-element interval: μ (0,2) = 0 μ (0,1) = − 1 0 Doesn’t hold if lattice contains 3-element interval, i.e., points x < z with exactly one solution to x < y < z . Conjecture: SSP holds i ff lattice contains no 3-element interval (lattice is relatively complemented ).

Relative complementation Lattice is relatively complemented if for every x < y < z there exists such that y ′ y ∧ y ′ = x and y ∨ y ′ = z . 2 μ (1,2) = − 1 1 Doesn’t hold for 3-element interval: y ′ 1 ∧ y ′ = 0 1 ∨ y ′ = 2 No satis fi es and . μ (0,2) = 0 μ (0,1) = − 1 0 Björner: A lattice is relatively complemented i ff it doesn’t contain a 3-element interval.

Partial results Conjecture: SSP holds i ff lattice is relatively complemented (RC). Babai and Frankl: If Möbius function never vanishes, lattice is SSP. Theorem 1: If lattice is RC and μ ( x , y ) = 0 only if x , y are minimal and maximal elements, then lattice is SSP. Theorem 2: Product of SSP lattices is SSP. Theorem 3: If lattice is RC then SSP holds for all families whose set of non-shattered elems contains a minimum. Thanks!