Fourier analysis of Boolean functions: Some beautiful examples - - PowerPoint PPT Presentation

Fourier analysis of Boolean functions: Some beautiful examples

Ronald de Wolf Centrum voor Wiskunde en Informatica Amsterdam

Fourier analysis of Boolean functions: Some beautiful examples – p.1/13

Fourier analysis

Fourier analysis of Boolean functions: Some beautiful examples – p.2/13

Fourier analysis

Many applications in math, physics, engineering,

Fourier analysis of Boolean functions: Some beautiful examples – p.2/13

Fourier analysis

Many applications in math, physics, engineering, . . . and in computer science

Fourier analysis of Boolean functions: Some beautiful examples – p.2/13

Fourier analysis

Many applications in math, physics, engineering, . . . and in computer science: Signal processing

Fourier analysis of Boolean functions: Some beautiful examples – p.2/13

Fourier analysis

Many applications in math, physics, engineering, . . . and in computer science: Signal processing Data compression

Fourier analysis of Boolean functions: Some beautiful examples – p.2/13

Fourier analysis

Many applications in math, physics, engineering, . . . and in computer science: Signal processing Data compression Multiplying two polynomials

Fourier analysis of Boolean functions: Some beautiful examples – p.2/13

Fourier analysis

Many applications in math, physics, engineering, . . . and in computer science: Signal processing Data compression Multiplying two polynomials These examples use Fourier analysis over cyclic groups

Fourier analysis of Boolean functions: Some beautiful examples – p.2/13

Fourier analysis

Many applications in math, physics, engineering, . . . and in computer science: Signal processing Data compression Multiplying two polynomials These examples use Fourier analysis over cyclic groups We will focus on Fourier analysis over the Boolean cube

Fourier analysis of Boolean functions: Some beautiful examples – p.2/13

Fourier analysis

Many applications in math, physics, engineering, . . . and in computer science: Signal processing Data compression Multiplying two polynomials These examples use Fourier analysis over cyclic groups We will focus on Fourier analysis over the Boolean cube

= {0, 1}n, set of all n-bit strings

Fourier analysis of Boolean functions: Some beautiful examples – p.2/13

This has been very useful in CS

Fourier analysis of Boolean functions: Some beautiful examples – p.3/13

This has been very useful in CS

Fourier coefficients measure correlations with parities

Fourier analysis of Boolean functions: Some beautiful examples – p.3/13

This has been very useful in CS

Fourier coefficients measure correlations with parities Analysis of error-correcting codes

Fourier analysis of Boolean functions: Some beautiful examples – p.3/13

This has been very useful in CS

Fourier coefficients measure correlations with parities Analysis of error-correcting codes Learning a function from examples

Fourier analysis of Boolean functions: Some beautiful examples – p.3/13

This has been very useful in CS

Fourier coefficients measure correlations with parities Analysis of error-correcting codes Learning a function from examples The influence of variables on a function

Fourier analysis of Boolean functions: Some beautiful examples – p.3/13

This has been very useful in CS

Fourier coefficients measure correlations with parities Analysis of error-correcting codes Learning a function from examples The influence of variables on a function Sensitivity of a function to noise on the inputs

Fourier analysis of Boolean functions: Some beautiful examples – p.3/13

This has been very useful in CS

Fourier coefficients measure correlations with parities Analysis of error-correcting codes Learning a function from examples The influence of variables on a function Sensitivity of a function to noise on the inputs PCPs, NP-hardness of approximation

Fourier analysis of Boolean functions: Some beautiful examples – p.3/13

This has been very useful in CS

Fourier coefficients measure correlations with parities Analysis of error-correcting codes Learning a function from examples The influence of variables on a function Sensitivity of a function to noise on the inputs PCPs, NP-hardness of approximation Cryptography

Fourier analysis of Boolean functions: Some beautiful examples – p.3/13

This has been very useful in CS

Fourier coefficients measure correlations with parities Analysis of error-correcting codes Learning a function from examples The influence of variables on a function Sensitivity of a function to noise on the inputs PCPs, NP-hardness of approximation Cryptography Lower bounds on communication complexity

Fourier analysis of Boolean functions: Some beautiful examples – p.3/13

This has been very useful in CS

Fourier coefficients measure correlations with parities Analysis of error-correcting codes Learning a function from examples The influence of variables on a function Sensitivity of a function to noise on the inputs PCPs, NP-hardness of approximation Cryptography Lower bounds on communication complexity Threshold phenomena in random graphs

Fourier analysis of Boolean functions: Some beautiful examples – p.3/13

This has been very useful in CS

Fourier coefficients measure correlations with parities Analysis of error-correcting codes Learning a function from examples The influence of variables on a function Sensitivity of a function to noise on the inputs PCPs, NP-hardness of approximation Cryptography Lower bounds on communication complexity Threshold phenomena in random graphs Quantum computing

Fourier analysis of Boolean functions: Some beautiful examples – p.3/13

This has been very useful in CS

Fourier coefficients measure correlations with parities Analysis of error-correcting codes Learning a function from examples The influence of variables on a function Sensitivity of a function to noise on the inputs PCPs, NP-hardness of approximation Cryptography Lower bounds on communication complexity Threshold phenomena in random graphs Quantum computing . . .

Fourier analysis of Boolean functions: Some beautiful examples – p.3/13

Fourier analysis over the Boolean cube

Fourier analysis of Boolean functions: Some beautiful examples – p.4/13

Fourier analysis over the Boolean cube

Consider the space of functions f : {0, 1}n → R

Fourier analysis of Boolean functions: Some beautiful examples – p.4/13

Fourier analysis over the Boolean cube

Consider the space of functions f : {0, 1}n → R, with normalized inner product f, g = 1

2n

• x∈{0,1}n

f(x)g(x)

Fourier analysis of Boolean functions: Some beautiful examples – p.4/13

Fourier analysis over the Boolean cube

Consider the space of functions f : {0, 1}n → R, with normalized inner product f, g = 1

2n

• x∈{0,1}n

f(x)g(x)

The parity-functions χs(x) = (−1)x·s =

i:si=1(−1)xi

form an orthonormal basis of this space

Fourier analysis of Boolean functions: Some beautiful examples – p.4/13

Fourier analysis over the Boolean cube

Consider the space of functions f : {0, 1}n → R, with normalized inner product f, g = 1

2n

• x∈{0,1}n

f(x)g(x)

The parity-functions χs(x) = (−1)x·s =

i:si=1(−1)xi

form an orthonormal basis of this space Hence we can write f =

• s∈{0,1}n
• f(s)χs

Fourier analysis of Boolean functions: Some beautiful examples – p.4/13

Fourier analysis over the Boolean cube

Consider the space of functions f : {0, 1}n → R, with normalized inner product f, g = 1

2n

• x∈{0,1}n

f(x)g(x)

The parity-functions χs(x) = (−1)x·s =

i:si=1(−1)xi

form an orthonormal basis of this space Hence we can write f =

• s∈{0,1}n
• f(s)χs

with

f(s) = f, χs = 1 2n

• x∈{0,1}n

f(x)χs(x)

Fourier analysis of Boolean functions: Some beautiful examples – p.4/13

Fourier analysis over the Boolean cube

Consider the space of functions f : {0, 1}n → R, with normalized inner product f, g = 1

2n

• x∈{0,1}n

f(x)g(x)

The parity-functions χs(x) = (−1)x·s =

i:si=1(−1)xi

form an orthonormal basis of this space Hence we can write f =

• s∈{0,1}n
• f(s)χs

with

f(s) = f, χs = 1 2n

• x∈{0,1}n

f(x)χs(x)

Map f →

f is proportional to unitary (length-preserving)

Fourier analysis of Boolean functions: Some beautiful examples – p.4/13

Fourier analysis over the Boolean cube

Consider the space of functions f : {0, 1}n → R, with normalized inner product f, g = 1

2n

• x∈{0,1}n

f(x)g(x)

The parity-functions χs(x) = (−1)x·s =

i:si=1(−1)xi

form an orthonormal basis of this space Hence we can write f =

• s∈{0,1}n
• f(s)χs

with

f(s) = f, χs = 1 2n

• x∈{0,1}n

f(x)χs(x)

Map f →

f is proportional to unitary (length-preserving) ⇒ 1 2n

• x

f(x)2 =

• s
• f(s)2 (Parseval’s identity)

Fourier analysis of Boolean functions: Some beautiful examples – p.4/13

Examples

Fourier analysis of Boolean functions: Some beautiful examples – p.5/13

Examples

OR on 2 bits:

Fourier analysis of Boolean functions: Some beautiful examples – p.5/13

Examples

OR on 2 bits:

f(x1, x2) = OR(x1, x2) ∈ {0, 1}

Fourier analysis of Boolean functions: Some beautiful examples – p.5/13

Examples

OR on 2 bits:

f(x1, x2) = OR(x1, x2) ∈ {0, 1}

• f(00)

Fourier analysis of Boolean functions: Some beautiful examples – p.5/13

Examples

OR on 2 bits:

f(x1, x2) = OR(x1, x2) ∈ {0, 1}

• f(00) = 1

4

• x∈{0,1}n f(x)χ00(x)

Fourier analysis of Boolean functions: Some beautiful examples – p.5/13

Examples

OR on 2 bits:

f(x1, x2) = OR(x1, x2) ∈ {0, 1}

• f(00) = 1

4

• x∈{0,1}n f(x)χ00(x) = 1

4(0 + 1 + 1 + 1)

Fourier analysis of Boolean functions: Some beautiful examples – p.5/13

Examples

OR on 2 bits:

f(x1, x2) = OR(x1, x2) ∈ {0, 1}

• f(00) = 1

4

• x∈{0,1}n f(x)χ00(x) = 1

4(0 + 1 + 1 + 1) = 3 4

Fourier analysis of Boolean functions: Some beautiful examples – p.5/13

Examples

OR on 2 bits:

f(x1, x2) = OR(x1, x2) ∈ {0, 1}

• f(00) = 1

4

• x∈{0,1}n f(x)χ00(x) = 1

4(0 + 1 + 1 + 1) = 3 4

• f(01) = −1

4,

Fourier analysis of Boolean functions: Some beautiful examples – p.5/13

Examples

OR on 2 bits:

f(x1, x2) = OR(x1, x2) ∈ {0, 1}

• f(00) = 1

4

• x∈{0,1}n f(x)χ00(x) = 1

4(0 + 1 + 1 + 1) = 3 4

• f(01) = −1

4,

f(10) = −1

4,

Fourier analysis of Boolean functions: Some beautiful examples – p.5/13

Examples

OR on 2 bits:

f(x1, x2) = OR(x1, x2) ∈ {0, 1}

• f(00) = 1

4

• x∈{0,1}n f(x)χ00(x) = 1

4(0 + 1 + 1 + 1) = 3 4

• f(01) = −1

4,

f(10) = −1

4,

f(11) = −1

4

Fourier analysis of Boolean functions: Some beautiful examples – p.5/13

Examples

OR on 2 bits:

f(x1, x2) = OR(x1, x2) ∈ {0, 1}

• f(00) = 1

4

• x∈{0,1}n f(x)χ00(x) = 1

4(0 + 1 + 1 + 1) = 3 4

• f(01) = −1

4,

f(10) = −1

4,

f(11) = −1

4

Note:

f(00) = Expx[f(x)]

Fourier analysis of Boolean functions: Some beautiful examples – p.5/13

Examples

OR on 2 bits:

f(x1, x2) = OR(x1, x2) ∈ {0, 1}

• f(00) = 1

4

• x∈{0,1}n f(x)χ00(x) = 1

4(0 + 1 + 1 + 1) = 3 4

• f(01) = −1

4,

f(10) = −1

4,

f(11) = −1

4

Note:

f(00) = Expx[f(x)], and f(00) =

s

f(s)

Fourier analysis of Boolean functions: Some beautiful examples – p.5/13

Examples

OR on 2 bits:

f(x1, x2) = OR(x1, x2) ∈ {0, 1}

• f(00) = 1

4

• x∈{0,1}n f(x)χ00(x) = 1

4(0 + 1 + 1 + 1) = 3 4

• f(01) = −1

4,

f(10) = −1

4,

f(11) = −1

4

Note:

f(00) = Expx[f(x)], and f(00) =

s

f(s)

Parseval: 1

4

• x

f(x)2

Fourier analysis of Boolean functions: Some beautiful examples – p.5/13

Examples

OR on 2 bits:

f(x1, x2) = OR(x1, x2) ∈ {0, 1}

• f(00) = 1

4

• x∈{0,1}n f(x)χ00(x) = 1

4(0 + 1 + 1 + 1) = 3 4

• f(01) = −1

4,

f(10) = −1

4,

f(11) = −1

4

Note:

f(00) = Expx[f(x)], and f(00) =

s

f(s)

Parseval: 1

4

• x

f(x)2 = 3 4

Fourier analysis of Boolean functions: Some beautiful examples – p.5/13

Examples

OR on 2 bits:

f(x1, x2) = OR(x1, x2) ∈ {0, 1}

• f(00) = 1

4

• x∈{0,1}n f(x)χ00(x) = 1

4(0 + 1 + 1 + 1) = 3 4

• f(01) = −1

4,

f(10) = −1

4,

f(11) = −1

4

Note:

f(00) = Expx[f(x)], and f(00) =

s

f(s)

Parseval: 1

4

• x

f(x)2 = 3 4 =

• s
• f(s)2

Fourier analysis of Boolean functions: Some beautiful examples – p.5/13

Examples

OR on 2 bits:

f(x1, x2) = OR(x1, x2) ∈ {0, 1}

• f(00) = 1

4

• x∈{0,1}n f(x)χ00(x) = 1

4(0 + 1 + 1 + 1) = 3 4

• f(01) = −1

4,

f(10) = −1

4,

f(11) = −1

4

Note:

f(00) = Expx[f(x)], and f(00) =

s

f(s)

Parseval: 1

4

• x

f(x)2 = 3 4 =

• s
• f(s)2

PARITY on n bits

Fourier analysis of Boolean functions: Some beautiful examples – p.5/13

Examples

OR on 2 bits:

f(x1, x2) = OR(x1, x2) ∈ {0, 1}

• f(00) = 1

4

• x∈{0,1}n f(x)χ00(x) = 1

4(0 + 1 + 1 + 1) = 3 4

• f(01) = −1

4,

f(10) = −1

4,

f(11) = −1

4

Note:

f(00) = Expx[f(x)], and f(00) =

s

f(s)

Parseval: 1

4

• x

f(x)2 = 3 4 =

• s
• f(s)2

PARITY on n bits, with TRUE= −1, FALSE= +1:

Fourier analysis of Boolean functions: Some beautiful examples – p.5/13

Examples

OR on 2 bits:

f(x1, x2) = OR(x1, x2) ∈ {0, 1}

• f(00) = 1

4

• x∈{0,1}n f(x)χ00(x) = 1

4(0 + 1 + 1 + 1) = 3 4

• f(01) = −1

4,

f(10) = −1

4,

f(11) = −1

4

Note:

f(00) = Expx[f(x)], and f(00) =

s

f(s)

Parseval: 1

4

• x

f(x)2 = 3 4 =

• s
• f(s)2

PARITY on n bits, with TRUE= −1, FALSE= +1:

f(x)

Fourier analysis of Boolean functions: Some beautiful examples – p.5/13

Examples

OR on 2 bits:

f(x1, x2) = OR(x1, x2) ∈ {0, 1}

• f(00) = 1

4

• x∈{0,1}n f(x)χ00(x) = 1

4(0 + 1 + 1 + 1) = 3 4

• f(01) = −1

4,

f(10) = −1

4,

f(11) = −1

4

Note:

f(00) = Expx[f(x)], and f(00) =

s

f(s)

Parseval: 1

4

• x

f(x)2 = 3 4 =

• s
• f(s)2

PARITY on n bits, with TRUE= −1, FALSE= +1:

f(x) = χ1n(x) = (−1)|x|

Fourier analysis of Boolean functions: Some beautiful examples – p.5/13

Examples

OR on 2 bits:

f(x1, x2) = OR(x1, x2) ∈ {0, 1}

• f(00) = 1

4

• x∈{0,1}n f(x)χ00(x) = 1

4(0 + 1 + 1 + 1) = 3 4

• f(01) = −1

4,

f(10) = −1

4,

f(11) = −1

4

Note:

f(00) = Expx[f(x)], and f(00) =

s

f(s)

Parseval: 1

4

• x

f(x)2 = 3 4 =

• s
• f(s)2

PARITY on n bits, with TRUE= −1, FALSE= +1:

f(x) = χ1n(x) = (−1)|x|, so f(1n) = 1

Fourier analysis of Boolean functions: Some beautiful examples – p.5/13

Examples

OR on 2 bits:

f(x1, x2) = OR(x1, x2) ∈ {0, 1}

• f(00) = 1

4

• x∈{0,1}n f(x)χ00(x) = 1

4(0 + 1 + 1 + 1) = 3 4

• f(01) = −1

4,

f(10) = −1

4,

f(11) = −1

4

Note:

f(00) = Expx[f(x)], and f(00) =

s

f(s)

Parseval: 1

4

• x

f(x)2 = 3 4 =

• s
• f(s)2

PARITY on n bits, with TRUE= −1, FALSE= +1:

f(x) = χ1n(x) = (−1)|x|, so f(1n) = 1

all other

f(s) are 0

Fourier analysis of Boolean functions: Some beautiful examples – p.5/13

(1) Approximating functions with parities

Fourier analysis of Boolean functions: Some beautiful examples – p.6/13

(1) Approximating functions with parities

Suppose f : {0, 1}n → {±1} has small Fourier degree d

Fourier analysis of Boolean functions: Some beautiful examples – p.6/13

(1) Approximating functions with parities

Suppose f : {0, 1}n → {±1} has small Fourier degree d:

f =

• s:|s|≤d
• f(s)χs

Fourier analysis of Boolean functions: Some beautiful examples – p.6/13

(1) Approximating functions with parities

Suppose f : {0, 1}n → {±1} has small Fourier degree d:

f =

• s:|s|≤d
• f(s)χs

Then there exists a parity-function on at most d bits that has non-trivial correlation with f

Fourier analysis of Boolean functions: Some beautiful examples – p.6/13

(1) Approximating functions with parities

Suppose f : {0, 1}n → {±1} has small Fourier degree d:

f =

• s:|s|≤d
• f(s)χs

Then there exists a parity-function on at most d bits that has non-trivial correlation with f Why?

Fourier analysis of Boolean functions: Some beautiful examples – p.6/13

(1) Approximating functions with parities

Suppose f : {0, 1}n → {±1} has small Fourier degree d:

f =

• s:|s|≤d
• f(s)χs

Then there exists a parity-function on at most d bits that has non-trivial correlation with f Why?

• s:|s|≤d
• f(s)2

Fourier analysis of Boolean functions: Some beautiful examples – p.6/13

(1) Approximating functions with parities

Suppose f : {0, 1}n → {±1} has small Fourier degree d:

f =

• s:|s|≤d
• f(s)χs

Then there exists a parity-function on at most d bits that has non-trivial correlation with f Why?

• s:|s|≤d
• f(s)2 = 1

2n

• x∈{0,1}n

f(x)2 = 1 (Parseval).

Fourier analysis of Boolean functions: Some beautiful examples – p.6/13

(1) Approximating functions with parities

Suppose f : {0, 1}n → {±1} has small Fourier degree d:

f =

• s:|s|≤d
• f(s)χs

Then there exists a parity-function on at most d bits that has non-trivial correlation with f Why?

• s:|s|≤d
• f(s)2 = 1

2n

• x∈{0,1}n

f(x)2 = 1 (Parseval).

This is a sum over ≤ nd terms.

Fourier analysis of Boolean functions: Some beautiful examples – p.6/13

(1) Approximating functions with parities

Suppose f : {0, 1}n → {±1} has small Fourier degree d:

f =

• s:|s|≤d
• f(s)χs

Then there exists a parity-function on at most d bits that has non-trivial correlation with f Why?

• s:|s|≤d
• f(s)2 = 1

2n

• x∈{0,1}n

f(x)2 = 1 (Parseval).

This is a sum over ≤ nd terms. Hence ∃s with

Fourier analysis of Boolean functions: Some beautiful examples – p.6/13

(1) Approximating functions with parities

Suppose f : {0, 1}n → {±1} has small Fourier degree d:

f =

• s:|s|≤d
• f(s)χs

Then there exists a parity-function on at most d bits that has non-trivial correlation with f Why?

• s:|s|≤d
• f(s)2 = 1

2n

• x∈{0,1}n

f(x)2 = 1 (Parseval).

This is a sum over ≤ nd terms. Hence ∃s with

1 nd ≤ f(s)2

Fourier analysis of Boolean functions: Some beautiful examples – p.6/13

(1) Approximating functions with parities

Suppose f : {0, 1}n → {±1} has small Fourier degree d:

f =

• s:|s|≤d
• f(s)χs

Then there exists a parity-function on at most d bits that has non-trivial correlation with f Why?

• s:|s|≤d
• f(s)2 = 1

2n

• x∈{0,1}n

f(x)2 = 1 (Parseval).

This is a sum over ≤ nd terms. Hence ∃s with

1 nd ≤ f(s)2 = | 1 2n

• x∈{0,1}n

f(x)χs(x)|2

Fourier analysis of Boolean functions: Some beautiful examples – p.6/13

(1) Approximating functions with parities

Suppose f : {0, 1}n → {±1} has small Fourier degree d:

f =

• s:|s|≤d
• f(s)χs

Then there exists a parity-function on at most d bits that has non-trivial correlation with f Why?

• s:|s|≤d
• f(s)2 = 1

2n

• x∈{0,1}n

f(x)2 = 1 (Parseval).

This is a sum over ≤ nd terms. Hence ∃s with

1 nd ≤ f(s)2 = | 1 2n

• x∈{0,1}n

f(x)χs(x)|2

So χs (or its negation) has non-trivial correlation with f

Fourier analysis of Boolean functions: Some beautiful examples – p.6/13

(2) Learning from uniform examples

Fourier analysis of Boolean functions: Some beautiful examples – p.7/13

(2) Learning from uniform examples

A Fourier coefficient is just a uniform expectation

Fourier analysis of Boolean functions: Some beautiful examples – p.7/13

(2) Learning from uniform examples

A Fourier coefficient is just a uniform expectation:

• f(s) = 1

2n

• x∈{0,1}n

f(x)χs(x)

Fourier analysis of Boolean functions: Some beautiful examples – p.7/13

(2) Learning from uniform examples

A Fourier coefficient is just a uniform expectation:

• f(s) = 1

2n

• x∈{0,1}n

f(x)χs(x) = Expx[f(x)χs(x)]

Fourier analysis of Boolean functions: Some beautiful examples – p.7/13

(2) Learning from uniform examples

A Fourier coefficient is just a uniform expectation:

• f(s) = 1

2n

• x∈{0,1}n

f(x)χs(x) = Expx[f(x)χs(x)]

We can approximate this given uniformly random examples (x1, f(x1)), . . . , (xm, f(xm)):

Fourier analysis of Boolean functions: Some beautiful examples – p.7/13

(2) Learning from uniform examples

A Fourier coefficient is just a uniform expectation:

• f(s) = 1

2n

• x∈{0,1}n

f(x)χs(x) = Expx[f(x)χs(x)]

We can approximate this given uniformly random examples (x1, f(x1)), . . . , (xm, f(xm)):

1 m

m

• i=1

f(xi)χs(xi)

Fourier analysis of Boolean functions: Some beautiful examples – p.7/13

(2) Learning from uniform examples

A Fourier coefficient is just a uniform expectation:

• f(s) = 1

2n

• x∈{0,1}n

f(x)χs(x) = Expx[f(x)χs(x)]

We can approximate this given uniformly random examples (x1, f(x1)), . . . , (xm, f(xm)):

1 m

m

• i=1

f(xi)χs(xi) → f(s)

Fourier analysis of Boolean functions: Some beautiful examples – p.7/13

(2) Learning from uniform examples

A Fourier coefficient is just a uniform expectation:

• f(s) = 1

2n

• x∈{0,1}n

f(x)χs(x) = Expx[f(x)χs(x)]

We can approximate this given uniformly random examples (x1, f(x1)), . . . , (xm, f(xm)):

1 m

m

• i=1

f(xi)χs(xi) → f(s)

Converges fast if |

f(s)| is not too small (Chernoff)

Fourier analysis of Boolean functions: Some beautiful examples – p.7/13

(2) Learning from uniform examples

A Fourier coefficient is just a uniform expectation:

• f(s) = 1

2n

• x∈{0,1}n

f(x)χs(x) = Expx[f(x)χs(x)]

We can approximate this given uniformly random examples (x1, f(x1)), . . . , (xm, f(xm)):

1 m

m

• i=1

f(xi)χs(xi) → f(s)

Converges fast if |

f(s)| is not too small (Chernoff)

Hence we can quickly learn (approximate) an unknown function f that is dominated by a few large coefficients

Fourier analysis of Boolean functions: Some beautiful examples – p.7/13

(2) Learning from uniform examples

A Fourier coefficient is just a uniform expectation:

• f(s) = 1

2n

• x∈{0,1}n

f(x)χs(x) = Expx[f(x)χs(x)]

We can approximate this given uniformly random examples (x1, f(x1)), . . . , (xm, f(xm)):

1 m

m

• i=1

f(xi)χs(xi) → f(s)

Converges fast if |

f(s)| is not too small (Chernoff)

Hence we can quickly learn (approximate) an unknown function f that is dominated by a few large coefficients (example from LMN 89: AC0-circuits)

Fourier analysis of Boolean functions: Some beautiful examples – p.7/13

(3) List-decoding of Hadamard code

Fourier analysis of Boolean functions: Some beautiful examples – p.8/13

(3) List-decoding of Hadamard code

Error-correcting code: E : {0, 1}n → {0, 1}m

Fourier analysis of Boolean functions: Some beautiful examples – p.8/13

(3) List-decoding of Hadamard code

Error-correcting code: E : {0, 1}n → {0, 1}m If all codewords have distance d(E(x), E(y)) ≥ 2e + 1, then we can uniquely recover x from corrupted codeword w ∈ {0, 1}m with e errors

Fourier analysis of Boolean functions: Some beautiful examples – p.8/13

(3) List-decoding of Hadamard code

Error-correcting code: E : {0, 1}n → {0, 1}m If all codewords have distance d(E(x), E(y)) ≥ 2e + 1, then we can uniquely recover x from corrupted codeword w ∈ {0, 1}m with e errors (d(w, E(x)) = e)

Fourier analysis of Boolean functions: Some beautiful examples – p.8/13

(3) List-decoding of Hadamard code

Error-correcting code: E : {0, 1}n → {0, 1}m If all codewords have distance d(E(x), E(y)) ≥ 2e + 1, then we can uniquely recover x from corrupted codeword w ∈ {0, 1}m with e errors (d(w, E(x)) = e) Hadamard code (m = 2n):

Fourier analysis of Boolean functions: Some beautiful examples – p.8/13

(3) List-decoding of Hadamard code

Error-correcting code: E : {0, 1}n → {0, 1}m If all codewords have distance d(E(x), E(y)) ≥ 2e + 1, then we can uniquely recover x from corrupted codeword w ∈ {0, 1}m with e errors (d(w, E(x)) = e) Hadamard code (m = 2n): E(x)y = x · y mod 2

Fourier analysis of Boolean functions: Some beautiful examples – p.8/13

(3) List-decoding of Hadamard code

Error-correcting code: E : {0, 1}n → {0, 1}m If all codewords have distance d(E(x), E(y)) ≥ 2e + 1, then we can uniquely recover x from corrupted codeword w ∈ {0, 1}m with e errors (d(w, E(x)) = e) Hadamard code (m = 2n): E(x)y = x · y mod 2 All codewords are at distance m/2

Fourier analysis of Boolean functions: Some beautiful examples – p.8/13

(3) List-decoding of Hadamard code

Error-correcting code: E : {0, 1}n → {0, 1}m If all codewords have distance d(E(x), E(y)) ≥ 2e + 1, then we can uniquely recover x from corrupted codeword w ∈ {0, 1}m with e errors (d(w, E(x)) = e) Hadamard code (m = 2n): E(x)y = x · y mod 2 All codewords are at distance m/2 ⇒ given w with

e < m/4 errors, there is a unique x with d(w, E(x)) ≤ e

Fourier analysis of Boolean functions: Some beautiful examples – p.8/13

(3) List-decoding of Hadamard code

Error-correcting code: E : {0, 1}n → {0, 1}m If all codewords have distance d(E(x), E(y)) ≥ 2e + 1, then we can uniquely recover x from corrupted codeword w ∈ {0, 1}m with e errors (d(w, E(x)) = e) Hadamard code (m = 2n): E(x)y = x · y mod 2 All codewords are at distance m/2 ⇒ given w with

e < m/4 errors, there is a unique x with d(w, E(x)) ≤ e

Problem: if e ≥ m/4 errors, then there may be many different x with d(w, E(x)) ≤ e

Fourier analysis of Boolean functions: Some beautiful examples – p.8/13

(3) List-decoding of Hadamard code

Error-correcting code: E : {0, 1}n → {0, 1}m If all codewords have distance d(E(x), E(y)) ≥ 2e + 1, then we can uniquely recover x from corrupted codeword w ∈ {0, 1}m with e errors (d(w, E(x)) = e) Hadamard code (m = 2n): E(x)y = x · y mod 2 All codewords are at distance m/2 ⇒ given w with

e < m/4 errors, there is a unique x with d(w, E(x)) ≤ e

Problem: if e ≥ m/4 errors, then there may be many different x with d(w, E(x)) ≤ e Example: w = 03m/41m/4 could’ve come from codewords

E(0n) = 0m or E(10n−1) = 0m/21m/2

Fourier analysis of Boolean functions: Some beautiful examples – p.8/13

(3) List-decoding of Hadamard code

Error-correcting code: E : {0, 1}n → {0, 1}m If all codewords have distance d(E(x), E(y)) ≥ 2e + 1, then we can uniquely recover x from corrupted codeword w ∈ {0, 1}m with e errors (d(w, E(x)) = e) Hadamard code (m = 2n): E(x)y = x · y mod 2 All codewords are at distance m/2 ⇒ given w with

e < m/4 errors, there is a unique x with d(w, E(x)) ≤ e

Problem: if e ≥ m/4 errors, then there may be many different x with d(w, E(x)) ≤ e Example: w = 03m/41m/4 could’ve come from codewords

E(0n) = 0m or E(10n−1) = 0m/21m/2

List-decoding: output the whole list (hopefully small)

Fourier analysis of Boolean functions: Some beautiful examples – p.8/13

List-decoding of Hadamard code (cntd)

Fourier analysis of Boolean functions: Some beautiful examples – p.9/13

List-decoding of Hadamard code (cntd)

List-decoding: given corrupted codeword w ∈ {0, 1}m and error bound e, output list {x : d(w, E(x)) ≤ e}

Fourier analysis of Boolean functions: Some beautiful examples – p.9/13

List-decoding of Hadamard code (cntd)

List-decoding: given corrupted codeword w ∈ {0, 1}m and error bound e, output list {x : d(w, E(x)) ≤ e} For Hadamard code: if e ≤ (1/2 − ε)m, then this list has only O(1/ε2) elements!

Fourier analysis of Boolean functions: Some beautiful examples – p.9/13

List-decoding of Hadamard code (cntd)

List-decoding: given corrupted codeword w ∈ {0, 1}m and error bound e, output list {x : d(w, E(x)) ≤ e} For Hadamard code: if e ≤ (1/2 − ε)m, then this list has only O(1/ε2) elements! Why?

Fourier analysis of Boolean functions: Some beautiful examples – p.9/13

List-decoding of Hadamard code (cntd)

List-decoding: given corrupted codeword w ∈ {0, 1}m and error bound e, output list {x : d(w, E(x)) ≤ e} For Hadamard code: if e ≤ (1/2 − ε)m, then this list has only O(1/ε2) elements! Why? Fourier analysis!

Fourier analysis of Boolean functions: Some beautiful examples – p.9/13

List-decoding of Hadamard code (cntd)

List-decoding: given corrupted codeword w ∈ {0, 1}m and error bound e, output list {x : d(w, E(x)) ≤ e} For Hadamard code: if e ≤ (1/2 − ε)m, then this list has only O(1/ε2) elements! Why? Fourier analysis!

• 1. View w as function w : {0, 1}n → {±1}, and E(s) = χs

Fourier analysis of Boolean functions: Some beautiful examples – p.9/13

List-decoding of Hadamard code (cntd)

List-decoding: given corrupted codeword w ∈ {0, 1}m and error bound e, output list {x : d(w, E(x)) ≤ e} For Hadamard code: if e ≤ (1/2 − ε)m, then this list has only O(1/ε2) elements! Why? Fourier analysis!

• 1. View w as function w : {0, 1}n → {±1}, and E(s) = χs
• 2. If d(w, E(s)) ≤ (1/2 − ε)m, then

w(s) ≥ 2ε

Fourier analysis of Boolean functions: Some beautiful examples – p.9/13

List-decoding of Hadamard code (cntd)

List-decoding: given corrupted codeword w ∈ {0, 1}m and error bound e, output list {x : d(w, E(x)) ≤ e} For Hadamard code: if e ≤ (1/2 − ε)m, then this list has only O(1/ε2) elements! Why? Fourier analysis!

• 1. View w as function w : {0, 1}n → {±1}, and E(s) = χs
• 2. If d(w, E(s)) ≤ (1/2 − ε)m, then

w(s) ≥ 2ε

3.

s

w(s)2 = 1 (by Parseval)

Fourier analysis of Boolean functions: Some beautiful examples – p.9/13

List-decoding of Hadamard code (cntd)

List-decoding: given corrupted codeword w ∈ {0, 1}m and error bound e, output list {x : d(w, E(x)) ≤ e} For Hadamard code: if e ≤ (1/2 − ε)m, then this list has only O(1/ε2) elements! Why? Fourier analysis!

• 1. View w as function w : {0, 1}n → {±1}, and E(s) = χs
• 2. If d(w, E(s)) ≤ (1/2 − ε)m, then

w(s) ≥ 2ε

3.

s

w(s)2 = 1 (by Parseval),

hence at most

1 4ε2 different s satisfy

w(s) ≥ 2ε

Fourier analysis of Boolean functions: Some beautiful examples – p.9/13

List-decoding of Hadamard code (cntd)

List-decoding: given corrupted codeword w ∈ {0, 1}m and error bound e, output list {x : d(w, E(x)) ≤ e} For Hadamard code: if e ≤ (1/2 − ε)m, then this list has only O(1/ε2) elements! Why? Fourier analysis!

• 1. View w as function w : {0, 1}n → {±1}, and E(s) = χs
• 2. If d(w, E(s)) ≤ (1/2 − ε)m, then

w(s) ≥ 2ε

3.

s

w(s)2 = 1 (by Parseval),

hence at most

1 4ε2 different s satisfy

w(s) ≥ 2ε

Goldreich and Levin show how to find this list efficiently

Fourier analysis of Boolean functions: Some beautiful examples – p.9/13

List-decoding of Hadamard code (cntd)

List-decoding: given corrupted codeword w ∈ {0, 1}m and error bound e, output list {x : d(w, E(x)) ≤ e} For Hadamard code: if e ≤ (1/2 − ε)m, then this list has only O(1/ε2) elements! Why? Fourier analysis!

• 1. View w as function w : {0, 1}n → {±1}, and E(s) = χs
• 2. If d(w, E(s)) ≤ (1/2 − ε)m, then

w(s) ≥ 2ε

3.

s

w(s)2 = 1 (by Parseval),

hence at most

1 4ε2 different s satisfy

w(s) ≥ 2ε

Goldreich and Levin show how to find this list efficiently There are codes with much better rate that are still efficiently list-decodable

Fourier analysis of Boolean functions: Some beautiful examples – p.9/13

List-decoding of Hadamard code (cntd)

List-decoding: given corrupted codeword w ∈ {0, 1}m and error bound e, output list {x : d(w, E(x)) ≤ e} For Hadamard code: if e ≤ (1/2 − ε)m, then this list has only O(1/ε2) elements! Why? Fourier analysis!

• 1. View w as function w : {0, 1}n → {±1}, and E(s) = χs
• 2. If d(w, E(s)) ≤ (1/2 − ε)m, then

w(s) ≥ 2ε

3.

s

w(s)2 = 1 (by Parseval),

hence at most

1 4ε2 different s satisfy

w(s) ≥ 2ε

Goldreich and Levin show how to find this list efficiently There are codes with much better rate that are still efficiently list-decodable (e.g. Reed-Solomon)

Fourier analysis of Boolean functions: Some beautiful examples – p.9/13

(4) Influence of variables

Fourier analysis of Boolean functions: Some beautiful examples – p.10/13

(4) Influence of variables

Consider a Boolean function f : {0, 1}n → {0, 1}

Fourier analysis of Boolean functions: Some beautiful examples – p.10/13

(4) Influence of variables

Consider a Boolean function f : {0, 1}n → {0, 1} The influence of variable i is the probability that xi determines the function value

Fourier analysis of Boolean functions: Some beautiful examples – p.10/13

(4) Influence of variables

Consider a Boolean function f : {0, 1}n → {0, 1} The influence of variable i is the probability that xi determines the function value:

Inff(i) =

Pr

x∈{0,1}n[f(x) = f(x ⊕ ei)]

Fourier analysis of Boolean functions: Some beautiful examples – p.10/13

(4) Influence of variables

Consider a Boolean function f : {0, 1}n → {0, 1} The influence of variable i is the probability that xi determines the function value:

Inff(i) =

Pr

x∈{0,1}n[f(x) = f(x ⊕ ei)]

For things like voting and distributed coin-flipping: would like to find a balanced f where each Inff(i) ≈ 1/n

Fourier analysis of Boolean functions: Some beautiful examples – p.10/13

(4) Influence of variables

Consider a Boolean function f : {0, 1}n → {0, 1} The influence of variable i is the probability that xi determines the function value:

Inff(i) =

Pr

x∈{0,1}n[f(x) = f(x ⊕ ei)]

For things like voting and distributed coin-flipping: would like to find a balanced f where each Inff(i) ≈ 1/n KKL 88: if f is balanced, then there always is an i with Inff(i) ≥ log(n)/n

Fourier analysis of Boolean functions: Some beautiful examples – p.10/13

(4) Influence of variables

Consider a Boolean function f : {0, 1}n → {0, 1} The influence of variable i is the probability that xi determines the function value:

Inff(i) =

Pr

x∈{0,1}n[f(x) = f(x ⊕ ei)]

For things like voting and distributed coin-flipping: would like to find a balanced f where each Inff(i) ≈ 1/n KKL 88: if f is balanced, then there always is an i with Inff(i) ≥ log(n)/n This implies there is a set of O(n/ log(n)) variables that controls f with high probability

Fourier analysis of Boolean functions: Some beautiful examples – p.10/13

The KKL proof

Fourier analysis of Boolean functions: Some beautiful examples – p.11/13

The KKL proof

Define fi(x) = f(x) − f(x ⊕ ei)

Fourier analysis of Boolean functions: Some beautiful examples – p.11/13

The KKL proof

Define fi(x) = f(x) − f(x ⊕ ei) ∈ {−1, 0, +1}

Fourier analysis of Boolean functions: Some beautiful examples – p.11/13

The KKL proof

Define fi(x) = f(x) − f(x ⊕ ei) ∈ {−1, 0, +1} Then

fi(s) = 2 f(s) if si = 1

Fourier analysis of Boolean functions: Some beautiful examples – p.11/13

The KKL proof

Define fi(x) = f(x) − f(x ⊕ ei) ∈ {−1, 0, +1} Then

fi(s) = 2 f(s) if si = 1 , and fi(s) = 0 if si = 0

Fourier analysis of Boolean functions: Some beautiful examples – p.11/13

The KKL proof

Define fi(x) = f(x) − f(x ⊕ ei) ∈ {−1, 0, +1} Then

fi(s) = 2 f(s) if si = 1 , and fi(s) = 0 if si = 0

Inff(i)

Fourier analysis of Boolean functions: Some beautiful examples – p.11/13

The KKL proof

Define fi(x) = f(x) − f(x ⊕ ei) ∈ {−1, 0, +1} Then

fi(s) = 2 f(s) if si = 1 , and fi(s) = 0 if si = 0

Inff(i) = Pr[fi = 0]

Fourier analysis of Boolean functions: Some beautiful examples – p.11/13

The KKL proof

Define fi(x) = f(x) − f(x ⊕ ei) ∈ {−1, 0, +1} Then

fi(s) = 2 f(s) if si = 1 , and fi(s) = 0 if si = 0

Inff(i) = Pr[fi = 0] = Exp[f2

i ]

Fourier analysis of Boolean functions: Some beautiful examples – p.11/13

The KKL proof

Define fi(x) = f(x) − f(x ⊕ ei) ∈ {−1, 0, +1} Then

fi(s) = 2 f(s) if si = 1 , and fi(s) = 0 if si = 0

Inff(i) = Pr[fi = 0] = Exp[f2

i ] =

• s
• fi(s)2

Fourier analysis of Boolean functions: Some beautiful examples – p.11/13

The KKL proof

Define fi(x) = f(x) − f(x ⊕ ei) ∈ {−1, 0, +1} Then

fi(s) = 2 f(s) if si = 1 , and fi(s) = 0 if si = 0

Inff(i) = Pr[fi = 0] = Exp[f2

i ] =

• s
• fi(s)2 = 4
• s:si=1
• f(s)2

Fourier analysis of Boolean functions: Some beautiful examples – p.11/13

The KKL proof

Define fi(x) = f(x) − f(x ⊕ ei) ∈ {−1, 0, +1} Then

fi(s) = 2 f(s) if si = 1 , and fi(s) = 0 if si = 0

Inff(i) = Pr[fi = 0] = Exp[f2

i ] =

• s
• fi(s)2 = 4
• s:si=1
• f(s)2

If L =

s:|s|>log n

f(s)2 ≥ 1/3

Fourier analysis of Boolean functions: Some beautiful examples – p.11/13

The KKL proof

Define fi(x) = f(x) − f(x ⊕ ei) ∈ {−1, 0, +1} Then

fi(s) = 2 f(s) if si = 1 , and fi(s) = 0 if si = 0

Inff(i) = Pr[fi = 0] = Exp[f2

i ] =

• s
• fi(s)2 = 4
• s:si=1
• f(s)2

If L =

s:|s|>log n

f(s)2 ≥ 1/3, then n

i=1 Inff(i)

Fourier analysis of Boolean functions: Some beautiful examples – p.11/13

The KKL proof

Define fi(x) = f(x) − f(x ⊕ ei) ∈ {−1, 0, +1} Then

fi(s) = 2 f(s) if si = 1 , and fi(s) = 0 if si = 0

Inff(i) = Pr[fi = 0] = Exp[f2

i ] =

• s
• fi(s)2 = 4
• s:si=1
• f(s)2

If L =

s:|s|>log n

f(s)2 ≥ 1/3, then n

i=1 Inff(i)

= 4

s |s|

f(s)2

Fourier analysis of Boolean functions: Some beautiful examples – p.11/13

The KKL proof

Define fi(x) = f(x) − f(x ⊕ ei) ∈ {−1, 0, +1} Then

fi(s) = 2 f(s) if si = 1 , and fi(s) = 0 if si = 0

Inff(i) = Pr[fi = 0] = Exp[f2

i ] =

• s
• fi(s)2 = 4
• s:si=1
• f(s)2

If L =

s:|s|>log n

f(s)2 ≥ 1/3, then n

i=1 Inff(i)

= 4

s |s|

f(s)2 ≥ Ω(log n)

Fourier analysis of Boolean functions: Some beautiful examples – p.11/13

The KKL proof

Define fi(x) = f(x) − f(x ⊕ ei) ∈ {−1, 0, +1} Then

fi(s) = 2 f(s) if si = 1 , and fi(s) = 0 if si = 0

Inff(i) = Pr[fi = 0] = Exp[f2

i ] =

• s
• fi(s)2 = 4
• s:si=1
• f(s)2

If L =

s:|s|>log n

f(s)2 ≥ 1/3, then n

i=1 Inff(i)

= 4

s |s|

f(s)2 ≥ Ω(log n) ⇒ maxi Inff(i) ≥ Ω(log(n)/n)

Fourier analysis of Boolean functions: Some beautiful examples – p.11/13

The KKL proof

Define fi(x) = f(x) − f(x ⊕ ei) ∈ {−1, 0, +1} Then

fi(s) = 2 f(s) if si = 1 , and fi(s) = 0 if si = 0

Inff(i) = Pr[fi = 0] = Exp[f2

i ] =

• s
• fi(s)2 = 4
• s:si=1
• f(s)2

If L =

s:|s|>log n

f(s)2 ≥ 1/3, then n

i=1 Inff(i)

= 4

s |s|

f(s)2 ≥ Ω(log n) ⇒ maxi Inff(i) ≥ Ω(log(n)/n)

If L < 1/3, then use KKL inequality (special case of Bonami-Beckner):

Fourier analysis of Boolean functions: Some beautiful examples – p.11/13

The KKL proof

Define fi(x) = f(x) − f(x ⊕ ei) ∈ {−1, 0, +1} Then

fi(s) = 2 f(s) if si = 1 , and fi(s) = 0 if si = 0

Inff(i) = Pr[fi = 0] = Exp[f2

i ] =

• s
• fi(s)2 = 4
• s:si=1
• f(s)2

If L =

s:|s|>log n

f(s)2 ≥ 1/3, then n

i=1 Inff(i)

= 4

s |s|

f(s)2 ≥ Ω(log n) ⇒ maxi Inff(i) ≥ Ω(log(n)/n)

If L < 1/3, then use KKL inequality (special case of Bonami-Beckner): ∀g : {0, 1}n → {−1, 0, +1}, δ ∈ [0, 1]

• s∈{0,1}n

δ|s| g(s)2 ≤ Pr[g = 0]2/(1+δ)

Fourier analysis of Boolean functions: Some beautiful examples – p.11/13

The KKL proof

Define fi(x) = f(x) − f(x ⊕ ei) ∈ {−1, 0, +1} Then

fi(s) = 2 f(s) if si = 1 , and fi(s) = 0 if si = 0

Inff(i) = Pr[fi = 0] = Exp[f2

i ] =

• s
• fi(s)2 = 4
• s:si=1
• f(s)2

If L =

s:|s|>log n

f(s)2 ≥ 1/3, then n

i=1 Inff(i)

= 4

s |s|

f(s)2 ≥ Ω(log n) ⇒ maxi Inff(i) ≥ Ω(log(n)/n)

If L < 1/3, then use KKL inequality (special case of Bonami-Beckner): ∀g : {0, 1}n → {−1, 0, +1}, δ ∈ [0, 1]

• s∈{0,1}n

δ|s| g(s)2 ≤ Pr[g = 0]2/(1+δ)

A calculation shows maxi Inff(i) ≥ Ω(log(n)/n)

Fourier analysis of Boolean functions: Some beautiful examples – p.11/13

Summary

Fourier analysis of Boolean functions: Some beautiful examples – p.12/13

Summary

Fourier analysis of Boolean functions is an increasingly prominent tool in theoretical computer science

Fourier analysis of Boolean functions: Some beautiful examples – p.12/13

Summary

Fourier analysis of Boolean functions is an increasingly prominent tool in theoretical computer science We showed a few simple but beautiful examples:

Fourier analysis of Boolean functions: Some beautiful examples – p.12/13

Summary

Fourier analysis of Boolean functions is an increasingly prominent tool in theoretical computer science We showed a few simple but beautiful examples:

• 1. Approximating low-degree functions by parities

Fourier analysis of Boolean functions: Some beautiful examples – p.12/13

Summary

Fourier analysis of Boolean functions is an increasingly prominent tool in theoretical computer science We showed a few simple but beautiful examples:

• 1. Approximating low-degree functions by parities
• 2. List-decoding of Hadamard codes

Fourier analysis of Boolean functions: Some beautiful examples – p.12/13

Summary

Fourier analysis of Boolean functions is an increasingly prominent tool in theoretical computer science We showed a few simple but beautiful examples:

• 1. Approximating low-degree functions by parities
• 2. List-decoding of Hadamard codes
• 3. Learning under the uniform distribution

Fourier analysis of Boolean functions: Some beautiful examples – p.12/13

Summary

Fourier analysis of Boolean functions is an increasingly prominent tool in theoretical computer science We showed a few simple but beautiful examples:

• 1. Approximating low-degree functions by parities
• 2. List-decoding of Hadamard codes
• 3. Learning under the uniform distribution
• 4. The influence of variables on Boolean functions

Fourier analysis of Boolean functions: Some beautiful examples – p.12/13

Warning: these are powerful techniques!

Fourier analysis of Boolean functions: Some beautiful examples – p.13/13