Martingales and the Method of Bounded Differences Advanced - - PowerPoint PPT Presentation

β–Ά
martingales
SMART_READER_LITE
LIVE PREVIEW

Martingales and the Method of Bounded Differences Advanced - - PowerPoint PPT Presentation

Jan 25, 2024 400 likes β€’2.13k views

Martingales and the Method of Bounded Differences Advanced Algorithms Nanjing University, Fall 2018 (Some) Concentration Inequalities Question: probability that X deviates more than from expectation? (Some) Concentration Inequalities

slide-1
SLIDE 1

Martingales and the Method of Bounded Differences

Advanced Algorithms Nanjing University, Fall 2018

slide-2
SLIDE 2

(Some) Concentration Inequalities

Question: probability that X deviates more than πœ€ from expectation?

slide-3
SLIDE 3

(Some) Concentration Inequalities

Question: probability that X deviates more than πœ€ from expectation?

slide-4
SLIDE 4

Conditional Probability

slide-5
SLIDE 5

Conditional Probability

Example: roll a fair six-sided dice β„°1 = the outcome is six β„°2 = the outcome is an even number

slide-6
SLIDE 6

Conditional Probability

Example: roll a fair six-sided dice β„°1 = the outcome is six β„°2 = the outcome is an even number

slide-7
SLIDE 7

Conditional Expectation

slide-8
SLIDE 8

Conditional Expectation

Example: sample a human being uniformly at random 𝑍: height of the chosen human being π‘Œ: country of origin of the chosen human being

slide-9
SLIDE 9

Conditional Expectation

Example: sample a human being uniformly at random 𝑍: height of the chosen human being π‘Œ: country of origin of the chosen human being 𝔽 𝑍 =?

slide-10
SLIDE 10

Conditional Expectation

Example: sample a human being uniformly at random 𝑍: height of the chosen human being π‘Œ: country of origin of the chosen human being 𝔽 𝑍 =? 𝔽 𝑍 | π‘Œ = "China" =?

slide-11
SLIDE 11

Conditional Expectation

Example: sample a human being uniformly at random 𝑍: height of the chosen human being π‘Œ: country of origin of the chosen human being 𝔽 𝑍 =? 𝔽 𝑍 | π‘Œ = "China" =? 𝔽 𝑍 | π‘Œ = "U.S." =?

slide-12
SLIDE 12

Conditional Expectation

Example: sample a human being uniformly at random 𝑍: height of the chosen human being π‘Œ: country of origin of the chosen human being 𝔽 𝑍 | π‘Œ = "China" =? 𝔽 𝑍 | π‘Œ = "U.S." =?

slide-13
SLIDE 13

Conditional Expectation

Example: sample a human being uniformly at random 𝑍: height of the chosen human being π‘Œ: country of origin of the chosen human being 𝔽 𝑍 | π‘Œ = "China" =? 𝔽 𝑍 | π‘Œ = "U.S." =?

𝔽 𝑍 | π‘Œ = 𝑦

slide-14
SLIDE 14

Conditional Expectation

Example: sample a human being uniformly at random 𝑍: height of the chosen human being π‘Œ: country of origin of the chosen human being 𝔽 𝑍 | π‘Œ = "China" =? 𝔽 𝑍 | π‘Œ = "U.S." =?

𝔽 𝑍 | π‘Œ = 𝑦 𝑔 𝑦 =

slide-15
SLIDE 15

Conditional Expectation

Example: sample a human being uniformly at random 𝑍: height of the chosen human being π‘Œ: country of origin of the chosen human being 𝔽 𝑍 | π‘Œ = "China" =? 𝔽 𝑍 | π‘Œ = "U.S." =?

𝑔 π‘Œ = 𝔽 𝑍 | π‘Œ

slide-16
SLIDE 16

Conditional Expectation

Example: sample a human being uniformly at random 𝑍: height of the chosen human being π‘Œ: country of origin of the chosen human being 𝔽 𝑍 | π‘Œ = "China" =? 𝔽 𝑍 | π‘Œ = "U.S." =?

𝑔 π‘Œ = 𝔽 𝑍 | π‘Œ

a random variable

slide-17
SLIDE 17

Conditional Expectation

Example: sample a human being uniformly at random 𝑍: height of the chosen human being π‘Œ: country of origin of the chosen human being 𝔽 𝑍 | π‘Œ = "China" =? 𝔽 𝑍 | π‘Œ = "U.S." =?

𝑔 π‘Œ = 𝔽 𝑍 | π‘Œ

a random variable

Example: throw a fair six-sided dice for π‘œ times π‘Œπ‘—: # of times 𝑗 appears in π‘œ throws

slide-18
SLIDE 18

Conditional Expectation

Example: sample a human being uniformly at random 𝑍: height of the chosen human being π‘Œ: country of origin of the chosen human being 𝔽 𝑍 | π‘Œ = "China" =? 𝔽 𝑍 | π‘Œ = "U.S." =?

𝑔 π‘Œ = 𝔽 𝑍 | π‘Œ

a random variable

Example: throw a fair six-sided dice for π‘œ times π‘Œπ‘—: # of times 𝑗 appears in π‘œ throws 𝔽 π‘Œ1 | π‘Œ2 = 𝑏 =

slide-19
SLIDE 19

Conditional Expectation

Example: sample a human being uniformly at random 𝑍: height of the chosen human being π‘Œ: country of origin of the chosen human being 𝔽 𝑍 | π‘Œ = "China" =? 𝔽 𝑍 | π‘Œ = "U.S." =?

𝑔 π‘Œ = 𝔽 𝑍 | π‘Œ

a random variable

Example: throw a fair six-sided dice for π‘œ times π‘Œπ‘—: # of times 𝑗 appears in π‘œ throws 𝔽 π‘Œ1 | π‘Œ2 = 𝑏 = (π‘œ βˆ’ 𝑏)/5

slide-20
SLIDE 20

Conditional Expectation

Example: sample a human being uniformly at random 𝑍: height of the chosen human being π‘Œ: country of origin of the chosen human being 𝔽 𝑍 | π‘Œ = "China" =? 𝔽 𝑍 | π‘Œ = "U.S." =?

𝑔 π‘Œ = 𝔽 𝑍 | π‘Œ

a random variable

Example: throw a fair six-sided dice for π‘œ times π‘Œπ‘—: # of times 𝑗 appears in π‘œ throws 𝔽 π‘Œ1 | π‘Œ2 = 𝑏 = (π‘œ βˆ’ 𝑏)/5 𝔽 π‘Œ1 | π‘Œ2 =

slide-21
SLIDE 21

Conditional Expectation

Example: sample a human being uniformly at random 𝑍: height of the chosen human being π‘Œ: country of origin of the chosen human being 𝔽 𝑍 | π‘Œ = "China" =? 𝔽 𝑍 | π‘Œ = "U.S." =?

𝑔 π‘Œ = 𝔽 𝑍 | π‘Œ

a random variable

Example: throw a fair six-sided dice for π‘œ times π‘Œπ‘—: # of times 𝑗 appears in π‘œ throws 𝔽 π‘Œ1 | π‘Œ2 = 𝑏 = (π‘œ βˆ’ 𝑏)/5 𝔽 π‘Œ1 | π‘Œ2 = (π‘œ βˆ’ π‘Œ2)/5

slide-22
SLIDE 22

Conditional Expectation

Example: sample a human being uniformly at random 𝑍: height of the chosen human being π‘Œ: country of origin of the chosen human being 𝔽 𝑍 | π‘Œ = "China" =? 𝔽 𝑍 | π‘Œ = "U.S." =?

𝑔 π‘Œ = 𝔽 𝑍 | π‘Œ

a random variable

Example: throw a fair six-sided dice for π‘œ times π‘Œπ‘—: # of times 𝑗 appears in π‘œ throws 𝔽 π‘Œ1 | π‘Œ2 = 𝑏 = (π‘œ βˆ’ 𝑏)/5 𝔽 π‘Œ1 | π‘Œ2 = (π‘œ βˆ’ π‘Œ2)/5 𝔽 π‘Œ1 | π‘Œ2 = 𝑏, π‘Œ3 = 𝑐 =

slide-23
SLIDE 23

Conditional Expectation

Example: sample a human being uniformly at random 𝑍: height of the chosen human being π‘Œ: country of origin of the chosen human being 𝔽 𝑍 | π‘Œ = "China" =? 𝔽 𝑍 | π‘Œ = "U.S." =?

𝑔 π‘Œ = 𝔽 𝑍 | π‘Œ

a random variable

Example: throw a fair six-sided dice for π‘œ times π‘Œπ‘—: # of times 𝑗 appears in π‘œ throws 𝔽 π‘Œ1 | π‘Œ2 = 𝑏 = (π‘œ βˆ’ 𝑏)/5 𝔽 π‘Œ1 | π‘Œ2 = (π‘œ βˆ’ π‘Œ2)/5 𝔽 π‘Œ1 | π‘Œ2 = 𝑏, π‘Œ3 = 𝑐 = (π‘œ βˆ’ 𝑏 βˆ’ 𝑐)/4

slide-24
SLIDE 24

Conditional Expectation

Example: sample a human being uniformly at random 𝑍: height of the chosen human being π‘Œ: country of origin of the chosen human being 𝔽 𝑍 | π‘Œ = "China" =? 𝔽 𝑍 | π‘Œ = "U.S." =?

𝑔 π‘Œ = 𝔽 𝑍 | π‘Œ

a random variable

Example: throw a fair six-sided dice for π‘œ times π‘Œπ‘—: # of times 𝑗 appears in π‘œ throws 𝔽 π‘Œ1 | π‘Œ2 = 𝑏 = (π‘œ βˆ’ 𝑏)/5 𝔽 π‘Œ1 | π‘Œ2 = (π‘œ βˆ’ π‘Œ2)/5 𝔽 π‘Œ1 | π‘Œ2 = 𝑏, π‘Œ3 = 𝑐 = (π‘œ βˆ’ 𝑏 βˆ’ 𝑐)/4 𝔽 π‘Œ1 | π‘Œ2, π‘Œ3 =

slide-25
SLIDE 25

Conditional Expectation

Example: sample a human being uniformly at random 𝑍: height of the chosen human being π‘Œ: country of origin of the chosen human being 𝔽 𝑍 | π‘Œ = "China" =? 𝔽 𝑍 | π‘Œ = "U.S." =?

𝑔 π‘Œ = 𝔽 𝑍 | π‘Œ

a random variable

Example: throw a fair six-sided dice for π‘œ times π‘Œπ‘—: # of times 𝑗 appears in π‘œ throws 𝔽 π‘Œ1 | π‘Œ2 = 𝑏 = (π‘œ βˆ’ 𝑏)/5 𝔽 π‘Œ1 | π‘Œ2 = (π‘œ βˆ’ π‘Œ2)/5 𝔽 π‘Œ1 | π‘Œ2 = 𝑏, π‘Œ3 = 𝑐 = (π‘œ βˆ’ 𝑏 βˆ’ 𝑐)/4 𝔽 π‘Œ1 | π‘Œ2, π‘Œ3 = (π‘œ βˆ’ π‘Œ2 βˆ’ π‘Œ3)/4

slide-26
SLIDE 26

Fundamental Facts about Conditional Expectation

𝑍: height of the chosen human being π‘Œ: country of origin of the chosen human being π‘Ž: gender of the chosen human being Example:

slide-27
SLIDE 27

Fundamental Facts about Conditional Expectation

𝑍: height of the chosen human being π‘Œ: country of origin of the chosen human being π‘Ž: gender of the chosen human being Example:

𝔽 𝑍 = 𝔽 𝔽 𝑍 | π‘Œ

slide-28
SLIDE 28

Fundamental Facts about Conditional Expectation

𝑍: height of the chosen human being π‘Œ: country of origin of the chosen human being π‘Ž: gender of the chosen human being Example:

𝔽 𝑍 = 𝔽 𝔽 𝑍 | π‘Œ

average height of all human beings = weighted average of the country-by-country average heights

slide-29
SLIDE 29

Fundamental Facts about Conditional Expectation

𝑍: height of the chosen human being π‘Œ: country of origin of the chosen human being π‘Ž: gender of the chosen human being Example:

𝔽 𝑍 = 𝔽 𝔽 𝑍 | π‘Œ

average height of all human beings = weighted average of the country-by-country average heights

𝔽 𝑍 | π‘Ž = 𝔽 𝔽 𝑍 | π‘Œ, π‘Ž | π‘Ž

slide-30
SLIDE 30

Fundamental Facts about Conditional Expectation

𝑍: height of the chosen human being π‘Œ: country of origin of the chosen human being π‘Ž: gender of the chosen human being Example:

𝔽 𝑍 = 𝔽 𝔽 𝑍 | π‘Œ

average height of all human beings = weighted average of the country-by-country average heights

𝔽 𝑍 | π‘Ž = 𝔽 𝔽 𝑍 | π‘Œ, π‘Ž | π‘Ž

average height of all male/female human beings = weighted average of the country-by-country average male/female heights

slide-31
SLIDE 31

Fundamental Facts about Conditional Expectation

𝑍: height of the chosen human being π‘Œ: country of origin of the chosen human being π‘Ž: gender of the chosen human being Example:

𝔽 𝑍 = 𝔽 𝔽 𝑍 | π‘Œ

average height of all human beings = weighted average of the country-by-country average heights

𝔽 𝑍 | π‘Ž = 𝔽 𝔽 𝑍 | π‘Œ, π‘Ž | π‘Ž

average height of all male/female human beings = weighted average of the country-by-country average male/female heights

𝔽 𝔽 𝑔 π‘Œ 𝑕 π‘Œ, 𝑍 | π‘Œ = 𝔽 𝑔 π‘Œ 𝔽 𝑕 π‘Œ, 𝑍 | π‘Œ

slide-32
SLIDE 32

Fundamental Facts about Conditional Expectation

𝑍: height of the chosen human being π‘Œ: country of origin of the chosen human being π‘Ž: gender of the chosen human being Example:

𝔽 𝑍 = 𝔽 𝔽 𝑍 | π‘Œ

average height of all human beings = weighted average of the country-by-country average heights

𝔽 𝑍 | π‘Ž = 𝔽 𝔽 𝑍 | π‘Œ, π‘Ž | π‘Ž

average height of all male/female human beings = weighted average of the country-by-country average male/female heights

𝔽 𝔽 𝑔 π‘Œ 𝑕 π‘Œ, 𝑍 | π‘Œ = 𝔽 𝑔 π‘Œ 𝔽 𝑕 π‘Œ, 𝑍 | π‘Œ

  • nce π‘Œ is fixed to some 𝑦,

𝔽 𝑔 π‘Œ 𝑕 π‘Œ, 𝑍 | π‘Œ = 𝑦 = 𝑔(𝑦)𝔽 𝑕 π‘Œ, 𝑍 | π‘Œ = 𝑦

slide-33
SLIDE 33

Fundamental Facts about Conditional Expectation (Cont.)

𝔽 𝑍 = 𝔽 𝔽 𝑍 | π‘Œ 𝔽 𝑍 | π‘Ž = 𝔽 𝔽 𝑍 | π‘Œ, π‘Ž | π‘Ž 𝔽 𝔽 𝑔 π‘Œ 𝑕 π‘Œ, 𝑍 | π‘Œ = 𝔽 𝑔 π‘Œ 𝔽 𝑕 π‘Œ, 𝑍 | π‘Œ

slide-34
SLIDE 34

Fundamental Facts about Conditional Expectation (Cont.)

𝔽 𝑍 = 𝔽 𝔽 𝑍 | π‘Œ 𝔽 𝑍 | π‘Ž = 𝔽 𝔽 𝑍 | π‘Œ, π‘Ž | π‘Ž 𝔽 𝔽 𝑔 π‘Œ 𝑕 π‘Œ, 𝑍 | π‘Œ = 𝔽 𝑔 π‘Œ 𝔽 𝑕 π‘Œ, 𝑍 | π‘Œ

Generalization to the multivariate case:

𝔽 𝑍 = 𝔽 𝔽 𝑍 | Τ¦ π‘Œ 𝔽 𝑍 | Τ¦ π‘Ž = 𝔽 𝔽 𝑍 | Τ¦ π‘Œ, Τ¦ π‘Ž | Τ¦ π‘Ž 𝔽 𝔽 𝑔 Τ¦ π‘Œ 𝑕 Τ¦ π‘Œ, 𝑍 | Τ¦ π‘Œ = 𝔽 𝑔 Τ¦ π‘Œ 𝔽 𝑕 Τ¦ π‘Œ, 𝑍 | Τ¦ π‘Œ

slide-35
SLIDE 35

Martingales

  • riginally refers to a betting strategy:

β€œdouble your bet after every loss”

slide-36
SLIDE 36

Martingales

  • riginally refers to a betting strategy:

β€œdouble your bet after every loss” when you get a win after π‘œ losses: 2π‘œ βˆ’ σ𝑗=0

π‘œβˆ’1 2𝑗 = 1

slide-37
SLIDE 37

Martingales

  • riginally refers to a betting strategy:

β€œdouble your bet after every loss” when you get a win after π‘œ losses: 2π‘œ βˆ’ σ𝑗=0

π‘œβˆ’1 2𝑗 = 1

consider a fair game, with any betting strategy

slide-38
SLIDE 38

Martingales

  • riginally refers to a betting strategy:

β€œdouble your bet after every loss” when you get a win after π‘œ losses: 2π‘œ βˆ’ σ𝑗=0

π‘œβˆ’1 2𝑗 = 1

consider a fair game, with any betting strategy let π‘Œπ‘— be our wealth after 𝑗 rounds

slide-39
SLIDE 39

Martingales

  • riginally refers to a betting strategy:

β€œdouble your bet after every loss” when you get a win after π‘œ losses: 2π‘œ βˆ’ σ𝑗=0

π‘œβˆ’1 2𝑗 = 1

consider a fair game, with any betting strategy let π‘Œπ‘— be our wealth after 𝑗 rounds 𝔽 π‘Œπ‘—+1 | π‘Œ0, π‘Œ1, β‹― , π‘Œπ‘— =

slide-40
SLIDE 40

Martingales

  • riginally refers to a betting strategy:

β€œdouble your bet after every loss” when you get a win after π‘œ losses: 2π‘œ βˆ’ σ𝑗=0

π‘œβˆ’1 2𝑗 = 1

consider a fair game, with any betting strategy let π‘Œπ‘— be our wealth after 𝑗 rounds 𝔽 π‘Œπ‘—+1 | π‘Œ0, π‘Œ1, β‹― , π‘Œπ‘— = π‘Œπ‘—

slide-41
SLIDE 41

Martingales

  • riginally refers to a betting strategy:

β€œdouble your bet after every loss” when you get a win after π‘œ losses: 2π‘œ βˆ’ σ𝑗=0

π‘œβˆ’1 2𝑗 = 1

consider a fair game, with any betting strategy let π‘Œπ‘— be our wealth after 𝑗 rounds 𝔽 π‘Œπ‘—+1 | π‘Œ0, π‘Œ1, β‹― , π‘Œπ‘— = π‘Œπ‘— since the game is fair, conditioned on past history, we expect no change to current value after one round

slide-42
SLIDE 42

Martingales

slide-43
SLIDE 43

Example: Coin Flipping

toss a fair coin for many times measure the differences between # of heads and # of tails

slide-44
SLIDE 44

Example: Coin Flipping

toss a fair coin for many times measure the differences between # of heads and # of tails

slide-45
SLIDE 45

Example: Coin Flipping

toss a fair coin for many times measure the differences between # of heads and # of tails

slide-46
SLIDE 46

Example: Coin Flipping

toss a fair coin for many times measure the differences between # of heads and # of tails

slide-47
SLIDE 47

Example: Coin Flipping

toss a fair coin for many times measure the differences between # of heads and # of tails

slide-48
SLIDE 48

Example: Coin Flipping

toss a fair coin for many times measure the differences between # of heads and # of tails

slide-49
SLIDE 49

Example: Coin Flipping

toss a fair coin for many times measure the differences between # of heads and # of tails

slide-50
SLIDE 50

Example: Coin Flipping

toss a fair coin for many times measure the differences between # of heads and # of tails

slide-51
SLIDE 51

Example: Random Walk

a dot starting from the origin in each step, move equiprobably to one of four neighbors

slide-52
SLIDE 52

Example: Random Walk

a dot starting from the origin in each step, move equiprobably to one of four neighbors after 𝑗 steps, use π‘Œπ‘— to denote # of hops to origin (Manhattan distance)

slide-53
SLIDE 53

Example: Random Walk

a dot starting from the origin in each step, move equiprobably to one of four neighbors after 𝑗 steps, use π‘Œπ‘— to denote # of hops to origin (Manhattan distance)

slide-54
SLIDE 54

Example: Random Walk

a dot starting from the origin in each step, move equiprobably to one of four neighbors after 𝑗 steps, use π‘Œπ‘— to denote # of hops to origin (Manhattan distance)

slide-55
SLIDE 55

How far the dot is away from the origin after π‘œ steps?

Example: Random Walk

a dot starting from the origin in each step, move equiprobably to one of four neighbors after 𝑗 steps, use π‘Œπ‘— to denote # of hops to origin (Manhattan distance)

slide-56
SLIDE 56

Azuma’s Inequality

slide-57
SLIDE 57

Azuma’s Inequality

π‘Œ0, π‘Œ1, β‹― are not necessarily independent

slide-58
SLIDE 58

Azuma’s Inequality in Action

After 𝑗 steps, use π‘Œπ‘— to denote # of hops to origin (Manhattan distance) How large is π‘Œπ‘œ?

slide-59
SLIDE 59

Azuma’s Inequality in Action

After 𝑗 steps, use π‘Œπ‘— to denote # of hops to origin (Manhattan distance) How large is π‘Œπ‘œ?

slide-60
SLIDE 60

Azuma’s Inequality in Action

After 𝑗 steps, use π‘Œπ‘— to denote # of hops to origin (Manhattan distance) How large is π‘Œπ‘œ? We know π‘Œ0 = 0, and π‘Œπ‘™ βˆ’ π‘Œπ‘™βˆ’1 ≀ 1

slide-61
SLIDE 61

Azuma’s Inequality in Action

After 𝑗 steps, use π‘Œπ‘— to denote # of hops to origin (Manhattan distance) How large is π‘Œπ‘œ? We know π‘Œ0 = 0, and π‘Œπ‘™ βˆ’ π‘Œπ‘™βˆ’1 ≀ 1

slide-62
SLIDE 62

Azuma’s Inequality in Action

After 𝑗 steps, use π‘Œπ‘— to denote # of hops to origin (Manhattan distance) How large is π‘Œπ‘œ? We know π‘Œ0 = 0, and π‘Œπ‘™ βˆ’ π‘Œπ‘™βˆ’1 ≀ 1 Within Ο( π‘œ log π‘œ) w.h.p.

slide-63
SLIDE 63

Azuma’s Inequality

slide-64
SLIDE 64

Azuma’s Inequality

For a sequence of r.v., if in each step: * on average make no change to current value (martingale) * no big jump (bounded difference)

slide-65
SLIDE 65

Azuma’s Inequality

For a sequence of r.v., if in each step: * on average make no change to current value (martingale) * no big jump (bounded difference) Then final value does not deviate a lot from the initial value.

slide-66
SLIDE 66

Proving Azuma’s Inequality

slide-67
SLIDE 67

Proving Azuma’s Inequality

Use similar strategy as in proving Chernoff bounds:

slide-68
SLIDE 68

Proving Azuma’s Inequality

Use similar strategy as in proving Chernoff bounds: (a) Apply generalized Markov’s inequality to MGF

slide-69
SLIDE 69

Proving Azuma’s Inequality

Use similar strategy as in proving Chernoff bounds: (a) Apply generalized Markov’s inequality to MGF (b)* Bound the value of MGF (use Hoeffding’s lemma)

slide-70
SLIDE 70

Proving Azuma’s Inequality

Use similar strategy as in proving Chernoff bounds: (a) Apply generalized Markov’s inequality to MGF (b)* Bound the value of MGF (use Hoeffding’s lemma) (c) Optimize the value of MGF

slide-71
SLIDE 71

Proving Azuma’s Inequality

slide-72
SLIDE 72

Proving Azuma’s Inequality

slide-73
SLIDE 73

Proving Azuma’s Inequality

slide-74
SLIDE 74

Proving Azuma’s Inequality

slide-75
SLIDE 75
slide-76
SLIDE 76

for πœ‡ > 0

slide-77
SLIDE 77

for πœ‡ > 0

slide-78
SLIDE 78

for πœ‡ > 0

slide-79
SLIDE 79

for πœ‡ > 0 𝔽 𝑍 = 𝔽 𝔽 𝑍 | π‘Œ

slide-80
SLIDE 80

for πœ‡ > 0 𝔽 𝔽 𝑔 π‘Œ 𝑕 π‘Œ, 𝑍 |π‘Œ = 𝔽 𝑔 π‘Œ 𝔽 𝑕 π‘Œ, 𝑍 |π‘Œ

slide-81
SLIDE 81

for πœ‡ > 0

slide-82
SLIDE 82

for πœ‡ > 0

slide-83
SLIDE 83

for πœ‡ > 0

slide-84
SLIDE 84

for πœ‡ > 0

slide-85
SLIDE 85

for πœ‡ > 0

slide-86
SLIDE 86

for πœ‡ > 0

slide-87
SLIDE 87

for πœ‡ > 0

slide-88
SLIDE 88

for πœ‡ > 0

slide-89
SLIDE 89

for πœ‡ > 0 minimized when πœ‡ =

𝑒 σ𝑙=1

π‘œ

𝑑𝑙

2

slide-90
SLIDE 90

Proving Azuma’s Inequality

slide-91
SLIDE 91

Proving Azuma’s Inequality

???

slide-92
SLIDE 92

Proving Azuma’s Inequality

???

slide-93
SLIDE 93

Generalized Martingales

slide-94
SLIDE 94

Generalized Martingales

betting on a fair game π‘Œπ‘—: gain/loss of the ith bet 𝑍

𝑗: wealth after the ith bet

slide-95
SLIDE 95

Generalized Martingales

betting on a fair game π‘Œπ‘—: gain/loss of the ith bet 𝑍

𝑗: wealth after the ith bet ← martingale (since game is fair)

slide-96
SLIDE 96

Generalized Azuma’s Inequality

slide-97
SLIDE 97

martingale π‘Œ0, π‘Œ1, β‹― , π‘Œπ‘œ

𝔽 π‘Œπ‘— π‘Œ0, π‘Œ1, β‹― , π‘Œπ‘—βˆ’1) = π‘Œπ‘—βˆ’1

martingale 𝑍

0, 𝑍 1, β‹― w.r.t. π‘Œ0, π‘Œ1, β‹―

𝑍

𝑗 = 𝑔(π‘Œ0, π‘Œ1, β‹― , π‘Œπ‘—)

𝔽 𝑍

𝑗

π‘Œ0, π‘Œ1, β‹― , π‘Œπ‘—βˆ’1) = 𝑍

π‘—βˆ’1

Azuma’s Inequality

martingale π‘Œ0, π‘Œ1, β‹― with π‘Œπ‘™ βˆ’ π‘Œπ‘™βˆ’1 ≀ 𝑑𝑙, then β„™ π‘Œπ‘œ βˆ’ π‘Œ0 β‰₯ 𝑒 ≀ β‹―

Generalized Azuma’s Inequality

martingale 𝑍

0, 𝑍 1, β‹― w.r.t. π‘Œ0, π‘Œ1, β‹―

with 𝑍

𝑙 βˆ’ 𝑍 π‘™βˆ’1 ≀ 𝑑𝑙,

then β„™ 𝑍

π‘œ βˆ’ 𝑍 0 β‰₯ 𝑒 ≀ β‹―

generalization generalization

slide-98
SLIDE 98

Doob Sequence

slide-99
SLIDE 99

Doob Sequence

𝑔( ) , , ,

slide-100
SLIDE 100

Doob Sequence

𝑔( ) , ,

𝔽 𝑔 no information

,

average over

slide-101
SLIDE 101

Doob Sequence

𝑔( ) , ,

𝔽 𝑔 no information 𝔽 𝑔|π‘Œ1

,

average over

1

randomized by

slide-102
SLIDE 102

Doob Sequence

𝑔( ) , ,

𝔽 𝑔 no information 𝔽 𝑔|π‘Œ1

,

average over

1

randomized by 𝔽 𝑔|π‘Œ2

slide-103
SLIDE 103

Doob Sequence

𝑔( ) , ,

𝔽 𝑔 no information

,

average over

1

randomized by 𝔽 𝑔|π‘Œ3 𝔽 𝑔|π‘Œ1 𝔽 𝑔|π‘Œ2

slide-104
SLIDE 104

Doob Sequence

𝑔( ) , ,

𝔽 𝑔 no information

,

1

randomized by

1

𝔽 𝑔|π‘Œ4 = 𝑔(π‘Œ4) full information 𝔽 𝑔|π‘Œ1 𝔽 𝑔|π‘Œ2 𝔽 𝑔|π‘Œ3

slide-105
SLIDE 105

Doob Martingale

slide-106
SLIDE 106

Doob Martingale

slide-107
SLIDE 107

Doob Martingale

𝔽 𝑍 | Τ¦ π‘Ž = 𝔽 𝔽 𝑍 | Τ¦ π‘Œ, Τ¦ π‘Ž | Τ¦ π‘Ž

slide-108
SLIDE 108

Doob Martingale

slide-109
SLIDE 109

π»π‘œ,π‘ž

slide-110
SLIDE 110

π»π‘œ,π‘ž Graph parameter: 𝑔(𝐻)

Example: components number, chromatic number, diameter

slide-111
SLIDE 111

π»π‘œ,π‘ž Graph parameter: 𝑔(𝐻)

Example: components number, chromatic number, diameter numbering all vertex pairs: 1,2,3, β‹― , π‘œ 2

slide-112
SLIDE 112

π»π‘œ,π‘ž Graph parameter: 𝑔(𝐻)

Example: components number, chromatic number, diameter numbering all vertex pairs: 1,2,3, β‹― , π‘œ 2 Define i.r.v. 𝐽

π‘˜ = α‰Š1 edge π‘˜ ∈ 𝐻

0 edge π‘˜ βˆ‰ 𝐻

slide-113
SLIDE 113

π»π‘œ,π‘ž Graph parameter: 𝑔(𝐻)

Example: components number, chromatic number, diameter numbering all vertex pairs: 1,2,3, β‹― , π‘œ 2 Define i.r.v. 𝐽

π‘˜ = α‰Š1 edge π‘˜ ∈ 𝐻

0 edge π‘˜ βˆ‰ 𝐻 𝑍

𝑗 = 𝔽 𝑔 𝐻 | 𝐽1, β‹― , 𝐽𝑗

slide-114
SLIDE 114

π»π‘œ,π‘ž Graph parameter: 𝑔(𝐻)

Example: components number, chromatic number, diameter numbering all vertex pairs: 1,2,3, β‹― , π‘œ 2 Define i.r.v. 𝐽

π‘˜ = α‰Š1 edge π‘˜ ∈ 𝐻

0 edge π‘˜ βˆ‰ 𝐻 𝑍

𝑗 = 𝔽 𝑔 𝐻 | 𝐽1, β‹― , 𝐽𝑗

𝑍

0, 𝑍 1, β‹― , 𝑍 π‘œ 2

is a Doob sequence, called edge exposure martingale In particular, 𝑍

0 = 𝔽(𝑔(𝐻)), and 𝑍 π‘œ 2

= 𝑔(𝐻)

slide-115
SLIDE 115

π»π‘œ,π‘ž Graph parameter: 𝑔(𝐻)

Example: components number, chromatic number, diameter numbering all vertices: 1,2,3, β‹― , π‘œ

slide-116
SLIDE 116

π»π‘œ,π‘ž Graph parameter: 𝑔(𝐻)

Example: components number, chromatic number, diameter numbering all vertices: 1,2,3, β‹― , π‘œ π‘Œπ‘—: subgraph of 𝐻 induced by the first 𝑗 vertices

slide-117
SLIDE 117

π»π‘œ,π‘ž Graph parameter: 𝑔(𝐻)

Example: components number, chromatic number, diameter numbering all vertices: 1,2,3, β‹― , π‘œ π‘Œπ‘—: subgraph of 𝐻 induced by the first 𝑗 vertices 𝑍

𝑗 = 𝔽 𝑔 𝐻 | π‘Œ1, β‹― , π‘Œπ‘—

slide-118
SLIDE 118

π»π‘œ,π‘ž Graph parameter: 𝑔(𝐻)

Example: components number, chromatic number, diameter numbering all vertices: 1,2,3, β‹― , π‘œ π‘Œπ‘—: subgraph of 𝐻 induced by the first 𝑗 vertices 𝑍

𝑗 = 𝔽 𝑔 𝐻 | π‘Œ1, β‹― , π‘Œπ‘—

𝑍

0, 𝑍 1, β‹― , 𝑍 π‘œ is a Doob sequence, called vertex exposure martingale

In particular, 𝑍

0 = 𝔽(𝑔(𝐻)), and 𝑍 π‘œ = 𝑔(𝐻)

slide-119
SLIDE 119

π»π‘œ,π‘ž

numbering all vertices: 1,2,3, β‹― , π‘œ π‘Œπ‘—: subgraph of 𝐻 induced by the first 𝑗 vertices 𝑍

𝑗 = 𝔽 πœ“ 𝐻 | π‘Œ1, β‹― , π‘Œπ‘—

𝑍

0, 𝑍 1, β‹― , 𝑍 π‘œ is a Doob sequence (vertex exposure martingale)

In particular, 𝑍

0 = 𝔽(πœ“(𝐻)), and 𝑍 π‘œ = πœ“(𝐻)

chromatic number πœ“(𝐻) is the smallest number of colors to properly color 𝐻

slide-120
SLIDE 120

Generalized Azuma’s Inequality in Action

Concentration of Chromatic Number

chromatic number πœ“(𝐻) π‘Œπ‘—: subgraph of 𝐻 induced by the first 𝑗 vertices 𝑍

𝑗 = 𝔽 πœ“ 𝐻 | π‘Œ1, β‹― , π‘Œπ‘—

𝑍

0, 𝑍 1, β‹― , 𝑍 π‘œ a Doob martingale: 𝑍 0 = 𝔽(πœ“(𝐻)), and 𝑍 π‘œ = πœ“(𝐻)

slide-121
SLIDE 121

Generalized Azuma’s Inequality in Action

Concentration of Chromatic Number

chromatic number πœ“(𝐻) π‘Œπ‘—: subgraph of 𝐻 induced by the first 𝑗 vertices 𝑍

𝑗 = 𝔽 πœ“ 𝐻 | π‘Œ1, β‹― , π‘Œπ‘—

𝑍

0, 𝑍 1, β‹― , 𝑍 π‘œ a Doob martingale: 𝑍 0 = 𝔽(πœ“(𝐻)), and 𝑍 π‘œ = πœ“(𝐻)

slide-122
SLIDE 122

Generalized Azuma’s Inequality in Action

Concentration of Chromatic Number

chromatic number πœ“(𝐻) π‘Œπ‘—: subgraph of 𝐻 induced by the first 𝑗 vertices 𝑍

𝑗 = 𝔽 πœ“ 𝐻 | π‘Œ1, β‹― , π‘Œπ‘—

𝑍

0, 𝑍 1, β‹― , 𝑍 π‘œ a Doob martingale: 𝑍 0 = 𝔽(πœ“(𝐻)), and 𝑍 π‘œ = πœ“(𝐻)

A new vertex can always be given a new color!

slide-123
SLIDE 123

Generalized Azuma’s Inequality in Action

Concentration of Chromatic Number

chromatic number πœ“(𝐻) π‘Œπ‘—: subgraph of 𝐻 induced by the first 𝑗 vertices 𝑍

𝑗 = 𝔽 πœ“ 𝐻 | π‘Œ1, β‹― , π‘Œπ‘—

𝑍

0, 𝑍 1, β‹― , 𝑍 π‘œ a Doob martingale: 𝑍 0 = 𝔽(πœ“(𝐻)), and 𝑍 π‘œ = πœ“(𝐻)

A new vertex can always be given a new color! 𝑍

𝑗 βˆ’ 𝑍 π‘—βˆ’1 ≀ 1

slide-124
SLIDE 124

Generalized Azuma’s Inequality in Action

Concentration of Chromatic Number

chromatic number πœ“(𝐻) π‘Œπ‘—: subgraph of 𝐻 induced by the first 𝑗 vertices 𝑍

𝑗 = 𝔽 πœ“ 𝐻 | π‘Œ1, β‹― , π‘Œπ‘—

𝑍

0, 𝑍 1, β‹― , 𝑍 π‘œ a Doob martingale: 𝑍 0 = 𝔽(πœ“(𝐻)), and 𝑍 π‘œ = πœ“(𝐻)

A new vertex can always be given a new color! 𝑍

𝑗 βˆ’ 𝑍 π‘—βˆ’1 ≀ 1

slide-125
SLIDE 125

Tight Concentration of Chromatic Number

slide-126
SLIDE 126

martingale π‘Œ0, π‘Œ1, β‹― , π‘Œπ‘œ

𝔽 π‘Œπ‘— π‘Œ0, π‘Œ1, β‹― , π‘Œπ‘—βˆ’1) = π‘Œπ‘—βˆ’1

martingale 𝑍

0, 𝑍 1, β‹― w.r.t. π‘Œ0, π‘Œ1, β‹―

𝑍

𝑗 = 𝑔(π‘Œ0, π‘Œ1, β‹― , π‘Œπ‘—)

𝔽 𝑍

𝑗

π‘Œ0, π‘Œ1, β‹― , π‘Œπ‘—βˆ’1) = 𝑍

π‘—βˆ’1

Azuma’s Inequality

martingale π‘Œ0, π‘Œ1, β‹― with π‘Œπ‘™ βˆ’ π‘Œπ‘™βˆ’1 ≀ 𝑑𝑙, then β„™ π‘Œπ‘œ βˆ’ π‘Œ0 β‰₯ 𝑒 ≀ β‹―

Generalized Azuma’s Inequality

martingale 𝑍

0, 𝑍 1, β‹― w.r.t. π‘Œ0, π‘Œ1, β‹―

with 𝑍

𝑙 βˆ’ 𝑍 π‘™βˆ’1 ≀ 𝑑𝑙,

then β„™ 𝑍

π‘œ βˆ’ 𝑍 0 β‰₯ 𝑒 ≀ β‹―

generalization generalization

slide-127
SLIDE 127

martingale π‘Œ0, π‘Œ1, β‹― , π‘Œπ‘œ

𝔽 π‘Œπ‘— π‘Œ0, π‘Œ1, β‹― , π‘Œπ‘—βˆ’1) = π‘Œπ‘—βˆ’1

martingale 𝑍

0, 𝑍 1, β‹― w.r.t. π‘Œ0, π‘Œ1, β‹―

𝑍

𝑗 = 𝑔(π‘Œ0, π‘Œ1, β‹― , π‘Œπ‘—)

𝔽 𝑍

𝑗

π‘Œ0, π‘Œ1, β‹― , π‘Œπ‘—βˆ’1) = 𝑍

π‘—βˆ’1

Doob martingale 𝑍

0, 𝑍 1, β‹―

𝑍

𝑗 = 𝔽 𝑔(π‘Œ0, π‘Œ1, β‹― , π‘Œπ‘œ) π‘Œ0, π‘Œ1, β‹― , π‘Œπ‘—βˆ’1)

Azuma’s Inequality

martingale π‘Œ0, π‘Œ1, β‹― with π‘Œπ‘™ βˆ’ π‘Œπ‘™βˆ’1 ≀ 𝑑𝑙, then β„™ π‘Œπ‘œ βˆ’ π‘Œ0 β‰₯ 𝑒 ≀ β‹―

Generalized Azuma’s Inequality

martingale 𝑍

0, 𝑍 1, β‹― w.r.t. π‘Œ0, π‘Œ1, β‹―

with 𝑍

𝑙 βˆ’ 𝑍 π‘™βˆ’1 ≀ 𝑑𝑙,

then β„™ 𝑍

π‘œ βˆ’ 𝑍 0 β‰₯ 𝑒 ≀ β‹―

generalization special case generalization

slide-128
SLIDE 128

martingale π‘Œ0, π‘Œ1, β‹― , π‘Œπ‘œ

𝔽 π‘Œπ‘— π‘Œ0, π‘Œ1, β‹― , π‘Œπ‘—βˆ’1) = π‘Œπ‘—βˆ’1

martingale 𝑍

0, 𝑍 1, β‹― w.r.t. π‘Œ0, π‘Œ1, β‹―

𝑍

𝑗 = 𝑔(π‘Œ0, π‘Œ1, β‹― , π‘Œπ‘—)

𝔽 𝑍

𝑗

π‘Œ0, π‘Œ1, β‹― , π‘Œπ‘—βˆ’1) = 𝑍

π‘—βˆ’1

Doob martingale 𝑍

0, 𝑍 1, β‹―

𝑍

𝑗 = 𝔽 𝑔(π‘Œ0, π‘Œ1, β‹― , π‘Œπ‘œ) π‘Œ0, π‘Œ1, β‹― , π‘Œπ‘—βˆ’1)

Azuma’s Inequality

martingale π‘Œ0, π‘Œ1, β‹― with π‘Œπ‘™ βˆ’ π‘Œπ‘™βˆ’1 ≀ 𝑑𝑙, then β„™ π‘Œπ‘œ βˆ’ π‘Œ0 β‰₯ 𝑒 ≀ β‹―

vertex exposure martingale Generalized Azuma’s Inequality

martingale 𝑍

0, 𝑍 1, β‹― w.r.t. π‘Œ0, π‘Œ1, β‹―

with 𝑍

𝑙 βˆ’ 𝑍 π‘™βˆ’1 ≀ 𝑑𝑙,

then β„™ 𝑍

π‘œ βˆ’ 𝑍 0 β‰₯ 𝑒 ≀ β‹―

generalization special case applied in random graphs generalization

slide-129
SLIDE 129

martingale π‘Œ0, π‘Œ1, β‹― , π‘Œπ‘œ

𝔽 π‘Œπ‘— π‘Œ0, π‘Œ1, β‹― , π‘Œπ‘—βˆ’1) = π‘Œπ‘—βˆ’1

martingale 𝑍

0, 𝑍 1, β‹― w.r.t. π‘Œ0, π‘Œ1, β‹―

𝑍

𝑗 = 𝑔(π‘Œ0, π‘Œ1, β‹― , π‘Œπ‘—)

𝔽 𝑍

𝑗

π‘Œ0, π‘Œ1, β‹― , π‘Œπ‘—βˆ’1) = 𝑍

π‘—βˆ’1

Doob martingale 𝑍

0, 𝑍 1, β‹―

𝑍

𝑗 = 𝔽 𝑔(π‘Œ0, π‘Œ1, β‹― , π‘Œπ‘œ) π‘Œ0, π‘Œ1, β‹― , π‘Œπ‘—βˆ’1)

Azuma’s Inequality

martingale π‘Œ0, π‘Œ1, β‹― with π‘Œπ‘™ βˆ’ π‘Œπ‘™βˆ’1 ≀ 𝑑𝑙, then β„™ π‘Œπ‘œ βˆ’ π‘Œ0 β‰₯ 𝑒 ≀ β‹―

vertex exposure martingale Generalized Azuma’s Inequality

martingale 𝑍

0, 𝑍 1, β‹― w.r.t. π‘Œ0, π‘Œ1, β‹―

with 𝑍

𝑙 βˆ’ 𝑍 π‘™βˆ’1 ≀ 𝑑𝑙,

then β„™ 𝑍

π‘œ βˆ’ 𝑍 0 β‰₯ 𝑒 ≀ β‹―

generalization special case applied in random graphs generalization

Sample Application: Tight Concentration of Chromatic number

slide-130
SLIDE 130

Doob Martingale + Generalized Azuma’s Inequality

  • For a function of (potentially dependent) r.v.:

𝑔(π‘Œ1, π‘Œ2, β‹― , π‘Œπ‘œ)

  • Define corresponding Doob martingale:

𝑍

𝑗 = 𝔽 𝑔 π‘Œ1, β‹― , π‘Œπ‘œ | π‘Œ1, β‹― , π‘Œπ‘—

In particular, 𝑍

0 = 𝔽 𝑔(π‘Œ1, β‹― , π‘Œπ‘œ) and 𝑍 π‘œ = 𝑔(π‘Œ1, β‹― , π‘Œπ‘œ)

  • As long as the differences 𝑍

𝑗 βˆ’ 𝑍 π‘—βˆ’1 are bounded

  • Generalized Azuma’s inequality implies 𝑍

π‘œ βˆ’ 𝑍 0 is bounded

𝑔(π‘Œ1, β‹― , π‘Œπ‘œ) is tightly concentration to its expectation

slide-131
SLIDE 131

The Method of Averaged Bounded Differences

slide-132
SLIDE 132

The Method of Averaged Bounded Differences

𝑍

𝑗

𝑍

π‘—βˆ’1

𝑍

π‘œ

𝑍 Doob Martingale

slide-133
SLIDE 133

The Method of Averaged Bounded Differences

𝑍

𝑗

𝑍

π‘—βˆ’1

𝑍

π‘œ

𝑍 Doob Martingale Generalized Azuma’s Inequality

+

slide-134
SLIDE 134

The Method of Averaged Bounded Differences

slide-135
SLIDE 135

The Method of Averaged Bounded Differences

May be hard to check!

slide-136
SLIDE 136
slide-137
SLIDE 137

Lipschitz Condition

slide-138
SLIDE 138

Average-case: Worst-case:

slide-139
SLIDE 139

The Method of Bounded Differences

slide-140
SLIDE 140

The Method of Bounded Differences

slide-141
SLIDE 141

The Method of Bounded Differences

Lipschitz condition + Independence bounded averaged differences

𝔽 𝑔 π‘Œ π‘Œ1, β‹― , π‘Œπ‘—) βˆ’ 𝔽 𝑔 π‘Œ π‘Œ1, β‹― , π‘Œπ‘—βˆ’1) ≀ 𝑑𝑗

slide-142
SLIDE 142

martingale π‘Œ0, π‘Œ1, β‹― , π‘Œπ‘œ

𝔽 π‘Œπ‘— π‘Œ0, π‘Œ1, β‹― , π‘Œπ‘—βˆ’1) = π‘Œπ‘—βˆ’1

martingale 𝑍

0, 𝑍 1, β‹― w.r.t. π‘Œ0, π‘Œ1, β‹― 𝑍

𝑗 = 𝑔(π‘Œ0, π‘Œ1, β‹― , π‘Œπ‘—)

𝔽 𝑍

𝑗

π‘Œ0, π‘Œ1, β‹― , π‘Œπ‘—βˆ’1) = 𝑍

π‘—βˆ’1

Doob martingale 𝑍

0, 𝑍 1, β‹― 𝑍

𝑗 = 𝔽 𝑔(π‘Œ0, π‘Œ1, β‹― , π‘Œπ‘œ) π‘Œ0, π‘Œ1, β‹― , π‘Œπ‘—βˆ’1)

Azuma’s Inequality

martingale π‘Œ0, π‘Œ1, β‹― with π‘Œπ‘™ βˆ’ π‘Œπ‘™βˆ’1 ≀ 𝑑𝑙, then β„™ π‘Œπ‘œ βˆ’ π‘Œ0 β‰₯ 𝑒 ≀ β‹―

Generalized Azuma’s Inequality

martingale 𝑍

0, 𝑍 1, β‹― w.r.t. π‘Œ0, π‘Œ1, β‹―

with 𝑍

𝑙 βˆ’ 𝑍 π‘™βˆ’1 ≀ 𝑑𝑙,

then β„™ 𝑍

π‘œ βˆ’ 𝑍 0 β‰₯ 𝑒 ≀ β‹―

generalization special case generalization

slide-143
SLIDE 143

martingale π‘Œ0, π‘Œ1, β‹― , π‘Œπ‘œ

𝔽 π‘Œπ‘— π‘Œ0, π‘Œ1, β‹― , π‘Œπ‘—βˆ’1) = π‘Œπ‘—βˆ’1

martingale 𝑍

0, 𝑍 1, β‹― w.r.t. π‘Œ0, π‘Œ1, β‹― 𝑍

𝑗 = 𝑔(π‘Œ0, π‘Œ1, β‹― , π‘Œπ‘—)

𝔽 𝑍

𝑗

π‘Œ0, π‘Œ1, β‹― , π‘Œπ‘—βˆ’1) = 𝑍

π‘—βˆ’1

Doob martingale 𝑍

0, 𝑍 1, β‹― 𝑍

𝑗 = 𝔽 𝑔(π‘Œ0, π‘Œ1, β‹― , π‘Œπ‘œ) π‘Œ0, π‘Œ1, β‹― , π‘Œπ‘—βˆ’1)

Azuma’s Inequality

martingale π‘Œ0, π‘Œ1, β‹― with π‘Œπ‘™ βˆ’ π‘Œπ‘™βˆ’1 ≀ 𝑑𝑙, then β„™ π‘Œπ‘œ βˆ’ π‘Œ0 β‰₯ 𝑒 ≀ β‹―

Generalized Azuma’s Inequality

martingale 𝑍

0, 𝑍 1, β‹― w.r.t. π‘Œ0, π‘Œ1, β‹―

with 𝑍

𝑙 βˆ’ 𝑍 π‘™βˆ’1 ≀ 𝑑𝑙,

then β„™ 𝑍

π‘œ βˆ’ 𝑍 0 β‰₯ 𝑒 ≀ β‹―

generalization special case generalization

The Method of Averaged Bounded Differences

𝑔 π‘Œ satisfying 𝔽 𝑔 π‘Œ π‘Œ1, β‹― , π‘Œπ‘— βˆ’ 𝔽 𝑔 π‘Œ π‘Œ1, β‹― , π‘Œπ‘—βˆ’1 ≀ 𝑑𝑗, then β„™ 𝑔(π‘Œ) βˆ’ 𝔽(𝑔(π‘Œ)) β‰₯ 𝑒 ≀ β‹―

slide-144
SLIDE 144

martingale π‘Œ0, π‘Œ1, β‹― , π‘Œπ‘œ

𝔽 π‘Œπ‘— π‘Œ0, π‘Œ1, β‹― , π‘Œπ‘—βˆ’1) = π‘Œπ‘—βˆ’1

martingale 𝑍

0, 𝑍 1, β‹― w.r.t. π‘Œ0, π‘Œ1, β‹― 𝑍

𝑗 = 𝑔(π‘Œ0, π‘Œ1, β‹― , π‘Œπ‘—)

𝔽 𝑍

𝑗

π‘Œ0, π‘Œ1, β‹― , π‘Œπ‘—βˆ’1) = 𝑍

π‘—βˆ’1

Doob martingale 𝑍

0, 𝑍 1, β‹― 𝑍

𝑗 = 𝔽 𝑔(π‘Œ0, π‘Œ1, β‹― , π‘Œπ‘œ) π‘Œ0, π‘Œ1, β‹― , π‘Œπ‘—βˆ’1)

Azuma’s Inequality

martingale π‘Œ0, π‘Œ1, β‹― with π‘Œπ‘™ βˆ’ π‘Œπ‘™βˆ’1 ≀ 𝑑𝑙, then β„™ π‘Œπ‘œ βˆ’ π‘Œ0 β‰₯ 𝑒 ≀ β‹―

Generalized Azuma’s Inequality

martingale 𝑍

0, 𝑍 1, β‹― w.r.t. π‘Œ0, π‘Œ1, β‹―

with 𝑍

𝑙 βˆ’ 𝑍 π‘™βˆ’1 ≀ 𝑑𝑙,

then β„™ 𝑍

π‘œ βˆ’ 𝑍 0 β‰₯ 𝑒 ≀ β‹―

generalization special case generalization

The Method of Averaged Bounded Differences

𝑔 π‘Œ satisfying 𝔽 𝑔 π‘Œ π‘Œ1, β‹― , π‘Œπ‘— βˆ’ 𝔽 𝑔 π‘Œ π‘Œ1, β‹― , π‘Œπ‘—βˆ’1 ≀ 𝑑𝑗, then β„™ 𝑔(π‘Œ) βˆ’ 𝔽(𝑔(π‘Œ)) β‰₯ 𝑒 ≀ β‹―

The Method of Bounded Differences

π‘Œ = (π‘Œ1, β‹― , π‘Œπ‘œ) are independent r.v., 𝑔 π‘Œ satisfying the Lipschitz condition, then β„™ 𝑔(π‘Œ) βˆ’ 𝔽(𝑔(π‘Œ)) β‰₯ 𝑒 ≀ β‹― independence + Lipschitz condition

slide-145
SLIDE 145

The Method of Bounded Differences in Action:

Pattern Matching

  • a random string of length π‘œ
  • a pattern of length 𝑙
  • # of matched substrings?
slide-146
SLIDE 146

The Method of Bounded Differences in Action:

Pattern Matching

  • a random string of length π‘œ
  • a pattern of length 𝑙
  • # of matched substrings?

an alphabet Ξ£ with Ξ£ = 𝑛, a fixed pattern 𝜌 ∈ Σ𝑙

slide-147
SLIDE 147

The Method of Bounded Differences in Action:

Pattern Matching

  • a random string of length π‘œ
  • a pattern of length 𝑙
  • # of matched substrings?

an alphabet Ξ£ with Ξ£ = 𝑛, a fixed pattern 𝜌 ∈ Σ𝑙 independently and uniformly generate: π‘Œ1, π‘Œ2, β‹― , π‘Œπ‘œ ∈ Ξ£ let 𝑍 be number of substrings 𝜌 in π‘Œ1, π‘Œ2, β‹― , π‘Œπ‘œ

slide-148
SLIDE 148

The Method of Bounded Differences in Action:

Pattern Matching

  • a random string of length π‘œ
  • a pattern of length 𝑙
  • # of matched substrings?

an alphabet Ξ£ with Ξ£ = 𝑛, a fixed pattern 𝜌 ∈ Σ𝑙 independently and uniformly generate: π‘Œ1, π‘Œ2, β‹― , π‘Œπ‘œ ∈ Ξ£ let 𝑍 be number of substrings 𝜌 in π‘Œ1, π‘Œ2, β‹― , π‘Œπ‘œ 𝔽 𝑍 = π‘œ βˆ’ 𝑙 + 1 1 𝑛

𝑙

slide-149
SLIDE 149

The Method of Bounded Differences in Action:

Pattern Matching

  • a random string of length π‘œ
  • a pattern of length 𝑙
  • # of matched substrings?

an alphabet Ξ£ with Ξ£ = 𝑛, a fixed pattern 𝜌 ∈ Σ𝑙 independently and uniformly generate: π‘Œ1, π‘Œ2, β‹― , π‘Œπ‘œ ∈ Ξ£ let 𝑍 be number of substrings 𝜌 in π‘Œ1, π‘Œ2, β‹― , π‘Œπ‘œ 𝔽 𝑍 = π‘œ βˆ’ 𝑙 + 1 1 𝑛

𝑙

Deviation?

slide-150
SLIDE 150

The Method of Bounded Differences in Action:

Pattern Matching

  • a random string of length π‘œ
  • a pattern of length 𝑙
  • # of matched substrings?

an alphabet Ξ£ with Ξ£ = 𝑛, a fixed pattern 𝜌 ∈ Σ𝑙 independently and uniformly generate: π‘Œ1, π‘Œ2, β‹― , π‘Œπ‘œ ∈ Ξ£ let 𝑍 be number of substrings 𝜌 in π‘Œ1, π‘Œ2, β‹― , π‘Œπ‘œ

slide-151
SLIDE 151

The Method of Bounded Differences in Action:

Pattern Matching

  • a random string of length π‘œ
  • a pattern of length 𝑙
  • # of matched substrings?

an alphabet Ξ£ with Ξ£ = 𝑛, a fixed pattern 𝜌 ∈ Σ𝑙 independently and uniformly generate: π‘Œ1, π‘Œ2, β‹― , π‘Œπ‘œ ∈ Ξ£ let 𝑍 be number of substrings 𝜌 in π‘Œ1, π‘Œ2, β‹― , π‘Œπ‘œ 𝑍 = 𝑔(π‘Œ1, π‘Œ2, β‹― , π‘Œπ‘œ)

slide-152
SLIDE 152

The Method of Bounded Differences in Action:

Pattern Matching

  • a random string of length π‘œ
  • a pattern of length 𝑙
  • # of matched substrings?

an alphabet Ξ£ with Ξ£ = 𝑛, a fixed pattern 𝜌 ∈ Σ𝑙 independently and uniformly generate: π‘Œ1, π‘Œ2, β‹― , π‘Œπ‘œ ∈ Ξ£ let 𝑍 be number of substrings 𝜌 in π‘Œ1, π‘Œ2, β‹― , π‘Œπ‘œ 𝑍 = 𝑔(π‘Œ1, π‘Œ2, β‹― , π‘Œπ‘œ)

slide-153
SLIDE 153

The Method of Bounded Differences in Action:

Pattern Matching

  • a random string of length π‘œ
  • a pattern of length 𝑙
  • # of matched substrings?

an alphabet Ξ£ with Ξ£ = 𝑛, a fixed pattern 𝜌 ∈ Σ𝑙 independently and uniformly generate: π‘Œ1, π‘Œ2, β‹― , π‘Œπ‘œ ∈ Ξ£ let 𝑍 be number of substrings 𝜌 in π‘Œ1, π‘Œ2, β‹― , π‘Œπ‘œ 𝑍 = 𝑔(π‘Œ1, π‘Œ2, β‹― , π‘Œπ‘œ)

slide-154
SLIDE 154

The Method of Bounded Differences in Action:

Pattern Matching

  • a random string of length π‘œ
  • a pattern of length 𝑙
  • # of matched substrings?

an alphabet Ξ£ with Ξ£ = 𝑛, a fixed pattern 𝜌 ∈ Σ𝑙 independently and uniformly generate: π‘Œ1, π‘Œ2, β‹― , π‘Œπ‘œ ∈ Ξ£ let 𝑍 be number of substrings 𝜌 in π‘Œ1, π‘Œ2, β‹― , π‘Œπ‘œ 𝑍 = 𝑔(π‘Œ1, π‘Œ2, β‹― , π‘Œπ‘œ) changing any π‘Œπ‘— changes 𝑔 for at most 𝑙

slide-155
SLIDE 155

The Method of Bounded Differences in Action:

Pattern Matching

  • a random string of length π‘œ
  • a pattern of length 𝑙
  • # of matched substrings?

an alphabet Ξ£ with Ξ£ = 𝑛, a fixed pattern 𝜌 ∈ Σ𝑙 independently and uniformly generate: π‘Œ1, π‘Œ2, β‹― , π‘Œπ‘œ ∈ Ξ£ let 𝑍 be number of substrings 𝜌 in π‘Œ1, π‘Œ2, β‹― , π‘Œπ‘œ 𝑍 = 𝑔(π‘Œ1, π‘Œ2, β‹― , π‘Œπ‘œ) changing any π‘Œπ‘— changes 𝑔 for at most 𝑙

slide-156
SLIDE 156

The Method of Bounded Differences in Action:

Occupancy Problem

  • 𝑛 balls into π‘œ bins
  • number of empty bins?
slide-157
SLIDE 157

The Method of Bounded Differences in Action:

Occupancy Problem

  • 𝑛 balls into π‘œ bins
  • number of empty bins?

let π‘Œπ‘— be i.r.v. denoting whether the ith bin is empty let π‘Œ = σ𝑗=1

π‘œ

π‘Œπ‘— denote the number of empty bins

slide-158
SLIDE 158

The Method of Bounded Differences in Action:

Occupancy Problem

  • 𝑛 balls into π‘œ bins
  • number of empty bins?

let π‘Œπ‘— be i.r.v. denoting whether the ith bin is empty let π‘Œ = σ𝑗=1

π‘œ

π‘Œπ‘— denote the number of empty bins

slide-159
SLIDE 159

The Method of Bounded Differences in Action:

Occupancy Problem

  • 𝑛 balls into π‘œ bins
  • number of empty bins?

let π‘Œπ‘— be i.r.v. denoting whether the ith bin is empty let π‘Œ = σ𝑗=1

π‘œ

π‘Œπ‘— denote the number of empty bins deviation: β„™ π‘Œ βˆ’ 𝔽 π‘Œ β‰₯ 𝑒 ≀ ? π‘Œπ‘— are not independent!

slide-160
SLIDE 160

The Method of Bounded Differences in Action:

Occupancy Problem

  • 𝑛 balls into π‘œ bins
  • number of empty bins?

let π‘Œ denote # of empty bins, we are interested in the deviation: β„™ π‘Œ βˆ’ 𝔽 π‘Œ β‰₯ 𝑒 ≀ ?

slide-161
SLIDE 161

The Method of Bounded Differences in Action:

Occupancy Problem

  • 𝑛 balls into π‘œ bins
  • number of empty bins?

let π‘Œ denote # of empty bins, we are interested in the deviation: β„™ π‘Œ βˆ’ 𝔽 π‘Œ β‰₯ 𝑒 ≀ ? let 𝑍

π‘˜ be the bin that ball π‘˜ landed in (thus 𝑍 π‘˜ are independent)

slide-162
SLIDE 162

The Method of Bounded Differences in Action:

Occupancy Problem

  • 𝑛 balls into π‘œ bins
  • number of empty bins?

let π‘Œ denote # of empty bins, we are interested in the deviation: β„™ π‘Œ βˆ’ 𝔽 π‘Œ β‰₯ 𝑒 ≀ ? let 𝑍

π‘˜ be the bin that ball π‘˜ landed in (thus 𝑍 π‘˜ are independent)

π‘Œ = 𝑔 𝑍

1, 𝑍 2, β‹― , 𝑍 𝑛 = | π‘œ \{𝑍 1, 𝑍 2, β‹― , 𝑍 𝑛}|

slide-163
SLIDE 163

The Method of Bounded Differences in Action:

Occupancy Problem

  • 𝑛 balls into π‘œ bins
  • number of empty bins?

let π‘Œ denote # of empty bins, we are interested in the deviation: β„™ π‘Œ βˆ’ 𝔽 π‘Œ β‰₯ 𝑒 ≀ ? let 𝑍

π‘˜ be the bin that ball π‘˜ landed in (thus 𝑍 π‘˜ are independent)

π‘Œ = 𝑔 𝑍

1, 𝑍 2, β‹― , 𝑍 𝑛 = | π‘œ \{𝑍 1, 𝑍 2, β‹― , 𝑍 𝑛}|

notice changing any 𝑍

π‘˜ changes π‘Œ for at most 1 (Lipschitz condition)

slide-164
SLIDE 164

The Method of Bounded Differences in Action:

Occupancy Problem

  • 𝑛 balls into π‘œ bins
  • number of empty bins?

let π‘Œ denote # of empty bins, we are interested in the deviation: β„™ π‘Œ βˆ’ 𝔽 π‘Œ β‰₯ 𝑒 ≀ ? let 𝑍

π‘˜ be the bin that ball π‘˜ landed in (thus 𝑍 π‘˜ are independent)

π‘Œ = 𝑔 𝑍

1, 𝑍 2, β‹― , 𝑍 𝑛 = | π‘œ \{𝑍 1, 𝑍 2, β‹― , 𝑍 𝑛}|

notice changing any 𝑍

π‘˜ changes π‘Œ for at most 1 (Lipschitz condition)

slide-165
SLIDE 165

Concentration Inequalities

Question: probability that X deviates more than πœ€ from expectation?

slide-166
SLIDE 166

Concentration Inequalities

Question: probability that X deviates more than πœ€ from expectation?

slide-167
SLIDE 167

Concentration Inequalities

Question: probability that X deviates more than πœ€ from expectation?

slide-168
SLIDE 168

Concentration Inequalities

Question: probability that X deviates more than πœ€ from expectation?

slide-169
SLIDE 169

Concentration Inequalities

Question: probability that X deviates more than πœ€ from expectation?

slide-170
SLIDE 170

Concentration Inequalities

Question: probability that X deviates more than πœ€ from expectation?

slide-171
SLIDE 171

Concentration Inequalities

Question: probability that X deviates more than πœ€ from expectation?