Lecture 14 Whats to come? Probability. A bag contains: What is the - - PowerPoint PPT Presentation

Lecture 14

What’s to come? Probability. A bag contains: What is the chance that a ball taken from the bag is blue? Count blue. Count total. Divide. Today: Counting! Later: Probability. Professor Walrand.

Outline: basics

• 1. Counting.
• 2. Tree
• 3. Rules of Counting
• 4. Sample with/without replacement where order does/doesn’t

matter.

Probability is soon..but first let’s count.

Count?

How many outcomes possible for k coin tosses? How many poker hands? How many handshakes for n people? How many diagonals in a convex polygon? How many 10 digit numbers? How many 10 digit numbers without repetition?

Using a tree..

How many 3-bit strings? How many different sequences of three bits from {0,1}? How would you make one sequence? How many different ways to do that making? 000 001 1 010 011 1 1 100 101 1 110 111 1 1 1 8 leaves which is 2×2×2. One leaf for each string. 8 3-bit srings!

First Rule of Counting: Product Rule

Objects made by choosing from n1, then n2, ..., then nk the number of objects is n1 ×n2 ···×nk. n1 ×n2 ×n3 · · · · · · · · · · · · In picture, 2×2×3 = 12!

Using the first rule..

How many outcomes possible for k coin tosses? 2 ways for first choice, 2 ways for second choice, ... 2×2··· ×2 = 2k How many 10 digit numbers? 10 ways for first choice, 10 ways for second choice, ... 10×10··· ×10 = 10k How many n digit base m numbers? m ways for first, m ways for second, ... mn

Functions, polynomials.

How many functions f mapping S to T? |T| ways to choose for f(s1), |T| ways to choose for f(s2), ... ....|T||S| How many polynomials of degree d modulo p? p ways to choose for first coefficient, p ways for second, ... ...pd+1 p values for first point, p values for second, ... ...pd+1

Permutations.

How many 10 digit numbers without repeating a digit? 10 ways for first, 9 ways for second, 8 ways for third, ... ... 10∗9∗8···∗1 = 10!.1 How many different samples of size k from n numbers without replacement. n ways for first choice, n −1 ways for second, n −2 choices for third, ... ... n ∗(n −1)∗(n −2)·∗(n −k +1) =

n! (n−k)!.

How many orderings of n objects are there? Permutations of n objects. n ways for first, n −1 ways for second, n −2 ways for third, ... ... n ∗(n −1)∗(n −2)·∗1 = n!.

1By definition: 0! = 1.

One-to-One Functions.

How many one-to-one functions from |S| to |S|. |S| choices for f(s1), |S|−1 choices for f(s2), ... So total number is |S|×|S|−1···1 = |S|! A one-to-one function is a permutation!

Counting sets..when order doesn’t matter.

How many poker hands? 52×51×50×49×48 ??? Are A,K,Q,10,J of spades and 10,J,Q,K,A of spades the same? Second Rule of Counting: If order doesn’t matter count ordered

• bjects and then divide by number of orderings.2

Number of orderings for a poker hand: 5!. 52×51×50×49×48 5! Can write as... 52! 5!×47! Generic: ways to choose 5 out of 52 possibilities.

2When each unordered object corresponds equal numbers of ordered

• bjects.

Ordered to unordered.

Second Rule of Counting: If order doesn’t matter count ordered

• bjects and then divide by number of orderings.

How many red nodes (ordered objects)? 9. How many red nodes mapped to one blue node? 3. How many blue nodes (unordered objects)? 9

3 = 3.

How many poker deals? 52·51·50·49·48. How many poker hands per deal? Map each deal to ordered deal. 5! How many poker hands? 52·51·50·49·48

5!

..order doesn’t matter.

Choose 2 out of n? n ×(n −1) 2 = n! (n −2)!×2 Choose 3 out of n? n ×(n −1)×(n −2) 3! = n! (n −3)!×3! Choose k out of n? n! (n −k)!×k! Notation: n

k

• and pronounced “n choose k.”

Example: Visualize the proof..

First rule: n1 ×n2 ···×n3. Product Rule. Second rule: when order doesn’t matter divide..when possible.

... ... ... ...

∆

3 card Poker deals: 52×51×50 = 52!

49!. First rule.

Poker hands: ∆? Hand: Q,K,A. Deals: Q,K,A, Q,A,K, K,A,Q,K,A,Q, A,K,Q, A,Q,K. ∆ = 3×2×1 First rule again. Total:

52! 49!3! Second Rule!

Choose k out of n. Ordered set:

n! (n−k)!

What is ∆? k! First rule again. = ⇒ Total:

n! (n−k)!k! Second rule.

Example: Anagram

First rule: n1 ×n2 ···×n3. Product Rule. Second rule: when order doesn’t matter divide..when possible.

... ... ... ...

∆

Orderings of ANAGRAM? Ordered Set: 7! First rule. A’s are the same! What is ∆? ANAGRAM A1NA2GRA3M , A2NA1GRA3M , ... ∆ = 3×2×1 = 3! First rule! = ⇒

7! 3!

Second rule!

Some Practice.

How many orderings of letters of CAT? 3 ways to choose first letter, 2 ways to choose second, 1 for last. = ⇒ 3×2×1 = 3! orderings How many orderings of the letters in ANAGRAM? Ordered, except for A! total orderings of 7 letters. 7! total “extra counts” or orderings of two A’s? 3! Total orderings? 7!

3!

How many orderings of MISSISSIPPI? 4 S’s, 4 I’s, 2 P’s. 11 letters total! 11! ordered objects! 4!×4!×2! ordered objects per “unordered object” = ⇒

11! 4!4!2!.

Sampling...

Sample k items out of n Without replacement: Order matters: n ×n −1×n −2... ×n −k +1 =

n! (n−k)!

Order does not matter: Second Rule: divide by number of orders – “k!” = ⇒

n! (n−k)!k!.

“n choose k” With Replacement. Order matters: n ×n ×...n = nk Order does not matter: Second rule ??? Problem: depends on how many of each item we chose! So different number of unordered elts map to each unordered elt. Unordered elt: 1,2,3 3! ordered elts map to it. Unordered elt: 1,2,2

3! 2! ordered elts map to it.

How do we deal with this mess?!?!

Splitting up some money....

How many ways can Bob and Alice split 5 dollars? For each of 5 dollars pick Bob or Alice(25), divide out order ??? 5 dollars for Bob and 0 for Alice:

• ne ordered set: (B,B,B,B,B).

4 for Bob and 1 for Alice: 5 ordered sets: (A,B,B,B,B) ; (B,A,B,B,B); ... “Sorted” way to specify, first Alice’s dollars, then Bob’s. (B,B,B,B,B) (B,B,B,B,B) (A,B,B,B,B) (A,B,B,B,B),(B,A,B,B,B),(B,B,A,B,B),... (A,A,B,B,B) (A,A,B,B,B),(A,B,A,B,B),(A,B,B,A,B),... and so on.

... ... ... ...

∆ ??

Second rule of counting is no good here!

Splitting 5 dollars..

How many ways can Alice, Bob, and Eve split 5 dollars. Alice gets 3, Bob gets 1, Eve gets 1: (A,A,A,B,E). Separate Alice’s dollars from Bob’s and then Bob’s from Eve’s. Five dollars are five stars: ⋆⋆⋆⋆⋆. Alice: 2, Bob: 1, Eve: 2. Stars and Bars: ⋆⋆|⋆|⋆⋆. Alice: 0, Bob: 1, Eve: 4. Stars and Bars: |⋆|⋆⋆⋆⋆. Each split “is” a sequence of stars and bars. Each sequence of stars and bars “is” a split. Counting Rule: if there is a one-to-one mapping between two sets they have the same size!

Stars and Bars.

How many different 5 star and 2 bar diagrams? | ⋆ | ⋆ ⋆ ⋆ ⋆. 7 positions in which to place the 2 bars. Alice: 0; Bob 1; Eve: 4 | ⋆ | ⋆ ⋆ ⋆ ⋆. Bars in first and third position. Alice: 1; Bob 4; Eve: 0 ⋆ | ⋆ ⋆ ⋆ ⋆ |. Bars in second and seventh position. 7

2

• ways to do so and

7

2

• ways to split 5 dollars among 3 people.

Stars and Bars.

Ways to add up n numbers to sum to k? or “ k from n with replacement where order doesn’t matter.” In general, k stars n −1 bars. ⋆⋆|⋆|···|⋆⋆. n +k −1 positions from which to choose n −1 bar positions. n +k −1 n −1

• Or: k unordered choices from set of n possibilities with replacement.

Sample with replacement where order doesn’t matter.

Summary.

First rule: n1 ×n2 ···×n3. k Samples with replacement from n items: nk. Sample without replacement:

n! (n−k)!

Second rule: when order doesn’t matter (sometimes) can divide... Sample without replacement and order doesn’t matter: n

k

• =

n! (n−k)!k!.

“n choose k” One-to-one rule: equal in number if one-to-one correspondence. pause Bijection! Sample k times from n objects with replacement and order doesn’t matter: k+n−1

n

• .

Quick review of the basics.

First rule: n1 ×n2 ···×n3. k Samples with replacement from n items: nk. Sample without replacement:

n! (n−k)!

Second rule: when order doesn’t matter divide..when possible. Sample without replacement and order doesn’t matter: n

k

• =

n! (n−k)!k!.

“n choose k” One-to-one rule: equal in number if one-to-one correspondence. Sample with replacement and order doesn’t matter: k+n−1

n

• .

Balls in bins.

“k Balls in n bins” ≡ “k samples from n possibilities.” “indistinguishable balls” ≡ “order doesn’t matter” “only one ball in each bin” ≡ “without replacement” 5 balls into 10 bins 5 samples from 10 possibilities with replacement Example: 5 digit numbers. 5 indistinguishable balls into 52 bins only one ball in each bin 5 samples from 52 possibilities without replacement Example: Poker hands. 5 indistinguishable balls into 3 bins 5 samples from 3 possibilities with replacement and no order Dividing 5 dollars among Alice, Bob and Eve.

Sum Rule

Two indistinguishable jokers in 54 card deck. How many 5 card poker hands? Sum rule: Can sum over disjoint sets. No jokers “exclusive” or One Joker “exclusive” or Two Jokers 52

5

• +

52

4

• +

52

3

• .

Two distinguishable jokers in 54 card deck. How many 5 card poker hands? Choose 4 cards plus one of 2 jokers! 52

5

• +2∗

52

4

• +

52

3

• Wait a minute! Same as choosing 5 cards from 54 or

54

5

• Theorem:

54

5

• =

52

5

• +2∗

52

4

• +

52

3

• .

Algebraic Proof: Why? Just why? Especially on Thursday! Above is combinatorial proof.

Combinatorial Proofs.

Theorem: n

k

• =

n

n−k

• Proof: How many subsets of size k?

n

k

• How many subsets of size k?

Choose a subset of size n −k and what’s left out is a subset of size k. Choosing a subset of size k is same as choosing n −k elements to not take. = ⇒ n

n−k

• subsets of size k.

Pascal’s Triangle

1 1 1 2 1 1 3 3 1 1 4 6 4 1 Row n: coefficients of (1+x)n = (1+x)(1+x)···(1+x). Foil (4 terms) on steroids: 2n terms: choose 1 or x froom each factor of (1+x). Simplify: collect all terms corresponding to xk. Coefficient of xk is n

k

• : choose k factors where x is in product.
• 1
• 1

1

• 2
• 2

1

• 2

2

• 3
• 3

1

• 3

2

• 3

3

• Pascal’s rule =

⇒ n+1

k

• =

n

k

• +

n

k−1

• .

Combinatorial Proofs.

Theorem: n+1

k

• =

n

k

• +

n

k−1

• .

Proof: How many size k subsets of n +1? n+1

k

• .

How many size k subsets of n +1? How many contain the first element? Chose first element, need to choose k −1 more from remaining n elements. = ⇒ n

k−1

• How many don’t contain the first element ?

Need to choose k elements from remaining n elts. = ⇒ n

k

• So,

n

k−1

• +

n

k

• =

n+1

k

• .

Combinatorial Proof.

Theorem: n

k

• =

n−1

k−1

• +···+

k−1

k−1

• .

Proof: Consider size k subset where i is the first element chosen. {1,...,i,...,n} Must choose k −1 elements from n −i remaining elements. = ⇒ n−i

k−1

• such subsets.

Add them up to get the total number of subsets of size k which is also n+1

k

• .

Binomial Theorem: x = 1

Theorem: 2n = n

n

• +

n

n−1

• +···+

n

• Proof: How many subsets of {1,...,n}?

Construct a subset with sequence of n choices: element i is in or is not in the subset: 2 poss. First rule of counting: 2×2···×2 = 2n subsets. How many subsets of {1,...,n}? n

i

• ways to choose i elts of {1,...,n}.

Sum over i to get total number of subsets..which is also 2n.

Simple Inclusion/Exclusion

Sum Rule: For disjoint sets S and T, |S ∪T| = |S|+|T| Used to reason about all subsets by adding number of subsets of size 1, 2, 3,. . . Also reasoned about subsets that contained

• r didn’t contain an element. (E.g., first element, first i elements.)

Inclusion/Exclusion Rule: For any S and T, |S ∪T| = |S|+|T|−|S ∩T|. Example: How many 10-digit phone numbers have 7 as their first or second digit? S = phone numbers with 7 as first digit.|S| = 109 T = phone numbers with 7 as second digit. |T| = 109. S ∩T = phone numbers with 7 as first and second digit. |S ∩T| = 108. Answer: |S|+|T|−|S ∩T| = 109 +109 −108.

Summary.

First Rule of counting: Objects from a sequence of choices: ni possibilitities for ith choice. n1 ×n2 ×···×nk objects. Second Rule of counting: If order does not matter. Count with order. Divide by number of orderings/sorted object. Typically: n

k

• .

Stars and Bars: Sample k objects with replacement from n. Order doesn’t matter. Typically: n+k−1

k−1

• .

Inclusion/Exclusion: two sets of objects. Add number of each subtract intersection of sets. Sum Rule: If disjoint just add. Combinatorial Proofs: Identity from counting same in two ways. Pascal’s Triangle Example: n+1

k

• =

n

k−1

• +

n

k

• .

RHS: Number of subsets of n +1 items size k. LHS: n

k−1

• counts subsets of n +1 items with first item.

n

k

• counts subsets of n +1 items without first item.

Disjoint – so add!