CPSC 121: Models of Computation Module 9: Proof Techniques (part 2) - - PowerPoint PPT Presentation
CPSC 121: Models of Computation Module 9: Proof Techniques (part 2) Mathematical Induction Module 9: Announcements Assignment #5 is due Wednesday November 28 th at 19:00. Pre-class quiz #10 is tentatively due on Wednesday November 21 st at
Module 9: Proof Techniques (part 2) Mathematical Induction
CPSC 121 – 2018W T1 2
Assignment #5 is due Wednesday November 28th at 19:00. Pre-class quiz #10 is tentatively due on Wednesday November 21st at 19:00.
Textbook sections:
Epp, 4th edition: 6.1, 7.1 Epp, 3rd edition: 5.1, 6.1 Rosen, 6th edition: 2.1, 2.3 up to the top of page 136. Rosen, 7th edition: 2.1, 2.3 down to the bottom of page 141.
CPSC 121 – 2018W T1 3
By the start of class, you should be able to
Convert sequences to and from explicit formulas that describe the sequence. Convert sums to and from summation/Σ notation. Convert products to and from product/π notation. Manipulate formulas in summation/product notation by adjusting their bounds, merging or splitting summations/products, and factoring out values.
CPSC 121 – 2018W T1 4
Pre-class quiz #9
Well done overall. You had a bit more trouble (77% avg) on the questions about the coin-changing induction proof. We will discuss induction proofs at length and do a lot of examples. Being comfortable with summations is important not
As usual, we will revisit the open-ended question shortly.
CPSC 121 – 2018W T1 5
If you haven’t already done so,
read the rest of the chapters on mathematical induction (Epp4, 5.2 to 5.4, etc).
After this reading, you will be able to:
Given a theorem to prove stated in terms of its induction variable (i.e., usually, in terms of n), write
the base case(s) that need to be proven, the induction hypothesis, and the inductive step that needs to be proven.
CPSC 121 – 2018W T1 6
By the end of this module, you should be able to:
Establish properties of self-referential structures using inductive proofs that naturally build on those self-references. Critique formal inductive proofs to determine whether they are valid and where the error(s) lie if they are invalid. Prove properties of the non-negative integers (or a subset) with appropriate self-referential structure using weak or strong induction as needed.
CPSC 121 – 2018W T1 7
CPSC 121: the BIG questions:
does what it's supposed to do?
is better than another one?
Mathematical induction is a very very useful tool when proving the correctness or efficiency of an algorithm. We will see several examples of this.
CPSC 121 – 2018W T1 8
Module Summary
Example: single-elimination tournaments. Defining and validating mathematical induction. More examples where we can use induction. A slightly different type of induction.
CPSC 121 – 2018W T1 9
Problem: single-elimination tournament
Teams play one another in pairs The winner of each pair advances to the next round
Los Angeles Montréal Toronto New York Isl. Vancouver
Pittsburgh Buffalo Round 2
CPSC 121 – 2018W T1 10
Question: if we have n rounds of playoffs, how many teams can participate?
a) n b) 2n c) n2 d) 2n e) None of the above.
▷
CPSC 121 – 2018W T1 12
How can we prove this result formally?
We will use a technique called mathematical induction. We will convince ourselves that it is a valid proof technique.
Many theorems involve summations, so it is important to be comfortable manipulating them.
Hence the reading for pre-class quiz #9.
CPSC 121 – 2018W T1 13
First we look at the following question instead:
If t teams can participate in a playoff with n rounds, how many teams can participate in a playoff with n+1 rounds? a) t+1 b) 2t c) t2 d) 2t e) None of the above.
▷
CPSC 121 – 2018W T1 15
Proof (with holes):
Consider an unspecified playoff with n rounds. Assume that We can think of a playoff with n+1 rounds as follows:
Two playoffs with n rounds proceed in parallel. The two winners then
Since each playoff with n rounds has
CPSC 121 – 2018W T1 16
How do we use this to prove what we want? Recall Universal Modus Ponens:
P(a) ∀x D, P(x) → Q(x) ∈ \ Q(a)
If we define
P(t, n): Up to t teams can participate in a playoff with n rounds
then we have proved
∀t Z ∈
+ n
Z ∀ ∈
+ , P(t, n)→P(2t, n+1)
CPSC 121 – 2018W T1 17
So
We know that only 2 teams can play with 1 round. This is P(2, 1).
Given
P(2, 1) ∀t Z ∈
+ n
Z ∀ ∈
+, P(t, n) → P(2t, n+1)
We can deduce:
P(4, 2)
CPSC 121 – 2018W T1 18
Given
P(4, 2) ∀t Z ∈
+ n
Z ∀ ∈
+, P(t, n) → P(2t, n+1)
We can deduce:
P(8, 3)
Given
P(8, 3) ∀t Z ∈
+ n
Z ∀ ∈
+, P(t, n) → P(2t, n+1)
We can deduce:
P(16, 4)
CPSC 121 – 2018W T1 19
Given
P(16, 4) ∀t Z ∈
+ n
Z ∀ ∈
+, P(t, n) → P(2t, n+1)
We can deduce:
P(32, 5)
Etc... Therefore we can show that
∀n Z ∈
+, P(2n, n)
CPSC 121 – 2018W T1 20
Another way to think about this:
Given an unspecified positive integer n. To prove P(2n+1, n+1):
First we prove P(2n, n). Then we use the fact that
∀t Z ∈
+ n
Z ∀ ∈
+, P(t, n) → P(2t, n+1)
Hence induction is more or less the same as recursion!
A fun example involving induction:
http://www.youtube.com/watch?v=daRS98L9rHs&feature=related
CPSC 121 – 2018W T1 21
Module Summary
Example: single-elimination tournaments. Defining and validating mathematical induction. More examples where we can use induction. A slightly different type of induction.
CPSC 121 – 2018W T1 22
Define Q(n) by:
Q(n): P(2n,n)
In the previous example, we proved
Q(1) ∀n Z ∈
+, Q(n) → Q(n+1)
and deduced that
∀n Z ∈
+, Q(n)
Why is this a valid proof technique?
CPSC 121 – 2018W T1 23
Theorem:
Q(1) ^ ( n Z ∀ ∈
+, Q(n) → Q(n+1)) → n
Z ∀ ∈
+, Q(n)
Proof:
We use a proof by contradiction. Suppose that the premises hold, but that n Z ∀ ∈
+,
Q(n) is false. There is at least one value of n for which Q(n) does not hold. Let y be the smallest such value. Clearly y > 1, so y – 1 Z ∈
+.
CPSC 121 – 2018W T1 24
Proof (continued):
Because y is the smallest value for which Q(n) is false, and y – 1 Z ∈
+, Q(y - 1) is true.
But then the 2nd premise implies that Q(y) is true, a contradiction (since we assumed Q(y) was false). Therefore the conclusion must be true, that is n ∀ ∈ Z+, Q(n) holds.
CPSC 121 – 2018W T1 25
So a proof by mathematical induction has
One (or more) base case(s): Q(1)
Usually very straightforward to prove.
An induction step:
∀n Z ∈
+, Q(n) → Q(n+1) OR n
Z ∀ ∈
+, Q(n-1) → Q(n)
We can use any proof technique we know for it. Usually a direct proof will work well.
CPSC 121 – 2018W T1 26
How do we decide which base cases we need?
Look at the induction step. Decide for which values
The induction step might not be usable for some n:
because it needs Q(some x < n) and x would be smaller than smallest n for which Q holds. because some mathematical steps only work for large enough values of n.
If the induction step is not usable for n, then n needs to be a base case!
CPSC 121 – 2018W T1 27
Module Summary
Example: single-elimination tournaments. Defining and validating mathematical induction. More examples where we can use induction. A slightly different type of induction.
CPSC 121 – 2018W T1 28
Example 1: recall the exercises from our first class:
Make group of 5 students. Then order each group from in order of day of birth by swapping pairs of adjacent students.
We claimed the maximum number of swaps for n students is n(n-1)/2.
CPSC 121 – 2018W T1 29
How many swaps do we need?
Suppose we place students from left to right.
The students already placed are ordered by day of birth. We swap each new student with his/her neighbour until he/she is at the right place.
The ith student may be swapped with all previous i-1 students. So the total number of swaps is at most Hence we need to prove that
∑
i=0 n−1
i
i=0 n−1
i=n(n−1) 2
CPSC 121 – 2018W T1 30
Which fact(s) do we need to prove?
a) b) c) d) Both (a) and (b) e) Both (a) and (c)
∑
i=0
i=0 ∀ n∈ℤ
+(∑ i=0 n−1
i= n(n−1) 2
i=0 n
i=(n+1) n 2
+ ,∑ i=0 n−1
i= n(n−1) 2
+ ,∑ i=0 n
i=(n+1)n 2
▷
CPSC 121 – 2018W T1 32
Proof:
Base case: n = 1
Clearly, with only 1 student, no swap is needed: which is equal to 1 (1-1)/2
Induction step:
Pick an unspecified n ≥ 1. Assume that when we have n students, we need at most n(n-1)/2 swaps. Equivalently we assume that This is called the Induction Hypothesis.
∑
i=0
i=0
∑
i=0 n−1
i=n(n−1) 2
CPSC 121 – 2018W T1 33
Proof (continued)
Induction step (continued)
For n+1 students, we need at most swaps. By the induction hypothesis, this is equal to swaps, as required for the induction step.
Hence by the principle of M.I., the theorem holds. QED
∑
i=0 n
i=(∑
i=0 n−1
i)+ n n(n−1) 2 + n=(n+ 1)n 2
CPSC 121 – 2018W T1 34
Example 2: geometric series
We will prove that for every value of a ≠0, 1: These summations occur frequently when we need to determine the running time of divide-and-conquer algorithms.
∑
i=0 t
a
i=a t+ 1−1
a−1
CPSC 121 – 2018W T1 35
Proof:
Base case: t = 0
In this case the summation is a0 = 1, and
Induction step:
Pick an unspecified t ≥ 0. Assume that Now
Hence by the principle of M.I., the theorem holds. QED
a
1−1
a−1 =1
CPSC 121 – 2018W T1 36
Example 3: prove that n ≥ 4, 2n < n! Rules for inequalities:
Start from one side (say the left side) Work step by step towards the other. When dealing with <, you are allowed to make the expression on the RHS larger, but never smaller.
Example: if I am smaller than you, then I am still smaller than you when you stand on a bench.
CPSC 121 – 2018W T1 37
Proof: by induction on n.
Base case: n = Induction step: we want to prove that n ≥ , if 2n-1 < (n-1)! then 2n < n!
Consider an unspecified n ≥ . Induction hypothesis: assume that 2n-1 < (n-1)! Then 2n = 2(2n-1) < 2(n-1)! < n(n-1)! = n!
this is where we “approximate” the induction hypothesis is used here
Hence by the principle of M.I., n ≥ 4, 2n < n!
CPSC 121 – 2018W T1 38
Example 4: prove that for every n ≥ 1,
Note (this doesn’t help with the proof; it’s simply an interesting mathematical fact):
∑
i=1 n 1
i
2≤2−1
n
∑
i=1 ∞ 1
i
2=π 2
6
CPSC 121 – 2018W T1 39
Example 5: consider the following DFA: 3 2 4 1
CPSC 121 – 2018W T1 40
This DFA accepts the empty string, and which
a) Strings whose length is divisible by 5. b) Unsigned binary integers that are prime numbers. c) Strings where every 1 is followed by 01. d) Unsigned binary integers that are divisible by 5. e) None of the above.
▷
CPSC 121 – 2018W T1 42
What should we prove (ignoring empty strings)?
a) The DFA is in state 0 if and only if it has seen one of the strings 1010, 1111 or 11001. b) The DFA is in state 0 if and only if s is divisible by 5. c) The DFA is in state 0 if and only if s is terminated by the string 101. d) The DFA is in state 0 if and only if s contains the substring 101 followed by zero or more 0's. e) None of the above.
▷
CPSC 121 – 2018W T1 44
Theorem: The DFA ends up in state r after reading a string s if and only if the integer xs that s represents can be written as xs = 5q+r where q is an integer. Proof: by induction on the length n of s.
Base case: n = 1
This can be verified easily by looking at the DFA.
Induction step: consider an unspecified n > 1.
Induction hypothesis: the theorem holds for strings with n – 1 characters. Consider an unspecified string s with n bits.
CPSC 121 – 2018W T1 45
Proof (continued):
Suppose that s = bn-1bn-2...b2b1b0
Let xs be the unsigned integer represented by the string s.
and that t = bn-1bn-2...b2b1.
Let xt be the unsigned integer represented by the string t.
After reading the bits of t, the DFA is in some state r. By the induction hypothesis, the binary integer xt = 5q + r for some integer q.
CPSC 121 – 2018W T1 46
Proof (continued):
Now, xs = 2xt + b0. This means xs = 2(5q+r) + b0
= 10q + 2r + b0 = 5 (2q) + (2r + b0).
So the DFA should have the following transitions:
r b0 2r+b0 1 2 2 4 3 5 + 1 4 5 + 3 r b0 2r+b0 1 1 1 1 3 2 1 5 + 0 3 1 5 + 2 4 1 5 + 4
CPSC 121 – 2018W T1 47
Proof (continued):
It does! Hence after reading all n bits of s, the DFA ends up in state r if and only if the integer xs that s represents can be written as xs = 5q+r where q is an integer.
Hence by the principle of mathematical induction, the DFA ends up in state r after reading a string s if and
Z, s = 5q+r ∈ . QED
Corollary: the DFA is in state 0 if and only if s is divisible by 5.
CPSC 121 – 2018W T1 48
Mathematical induction is how we analyze recursive algorithms: Example:
(define (sum n) (if (= n 0) (+ n (sum (- n 1)))))
The type of induction we discussed so far works when the recursive call is with (- n 1)
i=0 n
i=0 n−1
CPSC 121 – 2018W T1 49
Module Summary
Example: single-elimination tournaments. Defining and validating mathematical induction. More examples where we can use induction. A slightly different type of induction.
CPSC 121 – 2018W T1 50
How do we handle more general recursions?
A recursive function/method has
One or more base cases
When it can compute the answer directly
An induction step
It computes recursively the solutions to one or more “smaller” sub-problems, and combines them to obtain its answer.
Our induction proofs mimic this structure.
It is similar to the templates you used in CPSC 110.
CPSC 121 – 2018W T1 51
We use a slightly different induction hypothesis.
Instead of proving that
∀x Z ∈
+, Q(x) → Q(x+1)
We prove that
∀x Z ∈
+, (Q(1) ^ Q(2) ^ ... ^ Q(x)) → Q(x+1)
We can also show that this type of induction is a valid proof technique.
The proof is the same proof by contradiction that we looked at earlier.
CPSC 121 – 2018W T1 52
CPSC 110 review: A binary tree is a data structure that is defined recursively: it is either
Empty, or A node with some data, and two children that are themselves trees.
20 9 2 15 5 10 17 7
CPSC 121 – 2018W T1 53
Example 6: consider the following function:
(define (tree-size t) (if (null? t) (+ 1 (tree-size (left-child t)) (tree-size (right-child t)))))
How can we prove that it correctly computes the number of (non-null) nodes of the tree?
CPSC 121 – 2018W T1 54
We prove this using mathematical induction on the size of the tree.
Base case: t is null
In this case t contains exactly 0 nodes.
Induction step:
Assume the algorithm works for trees that are smaller than t. Because the left sub-tree of t is smaller than t, the 1st recursive calls correctly returns the size of the left sub- tree of t.
CPSC 121 – 2018W T1 55
Proof (continued)
Induction step (continued)
Similarly the right sub-tree of t is smaller than t, and so the 2nd recursive call correctly returns the size of the right sub-tree of t. But we return 1 + the sum of the values returned by the recursive calls. This is exactly the size of t (1 for the root, and the sum
Hence by the principle of M.I., our algorithm computes correctly the size of every tree. QED
CPSC 121 – 2018W T1 56
Example 7: every positive integer n greater than 1 can be written as a product of primes. For which case(s) is the proof obvious?
a) n = 1 b) n = 2 c) n = 2, 3 or 5. d) n is prime. e) None of the above.
▷
CPSC 121 – 2018W T1 58
Proof: we prove the result by induction on n.
Base case: n
We can write n = n and n is a product of primes (with
Induction step: n
Suppose that every value from 2 to n – 1 is a product of primes. Since n
We can write n =
CPSC 121 – 2018W T1 59
Proof (continued)
Since 2 ≤ a < n, Since 2 ≤ b < n, Then
Hence by the principle of M.I., every positive integer larger than 1 can be written as a product of primes. QED
CPSC 121 – 2018W T1 60
Example 8: we can find the ith smallest element in an unsorted list as follows:
Pick a random element x of the list. Divide the list into three sublists:
list-smaller: elements smaller than x list-equal: elements equal to x list-larger: elements larger than x Then search
list-smaller if i ≤ length of list-smaller list-larger if i > length of list smaller + length of list-equal
CPSC 121 – 2018W T1 61
Example 8: continued
This algorithm is called randomized-quick-select. Racket code will be posted on the course web site. A student shows the expected number of steps S(n)
S(1) = 4c S(2) = 12c S(3) = 20c S(n) ≤ 2cn + S(ë3n/4û) when n ≥ 4
CPSC 121 – 2018W T1 62
Example 8: continued
Prove that for every n ≥ 1, S(n) ≤ 8cn.
CPSC 121 – 2018W T1 63
Example 9: binary search
Suppose we have something like a list, but whose ith element and length can be found in a single step.
This structure is called a vector in Racket. It is similar to an ArrayList in Java.
We assume that we have such a vector, sorted in increasing order.
Examples: (“Anh”, “Charles”, “Dora”, “Gregor”, “Wei”).
We want to find the position of a given element (for instance, “Dora”).
CPSC 121 – 2018W T1 64
The following algorithm (formerly known as B) works:
(define (binary-search avector first-pos last-pos x) (if (> first-pos last-pos) (if (= first-pos last-pos) (local ((define mid-pos (quotient (+ first-pos last-pos) 2))) (if (= x (vector-ref avector mid-pos)) (if (< x (vector-ref avector mid-pos))
CPSC 121 – 2018W T1 66
Prove that binary search makes at most ⌈log2 (size+1)⌉ comparisons if size ≥ 1.
The proof is once again by induction on size. Base case: size =
The algorithm uses comparison, which is
Induction step:
Consider an unspecified size . Assume that