Induction and Recursion (Sections 4.1-4.3) [Section 4.4 optional] - - PDF document

SLIDE 1

1

Induction and Recursion (Sections 4.1-4.3) [Section 4.4 optional]

1 Based on Rosen and slides by K. Busch

Induction

2

Induction is a very useful proof technique In computer science, induction is used to prove properties of algorithms Induction and recursion are closely related

• Recursion is a description method for algorithms
• Induction is a proof method suitable

for recursive algorithms

SLIDE 2

2

3

Use induction to prove that a proposition is true:

) (n P

Inductive Basis: Inductive Hypothesis: Inductive Step: Prove that is true Assume is true Prove that is true

) 1 ( P ) (k P ) 1 (  k P

(for any positive integer k)

4

Inductive Step: Prove that is true

) 1 (  k P

Inductive Hypothesis: Assume is true

) (k P ) 1 ( ) (   k P k P

In other words in inductive step we prove:

(for any positive integer k)

for every positive integer k

SLIDE 3

3

5

) 1 ( ) (   k P k P ) 1 ( P

True True

     ) 4 ( ) 3 ( ) 2 ( ) 1 ( P P P P

Inductive basis Inductive Step Proposition true for all positive integers

6

Induction as a rule of inference:

) ( ))] 1 ( ) ( ( ) 1 ( [ n nP k P k P k P      

SLIDE 4

4

7

2 ) 1 ( 3 2 1 : ) (       n n n n P 

Inductive Basis:

2 ) 1 1 ( 1 1 : ) 1 (   P

Inductive Hypothesis:

2 ) 1 ( 2 1 : ) (     k k k k P 

Inductive Step: Theorem: Proof:

2 ) 1 ) 1 )(( 1 ( ) 1 ( 2 1 : ) 1 (          k k k k k P 

We will prove assume that it holds

8

2 ) 1 ) 1 )(( 1 ( 2 ) 1 ( 2 ) 1 ( ) 1 ( 2 ) 1 ( ) 1 ( 2 1 : ) 1 (                  k k k k k k k k k k k P 

Inductive Step: (inductive hypothesis) End of Proof

SLIDE 5

5

9

Harmonic numbers

j H j 1 4 1 3 1 2 1 1       

12 25 4 1 3 1 2 1 1

4

     H

Example:

 , 3 , 2 , 1  j

10

Theorem:

2 1

2

n H

n

 

Proof:

 n

Inductive Basis:

2 1 2 1 1

1 2 2

n H H H

n

        n

SLIDE 6

6

11

Inductive Hypothesis:

k n  2 1

2

k H

k

 

Suppose it holds: Inductive Step:

1   k n 2 1 1

1

2

  



k H

k

We will show:

2 1 1 2 1 2 1 2 1 2 2 1 2 1 1 2 1 2 1 2 1 1 2 1 2 1 1 2 1 2 1 3 1 2 1 1

1 1 1 2 1 2

1

                                               

   



k k k k H H

k k k k k k k k k

k k

   

12

End of Proof from inductive hypothesis

SLIDE 7

7

13

Theorem:

n H

n

 1

2

Proof:

 n

Inductive Basis:

n H H H

n

       1 1 1

1 2 2

 n

14

Inductive Hypothesis:

k n  k H

k

 1

2

Suppose it holds: Inductive Step:

1   k n ) 1 ( 1

1

2

  



k H

k

We will show:

SLIDE 8

8

15

     

) 1 ( 1 1 1 1 2 1 2 1 2 1 1 2 1 1 2 1 1 2 1 2 1 1 2 1 2 1 3 1 2 1 1

1 1 2 1 2

1

                              

  



k k k k H H

k k k k k k k k k

k k

   

End of Proof from inductive hypothesis

16

n H n

n

    1 2 1

2

We have shown:

   

k H k

k

log 1 2 log 1     k Hk log 

(for large k)

SLIDE 9

9

17

hole hole hole

2 2

2 2

2 2 

3 3

2 2 

Triominos

18

Theorem: Every checkerboard with one square removed can be tiled with triominoes

1 , 2 2   n

n n

Proof: Inductive Basis:

1  n

hole

1 1

2 2 

SLIDE 10

10

19

Inductive Hypothesis:

k n 

k k

2 2 

Hole can be anywhere Assume that a checkerboard can be tiled with the hole anywhere

k k

2 2 

20

Inductive Step:

1   k n

1 1

2 2

   k k

SLIDE 11

11

21

k k

2 2 

k k

2 2 

By inductive hypothesis squares with a hole can be tiled

k k

2 2 

add three artificial holes

22

23 x 23 case:

SLIDE 12

12

23

k k

2 2 

k k

2 2 

Replace the three holes with a triomino Now, the whole area can be tiled

24

End of Proof 23 x 23 case:

SLIDE 13

13

Strong Induction

25

Inductive Basis: Inductive Hypothesis: Inductive Step: Prove that is true Assume is true Prove that is true

) 1 ( P ) ( ) 2 ( ) 1 ( k P P P     ) 1 (  k P ) (n P

To prove :

26

Theorem: Every integer is a product of primes

(Fundamental Theorem of Arithmetic)

2  n

Proof: Inductive Basis:

2  n

Number 2 is a prime Inductive Hypothesis:

k n   2

Suppose that every integer between and is a product of primes

2 k

(Strong Induction)

SLIDE 14

14

27

Inductive Step:

1   k n

If is prime then the proof is finished

1  k

If is not a prime then it is composite:

1  k b a k   1 k b a   , 2

28

b a k   1 k b a   , 2

By the inductive hypothesis:

i

p p p a 

2 1



j

q q q b 

2 1

 1 ,  j i

j i

q q p p b a k  

1 1

1     primes primes End of Proof

SLIDE 15

15

29

Theorem: Every postage amount can be generated by using 4-cent and 5-cent stamps

12  n

Proof: Inductive Basis:

4 4 4 12     n

We examine four cases

(because of the inductive step)

5 4 4 13     n 4 5 5 14     n 5 5 5 15     n

(Strong Induction)

30

Inductive Hypothesis: Assume that every postage amount between and can be generated by using 4-cent and 5-cent stamps

k n   12 12

k

Inductive Step:

1   k n

If then the inductive step follows directly from inductive basis

14 12   k

5 4     b a n

SLIDE 16

16

Consider:

31

15  k 4 ) 3 ( 1     k k k k    ) 3 ( 12

Inductive hypothesis

5 4 ) 3 (        b a k 5 4 ) 1 ( 4 ) 3 ( 1            b a k k

End of Proof Factorial function

! ) ( n n f 

Recursion

32

Recursion is used to describe functions, sets, algorithms Example:

1 ) (  f ) ( ) 1 ( ) 1 ( n f n n f    

Recursive Step:

Recursive Basis:

SLIDE 17

17

33

factorial( ) { if then return else return }

n 1  n

1

) 1 factorial(n- n

//recursive basis //recursive step

Recursive algorithm for factorial

34

Fibonacci numbers

2 1   



n n n

f f f 1 ,

1

  f f

Recursive Basis: Recursive Step:

 , , , ,

3 2 1

f f f f  , 4 , 3 , 2  n

SLIDE 18

18

35

 13 5 8 8 3 5 5 2 3 3 1 2 2 1 1 1 1 1

5 6 7 4 5 6 3 4 5 2 3 4 1 2 3 1 2 1

                                f f f f f f f f f f f f f f f f f f f f

36

fibonacci( ) { if then return else return }

n

} 1 , {  n

n

) 2 fibonacci( ) 1 fibonacci( n- n- 

//recursive basis //recursive step

Recursive algorithm for Fibonacci function

SLIDE 19

19

37

fibonacci( ) { if then else { for to do { } return }

n

 n

Iterative algorithm for Fibonacci function

 y  x 1  y 1  i 1  n y x z   y x  z y  y

38

Theorem:

2 



n n

f 

2 5 1  

3  n

(golden ratio)

for Proof: Proof by (strong) induction Inductive Basis:

3  n 4  n    2

3

f

2 4

3    f

SLIDE 20

20

We will prove for

39

Inductive Hypothesis:

2 



n n

f  k n   3

Inductive Step:

1   k n

Suppose it holds

) 1 ( 1    k k

f  k  4

40

1 3 2 1 1     

    

k k k k k k

f f f    1

2

  



is the solution to equation

1

2

   x x

3 2 3 3 2 1

) 1 (

    

     

k k k k k

      

End of Proof induction hypothesis

SLIDE 21

21

41

Euclidean Algorithm for Greatest Common Divisor

a a  ) , gcd( ) mod , gcd( ) , gcd( b a b b a 

Recursive Step:

Recursive Basis:

b a 

42

gcd( ) { if then return else return }

b a,

 b

a

) mod gcd(b, b a

//recursive basis //recursive step

Recursive Euclidean algorithm for greatest common divisor

//assume a>b

SLIDE 22

22

43

Algorithm Mergesort 8 2 4 6 9 7 10 1 5 3 8 2 4 6 9 7 10 1 5 3 2 4 6 8 9 1 3 5 7 10 1 2 3 4 5 6 7 8 9 10 split sort sort merge

44

sort( ) { if then { return } else return }

n

a a a , , ,

2 1

 1  n

 

2 / n m 

) , , , sort(

2 1 m

a a a A   ) , , , sort(

1 n m m

a a a B 



 ) , merge( B A

1

a

SLIDE 23

23

45

8 2 4 6 9 7 10 1 5 3 Input values of recursive calls 8 2 4 6 9 7 10 1 5 3 8 2 4 6 9 7 10 1 5 3 4 8 2 6 9 8 2 7 10 1 7 10 5 3

46

1 2 3 4 5 6 7 8 9 10 Input and output values of merging 2 4 6 8 9 1 3 5 7 10 2 4 8 6 9 7 10 1 5 3 4 2 8 6 9 8 2 7 10 1 7 10 5 3

SLIDE 24

24

47

merge( ) { while do { Remove smaller first element of from its list and insert it to } if then { append remaining elements to } return }

  L B A,

//two sorted lists

    B A and B A,

L

    B A

• r

L L

48

2 4 6 8 9 2 4 8 6 9 merging

A B L

Comparison 2 4 8 4 8 8 8 6 9 6 9 6 9 9 9 2 2 4 2 4 6 3 4 6 8 2 4 6 8 9 2<6 4<6 6<8 8<9

SLIDE 25

25

49

The total number of comparisons to merge two lists is at most:

B A,

| | | | s comparison # B A  

Length of A Length of B Merged size

50

Recursive invocation tree

n

a a a , , ,

2 1



2 2 1

, , ,

n

a a a 

n n n

a a a , , ,

2 2 1 2



 

2 1,a

a

4 3,a

a

2 3,   n n

a a

n n

a a ,

1 

1

a

2

a

3

a

4

a

3  n

a

2  n

a

1  n

a

n

a

4 1

, , a a 

n n

a a , ,

3  

4 1

, ,

n

a a 

2 1 4

, ,

n n

a a 

 4 3 1 2

, ,

n n

a a 

 n n

a a , ,

1 4 3





k

n 2 

Assume

 

SLIDE 26

26

51

Elements per list

#levels of tree =

n log 1

k

n 2 

Assume

n n log log

2 1





log

2





n

n

1 log

2 2 /





n

n

) 1 (log log

2 2

 



n n



) 2 (log log

2 4

 



n n

Recursive invocation tree

2 log

2 4 /





n

n

 

n 2 / n 2 / n 4 / n 4 / n 4 / n 4 / n 4 4 2 2 2 2 1 1 1 1 1 1 1 1

52

merging tree

n

a a a , , ,

2 1



2 2 1

, , ,

n

a a a 

n n n

a a a , , ,

2 2 1 2



 

2 1,a

a

4 3,a

a

2 3,   n n

a a

n n

a a ,

1 

1

a

2

a

3

a

4

a

3  n

a

2  n

a

1  n

a

n

a

4 1

, , a a 

n n

a a , ,

3  

4 1

, ,

n

a a 

2 1 4

, ,

n n

a a 

 4 3 1 2

, ,

n n

a a 

 n n

a a , ,

1 4 3





 

SLIDE 27

27

53

merging tree 2 4 2 / n n



 

Elements per list

4 / n

1 n

2 / n 2 / n 4 / n 4 / n 4 / n 4 / n 4 4 2 2 2 2

1 1 1 1 1 1 1 1

n n  4 4 /

54

n n   2 / 2 n n   4 / 4 n n  2 2 / n n  1



Merges per level

Total cost: n n n log 1)

• levels

(#   merging tree

Comparisons per level

 

n 2 / n 2 / n 4 / n 4 / n 4 / n 4 / n 4 4 2 2 2 2 1 1 1 1 1 1 1 1

Elements per merge

SLIDE 28

28

If the number of comparisons is at most

55

k

n 2  n nlog

If the number of comparisons is at most

k

n 2  n n c n n m m log 2 log 2 log   

 

n m

n

2 2 log  

Therefore, worst-case running time of merge sort is

n nlog 

where