Feedback Difficulties: Complexity analysis Big O and Big Theta - - PowerPoint PPT Presentation

▶
feedback
SMART_READER_LITE
LIVE PREVIEW

Feedback Difficulties: Complexity analysis Big O and Big Theta - - PowerPoint PPT Presentation

Dec 01, 2022 155 likes •449 views

ICC Module Computation Lesson 2 Computation & Algorithms II Feedback Difficulties: Complexity analysis Big O and Big Theta notation 50% of students did 75-100% 37% of students did 50-75% 13% of students did <50% 1 ICC

slide-1
SLIDE 1

ICC Module Computation Lesson 2 – Computation & Algorithms II

1

§ Difficulties:

  • Complexity analysis
  • Big O and Big Theta notation

Feedback

50% of students did 75-100% 37% of students did 50-75% 13% of students did <50%

slide-2
SLIDE 2

ICC Module Computation Lesson 2 – Computation & Algorithms II

2

Information, Computation, and Communication

Algorithm 2

slide-3
SLIDE 3

ICC Module Computation Lesson 2 – Computation & Algorithms II

3

§ Recursion § Complexity of a recursive algorithm § Dynamic Programming § (Binary Search « Recherche dichotomique »)

Topics

slide-4
SLIDE 4

Understanding a Recurive Algorithm

A. 5 B. 8 C. 24 D. 32

What is the output if L={5,8,6,10,3}?

slide-5
SLIDE 5

ICC Module Computation Lesson 2 – Computation & Algorithms II

5

Complexity

Algo1({5,8,6,10,3}) Algo1({8,6,10,3}) Algo1({6,10,3})

Height? Cost? n+1

Algo1({10,3}) Algo1({3}) Algo1({})

3

  • -”--
  • -”--
  • -”--
  • -”--

∼ 10 T(n)=10・n + 3 = Θ(n)

slide-6
SLIDE 6

ICC Module Computation Lesson 2 – Computation & Algorithms II

6

§ Find the largest number in a list

  • Input: a list L with n numbers
  • Output: the largest number in this list

§ Idea of recursion:

  • Use a solution of a smaller problem (a shorter list)

§ You need to answer two questions:

  • 1. Assume we are given the largest element in the list

L[2:n] (list of length n-1), how would we compute the largest element of the list L[1:n]?

  • 2. What is the termination condition? What is the

largest element of a list of size 1?

Writing a Recursive Algorithm

slide-7
SLIDE 7

ICC Module Computation Lesson 2 – Computation & Algorithms II

7

Question 1: from n-1 to n

slide-8
SLIDE 8

ICC Module Computation Lesson 2 – Computation & Algorithms II

8

Question 2: Termination

slide-9
SLIDE 9

Find Max (Recursively)

A. Θ(log(n)) B. Θ(n) C. Θ(n2) D. Θ(2n)

What is the complexity of this algorithm?

slide-10
SLIDE 10

ICC Module Computation Lesson 2 – Computation & Algorithms II

10

Complexity

rec_max({5,8,6,10,3}) rec_max({8,6,10,3}) rec_max({6,10,3})

Height? Cost? n+1

rec_max({10,3}) Rec_max({3})

3

  • -”--
  • -”--
  • -”--
  • -”--

∼ 8 T(n)= 8・n + 3 = Θ(n)

slide-11
SLIDE 11

ICC Module Computation Lesson 2 – Computation & Algorithms II

11

§ The recursive solution is not always the only solution and rarely the most efficient... § ...but it is sometimes much simpler and more practical to implement ! § Examples : sorting, processing of recursive data structures (e.g. trees, graphs, ...), ...

Recursion or Not?

slide-12
SLIDE 12

ICC Module Computation Lesson 2 – Computation & Algorithms II

12

Fibonacci Numbers (Recursive Version)

F(n) = F(n-1) + F(n-2) for n>1 F(0) = 0 et F(1) = 1

Month 1: 1 Month 2: 1 Month 3: 2 Month 4: 3 Month 5: 5 Month 6: 8

Named after Leonardo Fibonacci (1175-1250)

slide-13
SLIDE 13

ICC Module Computation Lesson 2 – Computation & Algorithms II

13

Fibonacci Numbers in Nature

In these pictures there are 55 curves of seeds spiraling to the left as you go outwards and 34 spirals of seeds spiraling to the right. A little further towards the center you can count 34 spirals to the left and 21 spirals to the right. These pairs of numbers are (almost always) neighbors in the Fibonacci series.

The lengths of the squares describing naturally appear spirals are often Fibonacci numbers.

1 1 2 3 5 8 13

https://plus.maths.org/content/life-and-numbers-fibonacci Number of ancestors

  • f a drone (a male

honey bee) is a Fibonacci number.

slide-14
SLIDE 14

ICC Module Computation Lesson 2 – Computation & Algorithms II

14

Fibonacci Recursive

F(n) = F(n-1) + F(n-2) for n>1 F(0) = 0 et F(1) = 1

F(5) = F(4)+F(3) F(4) = F(3)+F(2) F(3) = F(2)+F(1) F(3) = F(2)+F(1) F(2) = F(1)+F(0) F(2) = F(1)+F(0) F(1) F(2) = F(1)+F(0) F(1) F(1) F(0) F(1) F(0) F(1) F(0)

Height? Cost? 1 2 4 6 2 In each box in this tree we have to performance a constant number of instructions.

  • 1. How many boxes (function calls) are in this tree?
  • 2. How high is the tree?

The height is n

> "

!"# $%& ' 2! = 2 $(& ' − 1 = Ω(2 $ ')

< "

!"# $

2! = 2$%& − 1 = 𝑃(2$)

slide-15
SLIDE 15

ICC Module Computation Lesson 2 – Computation & Algorithms II

15

§ Complexity F(n)? Exponential in n § Is there a better solution? Idea: do not perform the same calculation many times but store the already calculated value for later use

Fibonacci Recursive

slide-16
SLIDE 16

ICC Module Computation Lesson 2 – Computation & Algorithms II

16

§ Dynamic programming is strategy to solve problems.

  • It applies to problems for which we can find an optimal

solution by breaking it down into smaller optimal

  • verlapping sub-problems in a recursive manner.
  • To solve such problems efficiently we memorize the

solutions of sub-problems to avoid re-computation.

§ If the sub-problems are not overlapping the solving strategy is called ”divide and conquer” (like in binary search « Recherche dichotomique »).

Dynamic Programming

slide-17
SLIDE 17

ICC Module Computation Lesson 2 – Computation & Algorithms II

17

Memoize the last two computations (x and y)

Dynamic Programming - Fibonacci

slide-18
SLIDE 18

Complexity?

  • A. Θ(log n)
  • B. Θ(n)
  • C. Θ(n2)
  • D. Θ(2n)
slide-19
SLIDE 19

ICC Module Computation Lesson 2 – Computation & Algorithms II

19

Computation of the shortest path, for example between all the train stations of the CFF network. Given a graph G with vertices 1,2,..,N, consider a function called Dk-1(i,j) that returns the shortest path from i to j using only vertices from 1 to k-1 as intermediate points.

Dynamic Programming – Floyd

Key Idea of the algorithm: The shortest path to go from i to j (e.g., Lausanne to Zürich) is the shortest among:

  • 1. The shortest known path from Lausanne to

Zürich,

  • 2. The path from Lausanne to Zürich by

traversing a city not known yet (e.g., Bern) Dk (i, j) = min {Dk −1(i, j), Dk −1(i, k) + Dk −1(k , j)}

D (k,j)

k-1

Dk-1(i,j) D (i,k)

k-1

Zürich j Biel k-1 i Lausanne k Bern

slide-20
SLIDE 20

ICC Module Computation Lesson 2 – Computation & Algorithms II

20

Dynamic Programming – Floyd

slide-21
SLIDE 21

ICC Module Computation Lesson 2 – Computation & Algorithms II

21

Floyd’s Algorithm: Example

Lausanne Biel Bern Zürich 133 58 ∞ 133 45 38 58 45 102 ∞ 38 102

Zürich 3 1 Lausanne

(fictitious data)

4 Bern 102 58 45 38 133 Biel 2

D(i,j) Lausanne Biel Bern Zürich

Going throw (1) Lausanne does not give any improvements. Going throw (2) Biel improves the distances

  • from Lausanne to Zürich and
  • from Zürich to Bern
slide-22
SLIDE 22

ICC Module Computation Lesson 2 – Computation & Algorithms II

22

Floyd’s Algorithm: Example

D0 = D 1 Lausanne Biel Bern Zürich 133 58 ∞ 133 45 38 58 45 102 ∞ 38 102 D2 =

Zürich 3 1 Lausanne

(fictitious data)

4 Bern 102 58 45 38 133

Lausanne Biel Bern Zürich 133 0 58 45 171 38 83

Going throw (1) Lausanne does not give any improvements. Going throw (2) Biel improves the distances

  • from Lausanne to Zürich and
  • from Zürich to Bern

Biel 2

slide-23
SLIDE 23

ICC Module Computation Lesson 2 – Computation & Algorithms II

23

Lausanne Biel Bern Zürich 133 0 58 45 171 38 83

Floyd’s Algorithm: Example

D3 =

Zürich 3 1 Lausanne

(fictitious data)

4 Bern 102 58 45 38 133

Lausanne Biel Bern Zürich 103 0 58 45 141 38 83

Going throw (3) Bern improves the distances

  • from Lausanne to Biel and
  • from Zürich to Lausanne

Going throw (4) Zürich does not give any improvements

Biel 2

D2 =

Note: it also works for asymmetric graphs (directed graphs)

slide-24
SLIDE 24

Complexity?

A. Θ(n2) B. Θ(n3) C. Θ(n4) D. Θ(n5)

slide-25
SLIDE 25

ICC Module Computation Lesson 2 – Computation & Algorithms II

25

§ There is no miracle recipe to find an algorithm, but there exist big families of resolution strategies:

  • decompose (e.g., “recursion”): try to solve the

problem by decomposing it in simpler (or smaller) instances

  • decompose and regroup (e.g., “dynamic

programming”): memorize intermediate computations to avoid executing them several times

Conclusion

slide-26
SLIDE 26

ICC Module Computation Lesson 2 – Computation & Algorithms II

26

Questions ?

slide-27
SLIDE 27

ICC Module Computation Lesson 2 – Computation & Algorithms II

27

§ About 300 copies sorted by SCIPER number § How many copies do you have to look at (in the worst case) in order to find your copy? Binary Search: Inspection of Exam Results « Recherche dichotomique »

slide-28
SLIDE 28

ICC Module Computation Lesson 2 – Computation & Algorithms II

28

§ Roughly 9 copies:

1. Split the pile into two piles of about half the size (~150 copies) 2. Check the SCIPER number of the copy on the top of the second pile: a) if it is your copy, you are done b) if it is not your copy and if your SCIPER number is larger than the one of the copy continue your search in the second pile by going to Step 1, c)

  • therwise continue the search in the first pile by going to

Step 1

§

  • Approx. sizes of search piles:

300, 150, 125, 63, 32, 16, 8, 4, 2, 1

Binary Search: Inspection of Exam Results « Recherche dichotomique »