Example The sorting problems is defined as follows: n Input set: - - PDF document

Complexity Advanced Programming

Algorithms and Complexity

2

Algorithm

An algorithm is a calculation procedure (composed by a finite number of steps) which solves a certain problem, working on a set of input values and producing a set of output values

Complexity Advanced Programming

3

Example

The sorting problems is defined as follows:

n Input set: sequence of numbers <a1, a2,…,an> n Output set: permutation <a’ 1, a’ 2, …, a’ n> of

input sequence so that a’ 1 ≤ a’ 2 ≤ …≤ a’ n.

4

Insertion Sort

Insert A[ j ] in the sorted sequence A[1.. j --1]

Complexity Advanced Programming

5

Esempio

6

Analysis of algorithms

§ Analysing algorithms means to predict the amount of resources (I/O, memory, time) required by an algorithm during its execution.

§ Such analysis should be independent from the

kind of hardware platform on which the algorithm is executed.

§ We will assume in the rest of the course that the

hardware platform is a common machine with a single CPU (Random Access Machine or RAM).

Complexity Advanced Programming

7

Problem Dimension

Analysis of algorithms is usually performed with respect to one or more parameters which characterize the dimensions of the problem. Example

n In the sorting algorithm such parameter is the

number of elements of the input sequence.

n In the multiplication between integer numbers,

dimension is given by the number of bits used to represent the operands.

8

Hypothesis

n Each statement in the pseudo-code requires a

fixed time

n Each statement has a different execution time.

Complexity Advanced Programming

9

Analysis of insertion sort

times

tj is the number of times the statement is repeated, for a certain value of j

Insert A[ j ] in the ordered sequence A[1 .. j -1 ]

10

Execution time of insertion sort

Complexity Advanced Programming

11

Best Case

If the array is already sorted, the tj = 1 for every j. Then: Which can be written as follows T(n) = an + b

12

Worst Case

If an array is sorted with inverse order at step j, j comparisons and swaps: tj = j. Remember that:

Complexity Advanced Programming

13

Worst Case (2)

Therefore In the worst case: T(n) = an2+bn+c

14

Average Case

Worst case analysis is important as it defines an upper bound to resources required by an algorithm. In some cases the average case (or mean value) is worthy of being analyzed.

Complexity Advanced Programming

15

Importance of complexity analysis

Analysis of resources of an algorithm (also known as complexity analysis) allows describing performance of the algorithm depending on problem dimension. Choosing an algorithm with lower complexity is the best choice independently from the used technology and it can make a difference for problems with big dimensions.

16

Example

Suppose you can get the solution of a problem with 2 algorithms:

n One with complexity T(n) = 2n2 n The other with complexity T(n) = 50n log2 n

Suppose you have these two machines:

n Intel Core i7 Extreme 965EE performing 76000 MIPS at 3.2 GHz

for the first algorithm

n AMD Athlon FX-60 performing 18000 MIPS at 2.6 GHz for the

second algorithm.

n MIPS = Millions of Instructions Per Second

Complexity Advanced Programming

17

Example (II)

Execution time is proportional to T(n) / MIPS:

n When n = 1 K ( 103)

Intel - T(n)/76000 = 2*106 / 76000 = 0.026 ms

AMD – T(n)/18000 = 50*103*log2 (103) / 18000 =0.027 ms

n When n = 1 M ( 106)

Intel – 76000 MIPS : 26310 ms

AMD – 18000 MIPS : 55,36 ms

n When n = 1 G ( 109)

Intel – 76000 MIPS : 26,3 Ms = 7305 hours ! = 304 days !!!

AMD – 18000 MIPS : 83,04 s

18

Asymptotic Notation

In complexity analysis the asymptotic notation is

• ften used to discover the complexity of an

algorithm when problem dimension increases. The asymptotic notation is based on 3 notations:

n Theta Notation Θ n “Big O” Notation O n Omega Notation Ω.

Complexity Advanced Programming

19

Θ Notation

Given an algorithm of complexity T(n). T(n) = Θ( g(n) ) if and only if there exist three positive constants c1, c2 e n0 so that 0≤ c1g(n) ≤ T(n) ≤ c2g(n) for each n≥n0. In this case g(n) is the asymptotic limit for T(n).

20

“Big O” Notation

T(n) = O( g(n) ) if and only if there exist two positive constants c and n0 so that: 0≤ T(n) ≤ c g(n) for each n≥n0. In this case g(n) is a upper asymptotic limit for T(n).

Complexity Advanced Programming

21

Ω Notation

T(n) = Ω( g(n) ) if and only if there exist two positive constants c and n0 so that 0≤c g(n) ≤T(n) for each n≥n0. In this case g(n) is a lower asymptotic limit for T(n).

22

Complexity Advanced Programming

23

Theorem

Given two functions g(n) and T(n), T(n) = Θ( g(n) ) if and only if

n T(n) = O( g(n) ) and T(n) = Ω( g(n) )

Problems Complexity

24

Complexity Advanced Programming

25

Empirical complexity measure

Using the clock() function:

#include <time.h> int main(){ clock_t b= clock(); test_function(); clock_t e = clock(); double elapsed=(double)(e – b)/CLOCKS_PER_SEC; printf("Elapsed: %f seconds\n",elapsed); return 0; }

Example

26

strcpy(destination, source); mycpy(destination, source); Vs.

Complexity Advanced Programming

Example

27

strcpy(destination, source); mycpy(destination, source); Vs. void mycpy(char* d,char* s){ char* src; for(src=s;*src!='\0';++src){ char* org=s; char* dst=d; for(; origin<=src; ++org,++dst){ *dst = *origin; } } }; O(n2)

Measurement code

28

int main(){ int i,j; printf("Function,n,Clocks\n"); int n[10]={100,200,300,/*…*/,2000,2500,3000}; for(j=1;j<10;++j){ for(i=0; i<N_MEASURE;++i){ clock_t b= clock(); test_strcpy(n[j]); clock_t e = clock(); printf("strcpy(),%d,%d\n",n[j],(int)(e-b)); }} /*…*/ }

Complexity Advanced Programming

Results

29

y = 0.1498x R² = 0.95331 y = 1.5644x2 - 47.556x R² = 0.99922 2000000 4000000 6000000 8000000 10000000 12000000 14000000 16000000 500 1000 1500 2000 2500 3000 strcpy() mycpy()

14 s 463 μs