ECE 242 Data Structures Lecture 2 Algorithm Analysis September - - PDF document

ECE242 L2: Algorithm Analysis September 11, 2009

ECE 242 Data Structures

Lecture 2

Algorithm Analysis

ECE242 L2: Algorithm Analysis September 11, 2009

Overview °Problem: How can we quantify if one algorithm is better than another?

• Primarily concerned about growth of runtime as input set becomes

larger

• Important concept for programmers

°Complexity analysis is critical to allowing for scalable software °Lewis and Chase text has nice discussion on this topic.

• Note examples of nested loops

ECE242 L2: Algorithm Analysis September 11, 2009

Analysis of Algorithms °How do we compare two implementations of the same interface? °We need to account for all possible inputs. °Our performance measure should be conservative °Need metrics which are independent of target computer, data input set size, other applications.

ECE242 L2: Algorithm Analysis September 11, 2009

Algorithm Running Time How do we characterize the running time of a method?

public boolean contains (int x) { int i = 0; while ((i < data.length) && (data[i] != x)) { i++; } return (i != data.length); }

• Count number of time units utilized
• Parameter is the size of the collection.

ECE242 L2: Algorithm Analysis September 11, 2009

Big-O Notation °“ “ “ “O” stands for “the order of” °Also called asymptotic analysis °O(N): linear to N

• E.g. N and N/2

°O(logN): order of log N

• E.g. log2N and log3N

°O(N) > O(logN)

ECE242 L2: Algorithm Analysis September 11, 2009

Defining Running Time We can parameterize the running time of a method by its input.

public int sum(int n) { int sum = 0; for (int i = 1; i <= n; i++) sum += i; return sum; } public int sum(int n) { return (int) n*(n+1)/2; } n steps O(n) 3 steps O(1)

ECE242 L2: Algorithm Analysis September 11, 2009

Defining Running Time We can parameterize the running time of a method by its input.

public int sum(int n) { int sum = 0; for (int i = 1; i <= n; i++) sum += i; return sum; } public int sum(int n) { return (int) n*(n+1)/2; } n operations 3 operations

ECE242 L2: Algorithm Analysis September 11, 2009

Worst-case Analysis

°Which implementation is better? °Any single measure of running time should

describe performance over all possible inputs.

°Worst-case reasoning is conservative

• Can serve as a baseline measure for comparing

implementations.

°Also can consider average case and best case

ECE242 L2: Algorithm Analysis September 11, 2009

Growth Functions

° It's not usually necessary to know the exact growth function ° The key issue is the asymptotic complexity of the function – how it grows as n increases ° Determined by the dominant term in the growth function ° This is referred to as the order of the algorithm ° We often use Big-O notation to specify the order, such as O(n2)

ECE242 L2: Algorithm Analysis September 11, 2009

Growth Functions

° Analysis is defined in general terms, based on:

• the problem size (ex: number of items to sort)
• key operation (ex: comparison of two values)

° A growth function shows the relationship between the size of the problem (n) and the time it takes to solve the problem t(n) = 15n2 + 45 n

Growth here is dominated by the n2 term for addition and subtraction O(n2)

ECE242 L2: Algorithm Analysis September 11, 2009

Some growth functions and their asymptotic complexity Locate worst growth factor in each equation Note some equation can have multiplicative terms (e.g. n* logn)

ECE242 L2: Algorithm Analysis September 11, 2009

Analyzing Loop Execution °A loop executes a certain number of times (say n) °Thus the complexity of a loop is n times the complexity of the body of the loop °When loops are nested, the body of the outer loop includes the complexity of the inner loop °The following loop is O(n) because the loop executes n times and the body of the loop is O(1):

for (int i=0; i<n; i++) { x = x + 1; }

ECE242 L2: Algorithm Analysis September 11, 2009

Analyzing Loop Execution

° The following loop is O(n2) because the loop executes n times and the body of the loop, including a nested loop, is O(n):

for (int i=0; i<n; i++) { x = x + 1; for (int j=0; j<n; j++) { y = y - 1; } }

ECE242 L2: Algorithm Analysis September 11, 2009

Asymptotic Notation ° Method A is 10n² - 5 milliseconds to process n elements. ° Method B is 100n + 200 milliseconds.

ECE242 L2: Algorithm Analysis September 11, 2009

Big-O notation °O(1): constant

• E.g. initialize i

°O(N) > O(logN) > O(1) °Ignore less dominating term. °Ignore constant

• E.g. N/2+1 is O(N)

°O(N): < C·N linear time °O(logN): < C ·log2N time

ECE242 L2: Algorithm Analysis September 11, 2009

O(1) Example

i=0; j=0; k=0;

°Number of statements

• Constant

°Cost

• O(1)

Algorithms which take time which is independent of the input size are O(1)

ECE242 L2: Algorithm Analysis September 11, 2009

O (N) Example

int sum = 0; for (int i=0; i<N;i++) sum = sum + a[i];

°Number of statements:

• C·N+1

°Cost

• O(N)

Most loops are O(n), where n is the index limit

ECE242 L2: Algorithm Analysis September 11, 2009

O (N2) Example

int sum = 0; for (int i=0; i<N; i++) for (int j=0; j<N; j++) sum = sum + a[i][j];

°Number of statements

• Inner loop: C·N

°Cost

• N · C·N = O(N2)

Nested loops – note that constants are ignored

ECE242 L2: Algorithm Analysis September 11, 2009

O (logN) Example

int sum = 0; for (i=N;i>0;i=i/2) sum = sum + a[i]; °Cost

• O(logN)

Multiplier/divider in modifier

°log23·log3N = log2N °log23 is a constant °log2N, log3N, log4N are all O(logN)

ECE242 L2: Algorithm Analysis September 11, 2009

Another O (N2) Example int sum = 0; for (int i=0; i<N; i++) for (int j=0; j<i; j++) sum = sum + a[i][j]; °Number of statements in inner loop °Cost

• Summing all “i”s:

0+1+2+3+…+(N-1)=N(N-1)/2=1/2N2-1/2N

• O(N2)

i 1 2 3 …

# of statements in inner loop

1 2 3 …

ECE242 L2: Algorithm Analysis September 11, 2009

O (NlogN) Example

int sum = 0; for (i=0;i<N;i=++) for (j = N; j>0; j=j/2) sum = sum + a[i][j]; °Number of statements in inner loop:

log2N+1

°Total: N(log2N+1) °Cost

• O(NlogN)

ECE242 L2: Algorithm Analysis September 11, 2009

More Examples: O(N)

sum = 0; for (i=1; i<=N; i++) { sum = sum + i ; for( j=1; j<10000; j++) sum = sum + i*j ; }

°Number of statements:

• C·N+1

°Cost

• O(N)

ECE242 L2: Algorithm Analysis September 11, 2009

More Examples: O(N2)

sum1 = 0; sum2 = 0; for (i=1; i<=N; i++) { sum1 = sum1 + i ; for(int j=20*N; j<200*N; j++) sum2 = sum2 + i; }

°Number of statements:

• C1 · N + C2 · N2 + 1

°Cost

• O(N2)

ECE242 L2: Algorithm Analysis September 11, 2009

More Examples: O (N3)

int sum = 0; for (int i=0; i<N; i++) for (int j=0; j<N; j++) for (int k=0; k<j; k++) sum = sum + a[i][j][k];

°Cost for loop j and loop k

• O(N2), we talked about this before

°So, finally cost is

• N * O(N2) = O(N3)

ECE242 L2: Algorithm Analysis September 11, 2009

More Examples: O (logN)

int sum = 0; for (int i=1; i<=N; i=i*5) sum = sum + a[i]; °Cost

• O(logN)

Note multiplication in the loop modifier

ECE242 L2: Algorithm Analysis September 11, 2009

More Examples: Multiple Inputs

int sum = 0; for (int i=0;i<M;i++) for (int j = N; j>0; j=j/10) sum = sum + a[i]+b[j]; °Number of statements in inner loop:

log10N+1

°Total: M(log10N+1) °Cost

• O(MlogN)

ECE242 L2: Algorithm Analysis September 11, 2009

Asymptotic Notation ° (n):

• This notation indicates a “tight” bound on the complexity of a

function

• For a function f, f is an element of O(g) if an only if, for some

constant c > 0, there is a constant p >= 0 such that f(n) < cg(n) for any n >= p.

• 3n² and 0.2n² are both in (n²) and O(n²)
• O(n) indicates an “upper” bound

° A function's order is not changed by adding or subtracting a function in a lower order.

2 – n + log n is in (2)

ECE242 L2: Algorithm Analysis September 11, 2009

Asymptotic Notation ° Factorial takes time which is at least in O(2). ° Therefore we can prove that n! is in (2).

N! = 1*2*3*4…..*N 2 = 1*2*2*2*….. We won’t be using notation very much in this course

ECE242 L2: Algorithm Analysis September 11, 2009

Summary °Complexity analysis mostly involves examining loops °Attempt to characterize growth of functions.

• This will be important when we examine searching and sorting

°Big O notation helps us simplify complexity definitions – worst case

• Ignore constant and lower order terms
• Big Omega () examines lower bounds
• Big Theta () examines specific case

°Computer scientists often use proofs to defend complexity claims