Algorithms Programming for Engineers Winter 2015 Andreas Zeller, - - PowerPoint PPT Presentation

▶
algorithms
SMART_READER_LITE
LIVE PREVIEW

Algorithms Programming for Engineers Winter 2015 Andreas Zeller, - - PowerPoint PPT Presentation

Dec 09, 2023 44 likes •1.32k views

Algorithms Programming for Engineers Winter 2015 Andreas Zeller, Saarland University Todays Topics Algorithms Search and Sort Complexity Sounds! An Algorithm unambiguous instruction to solve a problem consists

slide-1
SLIDE 1

Programming for Engineers
 Winter 2015 Andreas Zeller, Saarland University

Algorithms

slide-2
SLIDE 2
slide-3
SLIDE 3
slide-4
SLIDE 4

Today’s Topics

  • Algorithms
  • Search and Sort
  • Complexity
  • Sounds!
slide-5
SLIDE 5

An Algorithm

  • unambiguous instruction


to solve a problem

  • consists of fjnitely many, well defjned

steps

  • typically implemented as computer

programs

slide-6
SLIDE 6

The First Algorithm

  • computes the GCD
  • Biggest integer by which AB and CD can

be divided

“However, if CD does not measure AB, and if one repeatedly and alternately deducts the smaller from the larger (of AB, CD), then fjnally a number must remain which measures the previous one.”
 (Euclid, Elements)

slide-7
SLIDE 7

Euclidean Algorithm

“However, if CD does not measure AB, and if one repeatedly and alternately deducts the smaller from the larger (of AB, CD), then fjnally a number must remain which measures the previous one.”
 (Euclid, Elements)

Example: The GCD of 12 and 44 is 4

slide-8
SLIDE 8

Euclidean Algorithm

int gcd(int a, int b) { if (a == 0) return b; while (b != 0) { if (a > b) a = a - b; else b = b - a; } return a; }

Example: The GCD of 12 and 44 is 4

slide-9
SLIDE 9

يمزراوخلا ىسوم نب دمحم رفعج وبأ

Muhammad ibn Musa al-Chwarizmi, * ~780 • †835–850

slide-10
SLIDE 10

رصتخلنا باتكلا ربجلا باسح يف ةلباقلناو

The concise Book about calculus

slide-11
SLIDE 11

The First Algorithm Developed for Computers

  • Computes Bernoulli numbers

  • Sequence of rational numbers that occur

in mathematics in many contexts

  • Example: Sum of powers

B0 = 1, B1 = ±1⁄2, B2 = 1⁄6, B3 = 0, B4 = −1⁄30, B5 = 0, B6 = 1⁄42, B7 = 0, B8 = −1⁄30.

slide-12
SLIDE 12

Ada Lovelace


1815–1852

slide-13
SLIDE 13

Analytical Engine


Charles Babbage, 1837

slide-14
SLIDE 14

Alan Turing

1912–1954

slide-15
SLIDE 15

Turing Machine

slide-16
SLIDE 16

Halting Problem

  • Not all problems can be solved by programs
  • E.g. the halting problem states that there is

no program which can decide for an arbitrary program P, whether it will (eventually) return a result or not.

slide-17
SLIDE 17

Collatz Conjecture


(Wolfgang Collatz, 1937)

  • Start with an integer n
  • If n is even, take n/2 next
  • If n is odd, take 3n+1next
  • repeat

19, 58, 29, 88, 44, 22, 11, 34, 17, 52, 26, 13, 40, 20, 10, 5, 16, 8, 4, 2, 1, …

slide-18
SLIDE 18

Collatz Conjecture


(Wolfgang Collatz, 1937)

  • Apparently every sequence defjned in this

manner ends in 4, 2, 1, …

  • This property remains unproven

19, 58, 29, 88, 44, 22, 11, 34, 17, 52, 26, 13, 40, 20, 10, 5, 16, 8, 4, 2, 1, …

slide-19
SLIDE 19

19, 58, 29, 88, 44, 22, 11, 34, 17, 52, 26, 13, 40, 20, 10, 5, 16, 8, 4, 2, 1, …

Halting Problem

  • Will collatz() return

for every n?

  • Solution only by trial


(in infjnite time)

void collatz(int n) { while (n != 1) { if (n % 2 == 0) n = n / 2; else n = 3 * n + 1; } }

It is impossible to show correctness automatically for all programs

slide-20
SLIDE 20

Halting Problem

To show that a real program fulfjls its requirements, we must either

  • use mathematical knowledge and

assumptions to prove it by hand (which is very hard), or

  • we must test it and hope that our tests

suffjce.

slide-21
SLIDE 21

Algorithms

Calculate Search Sort

  • Fibonacci
  • GCD
  • Collatz
  • Linear
  • Binary
  • Insertion
  • Merge
slide-22
SLIDE 22

Algorithms

Calculate Search Sort

  • Fibonacci
  • GCD
  • Collatz
  • Linear
  • Binary
  • Insertion
  • Merge
slide-23
SLIDE 23

Arrays

1 2 3 4 5 6 7 8 9 10

int a[11]; // 11 Elements

a

Type of
 the elements Name


  • f the

array Size


  • f the

array

slide-24
SLIDE 24

Search

1 2 3 4 5 6 7 8 9 10 a

// If x is in a[0..size], return index,
 // such that x == a[index]; // otherwise < 0 int find(int a[], int size, int x) { // what happens here? }

10 –10 7 2 2 –4 –7 –10 1 4 5

slide-25
SLIDE 25

Demo

slide-26
SLIDE 26

Search

// If x is in a[0..size], return index,
 // such that x == a[index]; // otherwise < 0

int find(int a[], int size, int x) { for (int i = 0; i < size; i++) { if (x == a[i]) return i; } return -1; }

slide-27
SLIDE 27

Search

1 2 3 4 5 6 7 8 9 10 a

int index = find(a, 11, -4);

10 –10 7 2 2 –4 –7 –10 1 4 5

i

slide-28
SLIDE 28

Search

1 2 3 4 5 6 7 8 9 10 a

int index = find(a, 11, -4);

10 –10 7 2 2 –4 –7 –10 1 4 5

i

slide-29
SLIDE 29

Search

1 2 3 4 5 6 7 8 9 10 a

int index = find(a, 11, -4);

10 –10 7 2 2 –4 –7 –10 1 4 5

i

slide-30
SLIDE 30

Search

1 2 3 4 5 6 7 8 9 10 a

int index = find(a, 11, -4);

10 –10 7 2 2 –4 –7 –10 1 4 5

i

slide-31
SLIDE 31

Search

1 2 3 4 5 6 7 8 9 10 a

int index = find(a, 11, -4);

10 –10 7 2 2 –4 –7 –10 1 4 5

i

slide-32
SLIDE 32

Search

1 2 3 4 5 6 7 8 9 10 a

int index = find(a, 11, -4);

10 –10 7 2 2 –4 –7 –10 1 4 5

i ✔

slide-33
SLIDE 33

// If x is in a[0..size], return index,
 // such that x == a[index]; // otherwise < 0

int find(int a[], int size, int x) { for (int i = 0; i < size; i++) { if (x == a[i]) return i; } return -1; }

Complexity

  • How many comparisons does fjnd() need?
slide-34
SLIDE 34

Complexity

Having n elements:

  • Linear search: n/2 comparisons
  • What to do when we have millions of data

points?

slide-35
SLIDE 35

Search

a

10 –10 7 2 2 –4 –7 –10 1 4 5

slide-36
SLIDE 36

Search

a

10 –10 7 2 2 –4 –7 –10 1 4 5

// If x is in a[0..size], return index,
 // such that x == a[index]; // otherwise < 0

int sorted_find(int a[], int size, int x) { // Would this be faster? }

slide-37
SLIDE 37

Search

a

10 –10 7 2 2 –4 –7 –10 1 4 5

i

int index = sorted_find(a, 11, -4);

slide-38
SLIDE 38

Search

a

10 –10 7 2 2 –4 –7 –10 1 4 5

i

int index = sorted_find(a, 11, -4);

slide-39
SLIDE 39

Search

a

10 –10 7 2 2 –4 –7 –10 1 4 5

i

int index = sorted_find(a, 11, -4);

✔

slide-40
SLIDE 40

The Plan

  • Remember the interval in which to search
  • Look at the element at the center of the

interval

  • If it is larger than the sought after element,

continue the search in the left half

  • Otherwise continue the search in the right

half

slide-41
SLIDE 41

Search

a

10 –10 7 2 2 –4 –7 –10 1 4 5

left

int index = sorted_find(a, 11, -4);

right mid

slide-42
SLIDE 42

Search

a

10 –10 7 2 2 –4 –7 –10 1 4 5

left

int index = sorted_find(a, 11, -4);

right mid

slide-43
SLIDE 43

Search

a

10 –10 7 2 2 –4 –7 –10 1 4 5

left

int index = sorted_find(a, 11, -4);

right mid

slide-44
SLIDE 44

Search

a

10 –10 7 2 2 –4 –7 –10 1 4 5

left

int index = sorted_find(a, 11, -4);

right mid

slide-45
SLIDE 45

Search

a

10 –10 7 2 2 –4 –7 –10 1 4 5

left

int index = sorted_find(a, 11, -4);

right mid

✔

  • Wie viele Vergleiche braucht sorted_fjnd()?
slide-46
SLIDE 46

Demo

slide-47
SLIDE 47

Binary Search

int sorted_find(int a[], int size, int x) { int left = 0; int right = size - 1; while (left <= right) { int mid = left + ((right - left) / 2); if (x == a[mid]) return mid; else if (x < a[mid]) right = mid - 1; else // (x > a[mid]) left = mid + 1; } return -1; }

slide-48
SLIDE 48

Complexity Compared

Having n elements:

  • Linear search: n/2 comparisons
  • Binary Search: log2 n comparisons
slide-49
SLIDE 49

Server Farm

slide-50
SLIDE 50

Algorithms

Calculate Search Sort

  • Fibonacci
  • GCD
  • Collatz
  • Linear
  • Binary
  • Insertion
  • Merge
slide-51
SLIDE 51

Algorithms

Calculate Search Sort

  • Fibonacci
  • GCD
  • Collatz
  • Linear
  • Binary
  • Insertion
  • Merge
slide-52
SLIDE 52

Sort

a

10 –10 7 2 2 –4 –7 –10 1 4 5

slide-53
SLIDE 53

Sort

a

10 –10 7 2 2 –4 –7 –10 1 4 5

How do I sort an array?

slide-54
SLIDE 54

Insertion Sort

a

10 –10 7 2 2 –4 –7 –10 1 4 5

slide-55
SLIDE 55

Insertion Sort

a

10 –10 7 2 2 –4 –7 –10 1 4 5

slide-56
SLIDE 56

Insertion Sort

a

10 –10 7 2 2 –4 –7 –10 1 4 5

slide-57
SLIDE 57

Insertion Sort

a

10 –10 7 2 2 –4 –7 –10 1 4 5

slide-58
SLIDE 58

Insertion Sort

a

10 –10 7 2 2 –4 –7 –10 1 4 5

slide-59
SLIDE 59

Insertion Sort

a

10 –10 7 2 2 –4 –7 –10 1 5 4

slide-60
SLIDE 60

Insertion Sort

a

10 –10 7 2 2 –4 –7 –10 1 5 4

slide-61
SLIDE 61

Insertion Sort

a

10 –10 7 2 2 –4 –7 –10 1 5 4

slide-62
SLIDE 62

Insertion Sort

a

10 –10 7 2 2 –4 –7 –10 1 5 4

slide-63
SLIDE 63

Insertion Sort

a

10 –10 7 2 2 –4 –7 –10 1 5 4

slide-64
SLIDE 64

Insertion Sort

a

10 –10 7 2 2 –4 –7 –10 1 5 4

slide-65
SLIDE 65

Insertion Sort

a

10 –10 7 2 2 –4 –7 –10 1 5 4

slide-66
SLIDE 66

Insertion Sort

a

10 –10 7 2 2 –4 –7 –10 1 5 4

slide-67
SLIDE 67

Insertion Sort

a

10 –10 7 2 2 –4 –7 –10 1 5 4

slide-68
SLIDE 68

Insertion Sort

a

10 –10 7 2 2 –4 –7 –10 1 5 4

slide-69
SLIDE 69

Sort In-Place

a

10 –10 7 2 2 –4 –7 –10 1 4 5

  • We want to sort inside of an array
  • Assume: array a[0…i–1] is already sorted
  • We inspect the element a[i]…
  • …and insert it into the sorted array
slide-70
SLIDE 70

Sort In-Place

a

10 –10 7 2 2 –4 –7 –10 1 4 5

already
 sorted

slide-71
SLIDE 71

Sort In-Place

a

10 –10 7 2 2 –4 –7 –10 1 4 5

already
 sorted

slide-72
SLIDE 72

Sort In-Place

a

10 –10 7 2 2 –4 –7 –10 1 4 5

already
 sorted

slide-73
SLIDE 73

Sort In-Place

a

10 –10 7 2 2 –4 –7 –10 1 4 5

already
 sorted

slide-74
SLIDE 74

Sort In-Place

a

10 –10 7 2 2 –4 –7 –10 1 4 5

already
 sorted

  • In order to insert, we swap

until the element reaches the correct position

slide-75
SLIDE 75
  • In order to insert, we swap

until the element reaches the correct position

Sort In-Place

a

10 –10 7 2 2 –4 –7 –10 1 4 5

already
 sorted

slide-76
SLIDE 76
  • In order to insert, we swap

until the element reaches the correct position

Sort In-Place

a

10 –10 7 2 2 –4 –7 –10 1 4 5

already
 sorted

slide-77
SLIDE 77
  • In order to insert, we swap

until the element reaches the correct position

Sort In-Place

a

10 –10 7 2 2 –4 –7 –10 1 4 5

already
 sorted

slide-78
SLIDE 78
  • In order to insert, we swap

until the element reaches the correct position

Sort In-Place

a

10 –10 7 2 2 –4 –7 –10 1 4 5

already
 sorted

slide-79
SLIDE 79
  • In order to insert, we swap

until the element reaches the correct position

Sort In-Place

a

10 –10 7 2 2 –4 –7 –10 1 4 5

already
 sorted

slide-80
SLIDE 80
  • In order to insert, we swap

until the element reaches the correct position

Sort In-Place

a

10 –10 7 2 2 –4 –7 –10 1 4 5

already
 sorted

slide-81
SLIDE 81

Demo

slide-82
SLIDE 82

Insertion Sort

void insertion_sort(int a[], int size) { for (int i = 1; i < size; i++) { int j = i; while (j > 0 && a[j - 1] > a[j]) { // Swap a[j] and a[j - 1] int tmp = a[j]; a[j] = a[j - 1]; a[j - 1] = tmp; j--; } }

}

  • How many comparisons does insertion_sort() need?
slide-83
SLIDE 83

Complexity

Having n elements:

  • Insertion sort: n2 comparisons
slide-84
SLIDE 84

Merge Sort

6 7 8 9 10

–7 –10 1 4 5

1 2 3 4 5

10 –10 7 2 2 –4

slide-85
SLIDE 85

Merge Sort

1 2 3 4 5

10 –10 7 2 2 –4

6 7 8 9 10

–7 –10 1 4 5

slide-86
SLIDE 86

Merge Sort

1 2 3 4 5

10 –10 7 2 2 –4

6 7 8 9 10

–7 –10 1 4 5

slide-87
SLIDE 87

Merge Sort

1 2 3 4 5

10 –10 7 2 2 –4

6 7 8 9 10

–7 –10 1 4 5

slide-88
SLIDE 88

Merge Sort

1 2 3 4 5

10 –10 7 2 2 –4

6 7 8 9 10

–7 –10 1 4 5

1 2 3 4 5 6 7 8 9 10

slide-89
SLIDE 89

Merge Sort

1 2 3 4 5

10 –10 7 2 2 –4

6 7 8 9 10

–7 –10 1 4 5

1 2 3 4 5 6 7 8 9 10

slide-90
SLIDE 90

Merge Sort

1 2 3 4 5

10 –10 7 2 2 –4

6 7 8 9 10

–7 –10 1 4 5

1 2 3 4 5 6 7 8 9 10

slide-91
SLIDE 91

Merge Sort

1 2 3 4 5

10 –10 7 2 2 –4

6 7 8 9 10

–7 –10 1 4 5

1 2 3 4 5 6 7 8 9 10

slide-92
SLIDE 92

Merge Sort

1 2 3 4 5

10 –10 7 2 2 –4

6 7 8 9 10

–7 –10 1 4 5

1 2 3 4 5 6 7 8 9 10

slide-93
SLIDE 93

Merge Sort

1 2 3 4 5

10 –10 7 2 2 –4

6 7 8 9 10

–7 –10 1 4 5

1 2 3 4 5 6 7 8 9 10

slide-94
SLIDE 94

Merge Sort

1 2 3 4 5

10 –10 7 2 2 –4

6 7 8 9 10

–7 –10 1 4 5

1 2 3 4 5 6 7 8 9 10

slide-95
SLIDE 95

Merge Sort

1 2 3 4 5

10 –10 7 2 2 –4

6 7 8 9 10

–7 –10 1 4 5

1 2 3 4 5 6 7 8 9 10

slide-96
SLIDE 96

Merge Sort

1 2 3 4 5

10 –10 7 2 2 –4

6 7 8 9 10

–7 –10 1 4 5

1 2 3 4 5 6 7 8 9 10

slide-97
SLIDE 97

Merge Sort

1 2 3 4 5

10 –10 7 2 2 –4

6 7 8 9 10

–7 –10 1 4 5

1 2 3 4 5 6 7 8 9 10

slide-98
SLIDE 98

Merge Sort

1 2 3 4 5

10 –10 7 2 2 –4

6 7 8 9 10

–7 –10 1 4 5

1 2 3 4 5 6 7 8 9 10

slide-99
SLIDE 99

Merge Sort

1 2 3 4 5

10 –10 7 2 2 –4

6 7 8 9 10

–7 –10 1 4 5

1 2 3 4 5 6 7 8 9 10

slide-100
SLIDE 100

Merge Sort

1 2 3 4 5

10 –10 7 2 2 –4

6 7 8 9 10

–7 –10 1 4 5

slide-101
SLIDE 101

Merge Sort

1 2 3 4 5

10 –10 7 2 2 –4

6 7 8 9 10

–7 –10 1 4 5

How do I sort partial arrays?

slide-102
SLIDE 102

Sorting Recursively

  • Basic idea: apply merge_sort() recursively

to the partial arrays

slide-103
SLIDE 103

John von Neumann

1903–1957

slide-104
SLIDE 104

Demo

slide-105
SLIDE 105

Call

// Sort a[0..size], using b[0..size] as a buffer void merge_sort(int a[], int b[], int size) { partial_merge_sort(a, b, 0, size); }

  • We use b[] as an auxiliary array
  • Must have the same size as a[]
slide-106
SLIDE 106

Sort

// Sort a[begin..end], using b[begin..end] as buffer void partial_merge_sort(int a[], int b[], int begin, int end) { if (end - begin < 2) return; // Split and sort int mid = begin + (end - begin) / 2; partial_merge_sort(a, b, begin, mid); partial_merge_sort(a, b, mid, end); // Merge and copy merge(a, b, begin, mid, end); copy(a, b, begin, end); }

slide-107
SLIDE 107

Merge

  • P && Q only evaluates Q if P is true
  • P || Q only evaluates Q if P is false
  • Protects the array borders from access

// Merge a[begin..mid - 1] and a[mid..end] into b[begin..end] void merge(int a[], int b[], int begin, int mid, int end) { int i_begin = begin; int i_mid = mid; for (int j = begin; j < end; j++) if (i_begin < mid && (i_mid >= end || a[i_begin] <= a[i_mid])) b[j] = a[i_begin++]; else b[j] = a[i_mid++]; }

slide-108
SLIDE 108

Copy

// Copy b[begin..end] into a[begin..end] void copy(int a[], int b[], int begin, int end) { for (int k = begin; k < end; k++) a[k] = b[k]; }

  • How many comparisons does merge_sort() need?
slide-109
SLIDE 109

Complexity Compared

Having n elements:

  • Insertion sort: n2 comparisons
  • Merge Sort: n log2 n comparisons
slide-110
SLIDE 110

5 10 15 20 25 5 10 15 20

exponential quadratic linear logarithmic linear- logarithmic

slide-111
SLIDE 111

Algorithms

Calculate Search Sort

  • Fibonacci
  • GCD
  • Collatz
  • Linear
  • Binary
  • Insertion
  • Merge
slide-112
SLIDE 112

Algorithms

Calculate Search Sort

  • Fibonacci
  • GCD
  • Collatz
  • Linear
  • Binary
  • Insertion
  • Merge
slide-113
SLIDE 113

Ultrasound!

  • HC-SR04 combines an ultrasound-speaker

and a microphone

  • Can measure signal delays

Speaker Microphone

slide-114
SLIDE 114

Buzzer

  • Simple


Piezo Buzzer

  • Can output


sounds in arbitrary
 frequencies

slide-115
SLIDE 115

Theremin

+

slide-116
SLIDE 116

Theremin

slide-117
SLIDE 117
slide-118
SLIDE 118

Theremin =

+

slide-119
SLIDE 119

Handouts

slide-120
SLIDE 120

19, 58, 29, 88, 44, 22, 11, 34, 17, 52, 26, 13, 40, 20, 10, 5, 16, 8, 4, 2, 1, …

Halting Problem

  • Will collatz() return

for every n?

  • Solution only by trial


(in infjnite time)

void collatz(int n) { while (n != 1) { if (n % 2 == 0) n = n / 2; else n = 3 * n + 1; } }

It is impossible to show correctness automatically for all programs

slide-121
SLIDE 121

Binary Search

int sorted_find(int a[], int size, int x) { int left = 0; int right = size - 1; while (left <= right) { int mid = left + ((right - left) / 2); if (x == a[mid]) return mid; else if (x < a[mid]) right = mid - 1; else // (x > a[mid]) left = mid + 1; } return -1; }

slide-122
SLIDE 122

Binary Search

a

10 –10 7 2 2 –4 –7 –10 1 4 5

left

int index = sorted_find(a, 11, -4);

right mid

slide-123
SLIDE 123

Insertion Sort

void insertion_sort(int a[], int size) { for (int i = 1; i < size; i++) { int j = i; while (j > 0 && a[j - 1] > a[j]) { // Swap a[j] and a[j - 1] int tmp = a[j]; a[j] = a[j - 1]; a[j - 1] = tmp; j--; } }

}

slide-124
SLIDE 124

Merge Sort

// Sort a[0..size], using b[0..size] as a buffer void merge_sort(int a[], int b[], int size) { partial_merge_sort(a, b, 0, size); }

  • We use b[] as an auxiliary array
  • Must have the same size as a[]
slide-125
SLIDE 125

Sort

// Sort a[begin..end], using b[begin..end] as buffer void partial_merge_sort(int a[], int b[], int begin, int end) { if (end - begin < 2) return; // Split and sort int mid = begin + (end - begin) / 2; partial_merge_sort(a, b, begin, mid); partial_merge_sort(a, b, mid, end); // Merge and copy merge(a, b, begin, mid, end); copy(a, b, begin, end); }

slide-126
SLIDE 126

Merge

  • P && Q only evaluates Q if P is true
  • P || Q only evaluates Q if P is false
  • Protects the array borders from access

// Merge a[begin..mid - 1] and a[mid..end] into b[begin..end] void merge(int a[], int b[], int begin, int mid, int end) { int i_begin = begin; int i_mid = mid; for (int j = begin; j < end; j++) if (i_begin < mid && (i_mid >= end || a[i_begin] <= a[i_mid])) b[j] = a[i_begin++]; else b[j] = a[i_mid++]; }

slide-127
SLIDE 127

Copy

// Copy b[begin..end] into a[begin..end] void copy(int a[], int b[], int begin, int end) { for (int k = begin; k < end; k++) a[k] = b[k]; }