Lecture 20: Topological Sort Algorithm Algorithm Slide 1 MP1 - - PowerPoint PPT Presentation

lecture 20 topological sort algorithm algorithm slide 1
SMART_READER_LITE
LIVE PREVIEW

Lecture 20: Topological Sort Algorithm Algorithm Slide 1 MP1 - - PowerPoint PPT Presentation

Chair of Softw are Engineering Einfhrung in die Programmierung Einfhrung in die Programmierung Introduction to Programming P Prof. Dr. Bertrand Meyer f D B t d M MP1 HS 2007 Lecture 20: Topological Sort Algorithm Algorithm Slide


slide-1
SLIDE 1

Chair of Softw are Engineering

Einführung in die Programmierung Einführung in die Programmierung Introduction to Programming

P f D B t d M

  • Prof. Dr. Bertrand Meyer

HS 2007

Lecture 20: Topological Sort Algorithm Algorithm

MP1

slide-2
SLIDE 2

Slide 1 MP1 Decide on footnote ("Info1" or "Intro-course" or whatever)

Michela Pedroni, 9/16/2003

slide-3
SLIDE 3

Back to software...

  • Intro. to Programming, lecture 20: Topological sort algorithm

2

slide-4
SLIDE 4

Overall structure (original)

Given: A type G

class TOPOLOGICAL SORTABLE [G ]

A type G A set of elements of type G l i i

TOPOLOGICAL_SORTABLE [G ] feature constraints : LINKED_LIST [TUPLE [G, G ]]

A relation constraints

  • n these elements

elements : LINKED_LIST [G ]

Required: An enumeration of the

topologically_sorted : LINKED_LIST [G ] require

elements in an order compatible with constraints

require no_cycle (constraints) do ... ensure compatible (Result, constraints) end

  • Intro. to Programming, lecture 20: Topological sort algorithm

3

end end

slide-5
SLIDE 5

Overall structure (improved)

class TOPOLOGICAL SORTED [G ] TOPOLOGICAL_SORTED [G ] feature constraints : LINKED_LIST [TUPLE [G, G ]] elements : LINKED_LIST [G ]

Instead of a function topologically_sorted, use:

sorted : LINKED_LIST [G ] process

  • A procedure process.
  • An attribute sorted

p require no_cycle (constraints) do

n attr ut sort (set by process), to hold the result.

... ensure compatible (sorted, constraints) end d

  • Intro. to Programming, lecture 20: Topological sort algorithm

4

end

slide-6
SLIDE 6

Non-uniqueness

In general there are several possible solutions

2

y

1 2

b d

1

a

1 2 3 c

In practice topological sort uses an optimization criterion to choose between possible solutions.

  • Intro. to Programming, lecture 20: Topological sort algorithm

5

slide-7
SLIDE 7

A partial order is acyclic

< The relation:

Must be a partial order: no cycle in the transitive

< The relation:

Must be a partial order: no cycle in the transitive

closure of constraints

This means there is no circular chain of the form This means there is no circular chain of the form

e0 e1 … en e0

< < < < If there is such a cycle there exists no solution to the If there is such a cycle, there exists no solution to the topological sort problem!

  • Intro. to Programming, lecture 20: Topological sort algorithm

6

slide-8
SLIDE 8

Cycles

In topological sort, we are not given the actual relation , but a relation constraints, through a set of pairs such as < but a relation constraints, through a set of pairs such as {[Dishes, Out], [Museum, Lunch], [Medicine, Lunch], [Lunch, Dishes]} Th l ti f i t t i The relation of interest is: = constraints + <

Partial

is acyclic if and only if constraints contains no set of pairs

Partial

  • rder

Acyclic

is acyclic if and only if constraints contains no set of pairs {[f0, f1], [f1, f2], …, [fm, f0]} <

1 1 2 m

When such a cycle exists, there can be no total order

  • Intro. to Programming, lecture 20: Topological sort algorithm

7

compatible with constraints.

slide-9
SLIDE 9

Overall structure (reminder)

class TOPOLOGICAL SORTED [G ] TOPOLOGICAL_SORTED [G ] feature constraints : LINKED_LIST [TUPLE [G, G ]] elements : LINKED_LIST [G ] sorted : LINKED_LIST [G ] process p require no_cycle (constraints) do ... ensure compatible (sorted, constraints) end d

  • Intro. to Programming, lecture 20: Topological sort algorithm

8

end

slide-10
SLIDE 10

Original assumption

process require require no_cycle (constraints) do ... ensure compatible (sorted constraints) compatible (sorted, constraints) end This assumes there are no cycles in the input. Such an assumption is not enforceable in practice. In particular: finding cycles is essentially as hard

  • Intro. to Programming, lecture 20: Topological sort algorithm

9

as topological sort.

slide-11
SLIDE 11

Dealing with cycles

Don’t assume anything; find cycles as byproduct of attempt to do topological sort attempt to do topological sort The scheme for process becomes: The scheme for process becomes:

“Att t t d t l i l t “Attempt to do topological sort, accounting for possible cycles” if “Cycles found” then “Report cycles” Report cycles end

  • Intro. to Programming, lecture 20: Topological sort algorithm

10

slide-12
SLIDE 12

Overall structure (as previously improved)

class TOPOLOGICAL_SORTED [G ] class TOPOLOGICAL_SORTED [G ] [ ] feature constraints : LINKED_LIST [TUPLE [G, G ]] elements : LINKED_LIST [G ] [ ] feature constraints : LINKED_LIST [TUPLE [G, G ]] elements : LINKED_LIST [G ] sorted : LINKED_LIST [G ] sorted : LINKED_LIST [G ] process process require no_cycle (constraints) do ... ensure compatible (sorted, constraints) end

  • Intro. to Programming, lecture 20: Topological sort algorithm

11

end end

slide-13
SLIDE 13

Overall structure (final)

class TOPOLOGICAL_SORTED [G ] f t feature constraints : LINKED_LIST [TUPLE [G, G ]] elements : LINKED_LIST [G ] sorted : LINKED LIST [G ] sorted : LINKED_LIST [G ]

process process require

  • - No precondition in this version

do ... ensure compatible (sorted, constraints)

d

“sorted contains all elements not initially involved in a cycle” end

  • Intro. to Programming, lecture 20: Topological sort algorithm

12

end

slide-14
SLIDE 14

The basic algorithm idea

p r t v

  • q

s u w

  • u
  • sorted

sorted

  • Intro. to Programming, lecture 20: Topological sort algorithm

13

slide-15
SLIDE 15

The basic loop scheme

… loop “Find a member next of elements for which constraints contains no pair of the form [x, next]” sorted.extend (next) “Remove next from elements, and remove from constraints any pairs of the form [next, y]” end

  • Intro. to Programming, lecture 20: Topological sort algorithm

14

slide-16
SLIDE 16

The loop invariant

Original architecture: “constraints + has no cycles” Revised architecture: “constraints + has no cycles other than any that were present originally”

  • Intro. to Programming, lecture 20: Topological sort algorithm

15

slide-17
SLIDE 17

Overall structure (as previously improved)

class TOPOLOGICAL_SORTED [G ] class TOPOLOGICAL_SORTED [G ] [ ] feature constraints : LINKED_LIST [TUPLE [G, G ]] elements : LINKED_LIST [G ] [ ] feature constraints : LINKED_LIST [TUPLE [G, G ]] elements : LINKED_LIST [G ] sorted : LINKED_LIST [G ] sorted : LINKED_LIST [G ] process process require no_cycle (constraints) do ... ensure compatible (sorted, constraints) end

  • Intro. to Programming, lecture 20: Topological sort algorithm

16

end end

slide-18
SLIDE 18

Overall structure (final)

class TOPOLOGICAL_SORTED [G ] f t feature constraints : LINKED_LIST [TUPLE [G, G ]] elements : LINKED_LIST [G ] sorted : LINKED LIST [G ] sorted : LINKED_LIST [G ]

process process require

  • - No precondition in this version

do ... ensure compatible (sorted, constraints)

d

“sorted contains all elements not initially involved in a cycle” end

  • Intro. to Programming, lecture 20: Topological sort algorithm

17

end

slide-19
SLIDE 19

The loop invariant

Original architecture: “constraints + has no cycles” Revised architecture: “constraints + has no cycles other than any that were present originally”

  • Intro. to Programming, lecture 20: Topological sort algorithm

18

slide-20
SLIDE 20

Terminology

If constraints has a pair [x, y ], we say that

x is a predecessor of y y is a successor of x

y

  • Intro. to Programming, lecture 20: Topological sort algorithm

19

slide-21
SLIDE 21

Algorithm scheme

process do from create {...} sorted.make invariant from create {...} sorted.make invariant “constraints includes no cycles other than original ones” and “sorted is compatible with constraints” and “All original elements are in either sorted or elements” i t variant “ Size of elements” until “ Every member of elements has a predecessor” Every member of elements has a predecessor loop next := “A member of elements with no predecessor” sorted.extend (next) ( ) “Remove next from elements ” “Remove from constraints all pairs [next, y]” end if “No more elements” then “Report that topological sort is complete” else “Report cycle in remaining constraints and elements”

  • Intro. to Programming, lecture 20: Topological sort algorithm

20

Report cycle in remaining constraints and elements end end

slide-22
SLIDE 22

Implementing the algorithm

We start with these data structures, directly reflecting input data:

(Number of elements: n Number of constraints: m)

constraints : LINKED_LIST [TUPLE [G, G ]] elements : LINKED_LIST [G ]

y

Example:

2

b d

p elements = {a, b, c, d} constraints = {[a, b], [a, d ], [b, d ], [c, d]}

1

a

{[a, b], [a, d ], [b, d ], [c, d]}

1 2 3 c

  • Intro. to Programming, lecture 20: Topological sort algorithm

21

slide-23
SLIDE 23

Data structures 1: original

elements = {a, b, c, d} constraints = {[a b] [a d] [b d] [c d]} constraints = {[a, b], [a, d], [b, d], [c, d]} b c d a n elements elements constraints a b a d b d c d m constraints Efficiency: The best we can hope for: O (m + n)

  • Intro. to Programming, lecture 20: Topological sort algorithm

22

slide-24
SLIDE 24

Basic operations

process do from create {...} sorted.make invariant “constraints includes no cycles other than original ones” and constraints includes no cycles other than original ones and “sorted is compatible with constraints” and “All original elements are in either sorted or elements” variant “Si f l m ts” “Size of elements” until “ Every member of elements has a predecessor” loop t “A b f l t ith d ” next := “A member of elements with no predecessor” sorted.extend (next) “Remove next from elements ” “Remove from constraints all pairs of the form [next, y] ” end if “No more elements” then “R h l i l i l ” “Report that topological sort is complete” else “Report cycle, in constraints and elements ” end

  • Intro. to Programming, lecture 20: Topological sort algorithm

23

end

slide-25
SLIDE 25

The operations we need (n times)

Find out if there’s any element with no predecessor

(and then get one)

Remove a given element from the set of elements Remove from the set of constraints all those starting

with a given element

Find out if there’s any element left

  • Intro. to Programming, lecture 20: Topological sort algorithm

24

slide-26
SLIDE 26

Data structures 1: original

elements = {a, b, c, d} constraints = {[a b] [a d] [b d] [c d]} constraints = {[a, b], [a, d], [b, d], [c, d]} elements b c d a n elements b d b d d m constraints constraints a b a d b d c d Effi i Th b t h f O ( ) m constraints Efficiency: The best we can hope for: O (m + n) Using elements and constraints as given wouldn’t allow hi thi !

  • Intro. to Programming, lecture 20: Topological sort algorithm

25

reaching this!

slide-27
SLIDE 27

Implementing the algorithm

Choose a better internal representation

Gi

l t b ( ll s si s)

Give every element a number (allows using arrays) Represent constraints in a form adapted to what we

want to do with this structure: want to do with this structure:

  • “Find next such that constraints has no pair of the

form [y next ]” form [y, next ]

  • “Given next , remove from constraints all pairs of

h f [ ]” the form [next, y ]”

  • Intro. to Programming, lecture 20: Topological sort algorithm

26

slide-28
SLIDE 28

Algorithm scheme (without invariant and variant)

process do from create { } sorted make until from create {...} sorted.make until “ Every member of elements has a predecessor” loop loop next := “A member of elements with no predecessor” sorted.extend (next) “Remove next from elements ” “Remove from constraints all pairs [next, y]” end end if “No more elements” then “Report that topological sort is complete” p p g p else “Report cycle in remaining constraints and elements” end d

  • Intro. to Programming, lecture 20: Topological sort algorithm

27

end

slide-29
SLIDE 29

Data structure 1: representing elements

elements : ARRAY [G ]

  • - Items subject to ordering constraints
  • - (Replaces the original list)

d

4

b c

2 3

a

1

elements

elements = {a, b, c, d } constraints = {[a b] [a d] [b d] [c d ]}

  • Intro. to Programming, lecture 20: Topological sort algorithm

28

constraints = {[a, b], [a, d], [b, d], [c, d ]}

slide-30
SLIDE 30

Data structure 2: representing constraints

successors: ARRAY [LINKED_LIST [INTEGER]]

  • - Items that must appear after any given one

3 4

4

2 1

2 4 4

successors

elements = {a, b, c, d } constraints = {[a b] [a d] [b d] [c d ]}

  • Intro. to Programming, lecture 20: Topological sort algorithm

29

constraints = {[a, b], [a, d], [b, d], [c, d ]}

slide-31
SLIDE 31

Data structure 3: representing constraints

predecessor_count: ARRAY [INTEGER]

  • - Number of items that must appear before a given
  • ne

3 4 3 2 1 1

predecessor_count

elements = {a, b, c, d } constraints = {[a b] [a d] [b d] [c d ]}

  • Intro. to Programming, lecture 20: Topological sort algorithm

30

constraints = {[a, b], [a, d], [b, d], [c, d ]}

slide-32
SLIDE 32

Reminder: basic algorithm idea

p r t v

  • q

s u w

  • u
  • sorted

sorted

  • Intro. to Programming, lecture 20: Topological sort algorithm

31

slide-33
SLIDE 33

Finding a “candidate” (element with no predecessor)

process do from create { } sorted make until from create {...} sorted.make until “ Every member of elements has a predecessor” loop loop next := “A member of elements with no predecessor” sorted.extend (next) “Remove next from elements ” “Remove from constraints all pairs [next, y]” end end if “No more elements” then “Report that topological sort is complete” p p g p else “Report cycle in remaining constraints and elements” end d

  • Intro. to Programming, lecture 20: Topological sort algorithm

32

end

slide-34
SLIDE 34

Finding a candidate (1)

Implement next := “A member of elements with no predecessors” as: Let next be an integer, not yet processed, such that predecessor_count [next] = 0 This requires an O (n) search through all indexes: bad! B i But wait...

  • Intro. to Programming, lecture 20: Topological sort algorithm

33

slide-35
SLIDE 35

Removing successors

process do from create { } sorted make until from create {...} sorted.make until “ Every member of elements has a predecessor” loop loop next := “A member of elements with no predecessor” sorted.extend (next) “Remove next from elements ” “Remove from constraints all pairs [next, y]” end end if “No more elements” then “Report that topological sort is complete” p p g p else “Report cycle in remaining constraints and elements ” end d

  • Intro. to Programming, lecture 20: Topological sort algorithm

34

end

slide-36
SLIDE 36

Removing successors

Implement

3 4

3 predecessor_count

Implement “Remove from constraints all pairs [next, y]”

2 1 3

1

as a loop over the successors of next :

3 4

4

targets := successors [next ] from targets.start until f

2 1

2 4 4

targets.after loop freed := targets.item

successors

f g

.

predecessor_count [freed ] := predecessor_count [freed ] − 1 targets.forth

  • Intro. to Programming, lecture 20: Topological sort algorithm

35

end

slide-37
SLIDE 37

Removing successors

3 4

3

Implement

predecessor_count

2 1 3

1

Implement “Remove from constraints all pairs [next, y]”

3 4

4

as a loop over the successors of next :

2 1

2 4 4

targets := successors [next ] from targets.start until f

successors

targets.after loop freed := targets.item f g

.

predecessor_count [freed ] := predecessor_count [freed ] − 1 targets.forth

  • Intro. to Programming, lecture 20: Topological sort algorithm

36

end

slide-38
SLIDE 38

Removing successors

3 4

3 2

Implement

predecessor_count

2 1 3

1

Implement “Remove from constraints all pairs [next, y]”

3 4

4

as a loop over the successors of next :

2 1

2 4 4

targets := successors [next ] from targets.start until f

successors

targets.after loop freed := targets.item f g

.

predecessor_count [freed ] := predecessor_count [freed ] − 1 targets.forth

  • Intro. to Programming, lecture 20: Topological sort algorithm

37

end

slide-39
SLIDE 39

Removing successors

3 4

3 21

Implement

predecessor_count

2 1 3

1

Implement “Remove from constraints all pairs [next, y]”

3 4

4

as a loop over the successors of next :

2 1

2 4 4

targets := successors [next ] from targets.start until f

successors

targets.after loop freed := targets.item f g

.

predecessor_count [freed ] := predecessor_count [freed ] − 1 targets.forth

  • Intro. to Programming, lecture 20: Topological sort algorithm

38

end

slide-40
SLIDE 40

Removing successors

3 4

3 210

Implement

predecessor_count

2 1 3

1

Implement “Remove from constraints all pairs [next, y]”

3 4

4

as a loop over the successors of next :

2 1

2 4 4

targets := successors [next ] from targets.start until f

successors

targets.after loop freed := targets.item f g

.

predecessor_count [freed ] := predecessor_count [freed ] − 1 targets.forth

  • Intro. to Programming, lecture 20: Topological sort algorithm

39

end

slide-41
SLIDE 41

Algorithm scheme

process do from create { } sorted make until from create {...} sorted.make until “ Every member of elements has a predecessor” loop loop next := “A member of elements with no predecessor” sorted.extend (next) “Remove next from elements ” “Remove from constraints all pairs [next, y]” end end if “No more elements” then “Report that topological sort is complete” p p g p else “Report cycle in remaining constraints and elements” end d

  • Intro. to Programming, lecture 20: Topological sort algorithm

40

end

slide-42
SLIDE 42

Finding a candidate (1)

Implement next := “A member of elements with no predecessors” as: Let next be an integer, not yet processed, such that predecessor_count [next] = 0 We said: “Seems to require an O (n) search through all indexes Seems to require an O (n) search through all indexes, but wait...”

  • Intro. to Programming, lecture 20: Topological sort algorithm

41

slide-43
SLIDE 43

Removing successors

3 4

3 210

Implement

predecessor_count

2 1 3

1

Implement “Remove from constraints all pairs [next, y]”

3 4

4

as a loop over the successors of next :

2 1

2 4 4

targets := successors [next ] from targets.start until f

successors

targets.after loop freed := targets.item f g

.

predecessor_count [freed ] := predecessor_count [freed ] − 1 targets.forth

  • Intro. to Programming, lecture 20: Topological sort algorithm

42

end

slide-44
SLIDE 44

Finding a candidate (2): on the spot

Complement predecessor_count [freed ] := predecessor count [freed ] 1 predecessor_count [freed ] − 1 by by if predecessor_count [freed ] = 0 then

  • - We have found a candidate!

We have found a candidate! candidates.put (freed ) end

  • Intro. to Programming, lecture 20: Topological sort algorithm

43

slide-45
SLIDE 45

Data structure 4: candidates

candidates : STACK [INTEGER ]

  • - Items with no predecessor

candidates

Instead of a stack, candidates can be any dispenser structure, e.g. queue, priority queue , g q , p y q The choice will determine which topological sort we get,

  • Intro. to Programming, lecture 20: Topological sort algorithm

44

p g g when there are several possible ones

slide-46
SLIDE 46

Algorithm scheme

process do from create { } sorted make until from create {...} sorted.make until “ Every member of elements has a predecessor” loop loop next := “A member of elements with no predecessor” sorted.extend (next) “Remove next from elements ” “Remove from constraints all pairs [next, y]” end end if “No more elements” then “Report that topological sort is complete” p p g p else “Report cycle in remaining constraints and elements” end d

  • Intro. to Programming, lecture 20: Topological sort algorithm

45

end

slide-47
SLIDE 47

Finding a candidate (2)

Implement next := “A member of elements with no predecessor” if candidates is not empty, as: next := candidates.item

  • Intro. to Programming, lecture 20: Topological sort algorithm

46

slide-48
SLIDE 48

Algorithm scheme

process do from create { } sorted make until from create {...} sorted.make until “ Every member of elements has a predecessor” loop loop next := “A member of elements with no predecessor” sorted.extend (next) “Remove next from elements ” “Remove from constraints all pairs [next, y]” end end if “No more elements” then “Report that topological sort is complete” p p g p else “Report cycle in remaining constraints and elements” end d

  • Intro. to Programming, lecture 20: Topological sort algorithm

47

end

slide-49
SLIDE 49

Finding a candidate (3)

Implement the test “ Every member of elements of has a predecessor” as not candidates.is_empty To implement the test “No more elements”, keep count of the processed elements and, at the end, compare it with p p the original number of elements.

  • Intro. to Programming, lecture 20: Topological sort algorithm

48

slide-50
SLIDE 50

Reminder: the operations we need (n times)

Find out if there’s any element with no predecessor

(and then get one)

Remove a given element from the set of elements Remove from the set of constraints all those starting

with a given element

Find out if there’s any element left

  • Intro. to Programming, lecture 20: Topological sort algorithm

49

slide-51
SLIDE 51

Detecting cycles

process do from create { } sorted make until from create {...} sorted.make until “ Every member of elements has a predecessor” loop loop next := “A member of elements with no predecessor” sorted.extend (next) “Remove next from elements ” “Remove from constraints all pairs [next, y]” end end if “No more elements” then “Report that topological sort is complete” p p g p else “Report cycle in remaining constraints and elements” end d

  • Intro. to Programming, lecture 20: Topological sort algorithm

50

end

slide-52
SLIDE 52

Detecting cycles

To implement the test “No more elements”, keep count of the processed elements and, at the end, compare it with th i i l b f l t the original number of elements.

  • Intro. to Programming, lecture 20: Topological sort algorithm

51

slide-53
SLIDE 53

Data structures: summary

elements : ARRAY [G ]

  • - Items subject to ordering constraints
  • - (Replaces the original list)

successors : ARRAY [LINKED LIST [INTEGER]] successors : ARRAY [LINKED_LIST [INTEGER]]

  • - Items that must appear after any given one

d RR Y [ N EGER] predecessor_count : ARRAY [INTEGER]

  • - Number of items that must appear before
  • - any given one

any given one candidates : STACK [INTEGER] It ith d

  • - Items with no predecessor
  • Intro. to Programming, lecture 20: Topological sort algorithm

52

slide-54
SLIDE 54

Initialization

Must process all elements and constraints to create these data structures This is O (m + n) So is the rest of the algorithm

  • Intro. to Programming, lecture 20: Topological sort algorithm

53

slide-55
SLIDE 55

Compiling: a useful heuristics

The data structure, in the way it is given, is often not the most i t f ifi l ith i i appropriate for specific algorithmic processing To obtain an efficient algorithm you may need to turn it into a specially To obtain an efficient algorithm, you may need to turn it into a specially suited form ll “ l We may call this “compiling” the data Often the “compilation” (initialization) is as costly as the actual Often, the compilation (initialization) is as costly as the actual processing, or more, but that’s not a problem if justified by the overall cost decrease

  • Intro. to Programming, lecture 20: Topological sort algorithm

54

slide-56
SLIDE 56

Another lesson

It may be OK to duplicate information in our data structures:

successors: ARRAY [LINKED_LIST [INTEGER]]

  • - Items that must appear after any given one

4 2 3

4 4

predecessor_count: ARRAY [INTEGER]

  • - Number of items that must appear before

1

2 4

  • - any given one

3 4

2 1

This is a simple space-time tradeoff

2 1

1

  • Intro. to Programming, lecture 20: Topological sort algorithm

55

slide-57
SLIDE 57

Key concepts

A very interesting algorithm, useful in many

applications

Mathematical basis: binary relations Remember binary relations & their properties Transitive closure, Reflexive transitive closure Algorithm: adapting the data structure is the key “Compilation”strategy Initialization can be as costly as processing Algorithm not enough: need API (convenient,

extendible, reusable)

This is the difference between algorithms and

f i i

  • Intro. to Programming, lecture 20: Topological sort algorithm

56

software engineering

slide-58
SLIDE 58

Software engineering lessons

Great algorithms are not enough We must provide a solution with a clear interface (API), easy to use Turn patterns into components

  • Intro. to Programming, lecture 20: Topological sort algorithm

57

slide-59
SLIDE 59

End of lecture 22

  • Intro. to Programming, lecture 20: Topological sort algorithm

58