Lecture 20: Topological Sort Algorithm Algorithm Slide 1 MP1 - - PowerPoint PPT Presentation

Chair of Softw are Engineering

Einführung in die Programmierung Einführung in die Programmierung Introduction to Programming

P f D B t d M

• Prof. Dr. Bertrand Meyer

HS 2007

Lecture 20: Topological Sort Algorithm Algorithm

MP1

Slide 1 MP1 Decide on footnote ("Info1" or "Intro-course" or whatever)

Michela Pedroni, 9/16/2003

Back to software...

• Intro. to Programming, lecture 20: Topological sort algorithm

2

Overall structure (original)

Given: A type G

class TOPOLOGICAL SORTABLE [G ]

A type G A set of elements of type G l i i

TOPOLOGICAL_SORTABLE [G ] feature constraints : LINKED_LIST [TUPLE [G, G ]]

A relation constraints

• n these elements

elements : LINKED_LIST [G ]

Required: An enumeration of the

topologically_sorted : LINKED_LIST [G ] require

elements in an order compatible with constraints

require no_cycle (constraints) do ... ensure compatible (Result, constraints) end

• Intro. to Programming, lecture 20: Topological sort algorithm

3

end end

Overall structure (improved)

class TOPOLOGICAL SORTED [G ] TOPOLOGICAL_SORTED [G ] feature constraints : LINKED_LIST [TUPLE [G, G ]] elements : LINKED_LIST [G ]

Instead of a function topologically_sorted, use:

sorted : LINKED_LIST [G ] process

• A procedure process.
• An attribute sorted

p require no_cycle (constraints) do

n attr ut sort (set by process), to hold the result.

... ensure compatible (sorted, constraints) end d

• Intro. to Programming, lecture 20: Topological sort algorithm

4

end

Non-uniqueness

In general there are several possible solutions

2

y

1 2

b d

1

a

1 2 3 c

In practice topological sort uses an optimization criterion to choose between possible solutions.

• Intro. to Programming, lecture 20: Topological sort algorithm

5

A partial order is acyclic

< The relation:

Must be a partial order: no cycle in the transitive

< The relation:

Must be a partial order: no cycle in the transitive

closure of constraints

This means there is no circular chain of the form This means there is no circular chain of the form

e0 e1 … en e0

< < < < If there is such a cycle there exists no solution to the If there is such a cycle, there exists no solution to the topological sort problem!

• Intro. to Programming, lecture 20: Topological sort algorithm

6

Cycles

In topological sort, we are not given the actual relation , but a relation constraints, through a set of pairs such as < but a relation constraints, through a set of pairs such as {[Dishes, Out], [Museum, Lunch], [Medicine, Lunch], [Lunch, Dishes]} Th l ti f i t t i The relation of interest is: = constraints + <

Partial

is acyclic if and only if constraints contains no set of pairs

Partial

• rder

Acyclic

is acyclic if and only if constraints contains no set of pairs {[f0, f1], [f1, f2], …, [fm, f0]} <

1 1 2 m

When such a cycle exists, there can be no total order

• Intro. to Programming, lecture 20: Topological sort algorithm

7

compatible with constraints.

Overall structure (reminder)

class TOPOLOGICAL SORTED [G ] TOPOLOGICAL_SORTED [G ] feature constraints : LINKED_LIST [TUPLE [G, G ]] elements : LINKED_LIST [G ] sorted : LINKED_LIST [G ] process p require no_cycle (constraints) do ... ensure compatible (sorted, constraints) end d

• Intro. to Programming, lecture 20: Topological sort algorithm

8

end

Original assumption

process require require no_cycle (constraints) do ... ensure compatible (sorted constraints) compatible (sorted, constraints) end This assumes there are no cycles in the input. Such an assumption is not enforceable in practice. In particular: finding cycles is essentially as hard

• Intro. to Programming, lecture 20: Topological sort algorithm

9

as topological sort.

Dealing with cycles

Don’t assume anything; find cycles as byproduct of attempt to do topological sort attempt to do topological sort The scheme for process becomes: The scheme for process becomes:

“Att t t d t l i l t “Attempt to do topological sort, accounting for possible cycles” if “Cycles found” then “Report cycles” Report cycles end

• Intro. to Programming, lecture 20: Topological sort algorithm

10

Overall structure (as previously improved)

class TOPOLOGICAL_SORTED [G ] class TOPOLOGICAL_SORTED [G ] [ ] feature constraints : LINKED_LIST [TUPLE [G, G ]] elements : LINKED_LIST [G ] [ ] feature constraints : LINKED_LIST [TUPLE [G, G ]] elements : LINKED_LIST [G ] sorted : LINKED_LIST [G ] sorted : LINKED_LIST [G ] process process require no_cycle (constraints) do ... ensure compatible (sorted, constraints) end

• Intro. to Programming, lecture 20: Topological sort algorithm

11

end end

Overall structure (final)

class TOPOLOGICAL_SORTED [G ] f t feature constraints : LINKED_LIST [TUPLE [G, G ]] elements : LINKED_LIST [G ] sorted : LINKED LIST [G ] sorted : LINKED_LIST [G ]

process process require

• - No precondition in this version

do ... ensure compatible (sorted, constraints)

d

“sorted contains all elements not initially involved in a cycle” end

• Intro. to Programming, lecture 20: Topological sort algorithm

12

end

The basic algorithm idea

p r t v

• q

s u w

• u
• sorted

sorted

• Intro. to Programming, lecture 20: Topological sort algorithm

13

The basic loop scheme

… loop “Find a member next of elements for which constraints contains no pair of the form [x, next]” sorted.extend (next) “Remove next from elements, and remove from constraints any pairs of the form [next, y]” end

• Intro. to Programming, lecture 20: Topological sort algorithm

14

The loop invariant

Original architecture: “constraints + has no cycles” Revised architecture: “constraints + has no cycles other than any that were present originally”

• Intro. to Programming, lecture 20: Topological sort algorithm

15

Overall structure (as previously improved)

class TOPOLOGICAL_SORTED [G ] class TOPOLOGICAL_SORTED [G ] [ ] feature constraints : LINKED_LIST [TUPLE [G, G ]] elements : LINKED_LIST [G ] [ ] feature constraints : LINKED_LIST [TUPLE [G, G ]] elements : LINKED_LIST [G ] sorted : LINKED_LIST [G ] sorted : LINKED_LIST [G ] process process require no_cycle (constraints) do ... ensure compatible (sorted, constraints) end

• Intro. to Programming, lecture 20: Topological sort algorithm

16

end end

Overall structure (final)

class TOPOLOGICAL_SORTED [G ] f t feature constraints : LINKED_LIST [TUPLE [G, G ]] elements : LINKED_LIST [G ] sorted : LINKED LIST [G ] sorted : LINKED_LIST [G ]

process process require

• - No precondition in this version

do ... ensure compatible (sorted, constraints)

d

“sorted contains all elements not initially involved in a cycle” end

• Intro. to Programming, lecture 20: Topological sort algorithm

17

end

The loop invariant

Original architecture: “constraints + has no cycles” Revised architecture: “constraints + has no cycles other than any that were present originally”

• Intro. to Programming, lecture 20: Topological sort algorithm

18

Terminology

If constraints has a pair [x, y ], we say that

x is a predecessor of y y is a successor of x

y

• Intro. to Programming, lecture 20: Topological sort algorithm

19

Algorithm scheme

process do from create {...} sorted.make invariant from create {...} sorted.make invariant “constraints includes no cycles other than original ones” and “sorted is compatible with constraints” and “All original elements are in either sorted or elements” i t variant “ Size of elements” until “ Every member of elements has a predecessor” Every member of elements has a predecessor loop next := “A member of elements with no predecessor” sorted.extend (next) ( ) “Remove next from elements ” “Remove from constraints all pairs [next, y]” end if “No more elements” then “Report that topological sort is complete” else “Report cycle in remaining constraints and elements”

• Intro. to Programming, lecture 20: Topological sort algorithm

20

Report cycle in remaining constraints and elements end end

Implementing the algorithm

We start with these data structures, directly reflecting input data:

(Number of elements: n Number of constraints: m)

constraints : LINKED_LIST [TUPLE [G, G ]] elements : LINKED_LIST [G ]

y

Example:

2

b d

p elements = {a, b, c, d} constraints = {[a, b], [a, d ], [b, d ], [c, d]}

1

a

{[a, b], [a, d ], [b, d ], [c, d]}

1 2 3 c

• Intro. to Programming, lecture 20: Topological sort algorithm

21

Data structures 1: original

elements = {a, b, c, d} constraints = {[a b] [a d] [b d] [c d]} constraints = {[a, b], [a, d], [b, d], [c, d]} b c d a n elements elements constraints a b a d b d c d m constraints Efficiency: The best we can hope for: O (m + n)

• Intro. to Programming, lecture 20: Topological sort algorithm

22

Basic operations

process do from create {...} sorted.make invariant “constraints includes no cycles other than original ones” and constraints includes no cycles other than original ones and “sorted is compatible with constraints” and “All original elements are in either sorted or elements” variant “Si f l m ts” “Size of elements” until “ Every member of elements has a predecessor” loop t “A b f l t ith d ” next := “A member of elements with no predecessor” sorted.extend (next) “Remove next from elements ” “Remove from constraints all pairs of the form [next, y] ” end if “No more elements” then “R h l i l i l ” “Report that topological sort is complete” else “Report cycle, in constraints and elements ” end

• Intro. to Programming, lecture 20: Topological sort algorithm

23

end

The operations we need (n times)

Find out if there’s any element with no predecessor

(and then get one)

Remove a given element from the set of elements Remove from the set of constraints all those starting

with a given element

Find out if there’s any element left

• Intro. to Programming, lecture 20: Topological sort algorithm

24

Data structures 1: original

elements = {a, b, c, d} constraints = {[a b] [a d] [b d] [c d]} constraints = {[a, b], [a, d], [b, d], [c, d]} elements b c d a n elements b d b d d m constraints constraints a b a d b d c d Effi i Th b t h f O ( ) m constraints Efficiency: The best we can hope for: O (m + n) Using elements and constraints as given wouldn’t allow hi thi !

• Intro. to Programming, lecture 20: Topological sort algorithm

25

reaching this!

Implementing the algorithm

Choose a better internal representation

Gi

l t b ( ll s si s)

Give every element a number (allows using arrays) Represent constraints in a form adapted to what we

want to do with this structure: want to do with this structure:

• “Find next such that constraints has no pair of the

form [y next ]” form [y, next ]

• “Given next , remove from constraints all pairs of

h f [ ]” the form [next, y ]”

• Intro. to Programming, lecture 20: Topological sort algorithm

26

Algorithm scheme (without invariant and variant)

process do from create { } sorted make until from create {...} sorted.make until “ Every member of elements has a predecessor” loop loop next := “A member of elements with no predecessor” sorted.extend (next) “Remove next from elements ” “Remove from constraints all pairs [next, y]” end end if “No more elements” then “Report that topological sort is complete” p p g p else “Report cycle in remaining constraints and elements” end d

• Intro. to Programming, lecture 20: Topological sort algorithm

27

end

Data structure 1: representing elements

elements : ARRAY [G ]

• - Items subject to ordering constraints
• - (Replaces the original list)

d

4

b c

2 3

a

1

elements

elements = {a, b, c, d } constraints = {[a b] [a d] [b d] [c d ]}

• Intro. to Programming, lecture 20: Topological sort algorithm

28

constraints = {[a, b], [a, d], [b, d], [c, d ]}

Data structure 2: representing constraints

successors: ARRAY [LINKED_LIST [INTEGER]]

• - Items that must appear after any given one

3 4

4

2 1

2 4 4

successors

elements = {a, b, c, d } constraints = {[a b] [a d] [b d] [c d ]}

• Intro. to Programming, lecture 20: Topological sort algorithm

29

constraints = {[a, b], [a, d], [b, d], [c, d ]}

Data structure 3: representing constraints

predecessor_count: ARRAY [INTEGER]

• - Number of items that must appear before a given
• ne

3 4 3 2 1 1

predecessor_count

elements = {a, b, c, d } constraints = {[a b] [a d] [b d] [c d ]}

• Intro. to Programming, lecture 20: Topological sort algorithm

30

constraints = {[a, b], [a, d], [b, d], [c, d ]}

Reminder: basic algorithm idea

p r t v

• q

s u w

• u
• sorted

sorted

• Intro. to Programming, lecture 20: Topological sort algorithm

31

Finding a “candidate” (element with no predecessor)

process do from create { } sorted make until from create {...} sorted.make until “ Every member of elements has a predecessor” loop loop next := “A member of elements with no predecessor” sorted.extend (next) “Remove next from elements ” “Remove from constraints all pairs [next, y]” end end if “No more elements” then “Report that topological sort is complete” p p g p else “Report cycle in remaining constraints and elements” end d

• Intro. to Programming, lecture 20: Topological sort algorithm

32

end

Finding a candidate (1)

Implement next := “A member of elements with no predecessors” as: Let next be an integer, not yet processed, such that predecessor_count [next] = 0 This requires an O (n) search through all indexes: bad! B i But wait...

• Intro. to Programming, lecture 20: Topological sort algorithm

33

Removing successors

process do from create { } sorted make until from create {...} sorted.make until “ Every member of elements has a predecessor” loop loop next := “A member of elements with no predecessor” sorted.extend (next) “Remove next from elements ” “Remove from constraints all pairs [next, y]” end end if “No more elements” then “Report that topological sort is complete” p p g p else “Report cycle in remaining constraints and elements ” end d

• Intro. to Programming, lecture 20: Topological sort algorithm

34

end

Removing successors

Implement

3 4

3 predecessor_count

Implement “Remove from constraints all pairs [next, y]”

2 1 3

1

as a loop over the successors of next :

3 4

4

targets := successors [next ] from targets.start until f

2 1

2 4 4

targets.after loop freed := targets.item

successors

f g

.

predecessor_count [freed ] := predecessor_count [freed ] − 1 targets.forth

• Intro. to Programming, lecture 20: Topological sort algorithm

35

end

Removing successors

3 4

3

Implement

predecessor_count

2 1 3

1

Implement “Remove from constraints all pairs [next, y]”

3 4

4

as a loop over the successors of next :

2 1

2 4 4

targets := successors [next ] from targets.start until f

successors

targets.after loop freed := targets.item f g

.

predecessor_count [freed ] := predecessor_count [freed ] − 1 targets.forth

• Intro. to Programming, lecture 20: Topological sort algorithm

36

end

Removing successors

3 4

3 2

Implement

predecessor_count

2 1 3

1

Implement “Remove from constraints all pairs [next, y]”

3 4

4

as a loop over the successors of next :

2 1

2 4 4

targets := successors [next ] from targets.start until f

successors

targets.after loop freed := targets.item f g

.

predecessor_count [freed ] := predecessor_count [freed ] − 1 targets.forth

• Intro. to Programming, lecture 20: Topological sort algorithm

37

end

Removing successors

3 4

3 21

Implement

predecessor_count

2 1 3

1

Implement “Remove from constraints all pairs [next, y]”

3 4

4

as a loop over the successors of next :

2 1

2 4 4

targets := successors [next ] from targets.start until f

successors

targets.after loop freed := targets.item f g

.

predecessor_count [freed ] := predecessor_count [freed ] − 1 targets.forth

• Intro. to Programming, lecture 20: Topological sort algorithm

38

end

Removing successors

3 4

3 210

Implement

predecessor_count

2 1 3

1

Implement “Remove from constraints all pairs [next, y]”

3 4

4

as a loop over the successors of next :

2 1

2 4 4

targets := successors [next ] from targets.start until f

successors

targets.after loop freed := targets.item f g

.

predecessor_count [freed ] := predecessor_count [freed ] − 1 targets.forth

• Intro. to Programming, lecture 20: Topological sort algorithm

39

end

Algorithm scheme

process do from create { } sorted make until from create {...} sorted.make until “ Every member of elements has a predecessor” loop loop next := “A member of elements with no predecessor” sorted.extend (next) “Remove next from elements ” “Remove from constraints all pairs [next, y]” end end if “No more elements” then “Report that topological sort is complete” p p g p else “Report cycle in remaining constraints and elements” end d

• Intro. to Programming, lecture 20: Topological sort algorithm

40

end

Finding a candidate (1)

Implement next := “A member of elements with no predecessors” as: Let next be an integer, not yet processed, such that predecessor_count [next] = 0 We said: “Seems to require an O (n) search through all indexes Seems to require an O (n) search through all indexes, but wait...”

• Intro. to Programming, lecture 20: Topological sort algorithm

41

Removing successors

3 4

3 210

Implement

predecessor_count

2 1 3

1

Implement “Remove from constraints all pairs [next, y]”

3 4

4

as a loop over the successors of next :

2 1

2 4 4

targets := successors [next ] from targets.start until f

successors

targets.after loop freed := targets.item f g

.

predecessor_count [freed ] := predecessor_count [freed ] − 1 targets.forth

• Intro. to Programming, lecture 20: Topological sort algorithm

42

end

Finding a candidate (2): on the spot

Complement predecessor_count [freed ] := predecessor count [freed ] 1 predecessor_count [freed ] − 1 by by if predecessor_count [freed ] = 0 then

• - We have found a candidate!

We have found a candidate! candidates.put (freed ) end

• Intro. to Programming, lecture 20: Topological sort algorithm

43

Data structure 4: candidates

candidates : STACK [INTEGER ]

• - Items with no predecessor

candidates

Instead of a stack, candidates can be any dispenser structure, e.g. queue, priority queue , g q , p y q The choice will determine which topological sort we get,

• Intro. to Programming, lecture 20: Topological sort algorithm

44

p g g when there are several possible ones

Algorithm scheme

process do from create { } sorted make until from create {...} sorted.make until “ Every member of elements has a predecessor” loop loop next := “A member of elements with no predecessor” sorted.extend (next) “Remove next from elements ” “Remove from constraints all pairs [next, y]” end end if “No more elements” then “Report that topological sort is complete” p p g p else “Report cycle in remaining constraints and elements” end d

• Intro. to Programming, lecture 20: Topological sort algorithm

45

end

Finding a candidate (2)

Implement next := “A member of elements with no predecessor” if candidates is not empty, as: next := candidates.item

• Intro. to Programming, lecture 20: Topological sort algorithm

46

Algorithm scheme

process do from create { } sorted make until from create {...} sorted.make until “ Every member of elements has a predecessor” loop loop next := “A member of elements with no predecessor” sorted.extend (next) “Remove next from elements ” “Remove from constraints all pairs [next, y]” end end if “No more elements” then “Report that topological sort is complete” p p g p else “Report cycle in remaining constraints and elements” end d

• Intro. to Programming, lecture 20: Topological sort algorithm

47

end

Finding a candidate (3)

Implement the test “ Every member of elements of has a predecessor” as not candidates.is_empty To implement the test “No more elements”, keep count of the processed elements and, at the end, compare it with p p the original number of elements.

• Intro. to Programming, lecture 20: Topological sort algorithm

48

Reminder: the operations we need (n times)

Find out if there’s any element with no predecessor

(and then get one)

Remove a given element from the set of elements Remove from the set of constraints all those starting

with a given element

Find out if there’s any element left

• Intro. to Programming, lecture 20: Topological sort algorithm

49

Detecting cycles

process do from create { } sorted make until from create {...} sorted.make until “ Every member of elements has a predecessor” loop loop next := “A member of elements with no predecessor” sorted.extend (next) “Remove next from elements ” “Remove from constraints all pairs [next, y]” end end if “No more elements” then “Report that topological sort is complete” p p g p else “Report cycle in remaining constraints and elements” end d

• Intro. to Programming, lecture 20: Topological sort algorithm

50

end

Detecting cycles

To implement the test “No more elements”, keep count of the processed elements and, at the end, compare it with th i i l b f l t the original number of elements.

• Intro. to Programming, lecture 20: Topological sort algorithm

51

Data structures: summary

elements : ARRAY [G ]

• - Items subject to ordering constraints
• - (Replaces the original list)

successors : ARRAY [LINKED LIST [INTEGER]] successors : ARRAY [LINKED_LIST [INTEGER]]

• - Items that must appear after any given one

d RR Y [ N EGER] predecessor_count : ARRAY [INTEGER]

• - Number of items that must appear before
• - any given one

any given one candidates : STACK [INTEGER] It ith d

• - Items with no predecessor
• Intro. to Programming, lecture 20: Topological sort algorithm

52

Initialization

Must process all elements and constraints to create these data structures This is O (m + n) So is the rest of the algorithm

• Intro. to Programming, lecture 20: Topological sort algorithm

53

Compiling: a useful heuristics

The data structure, in the way it is given, is often not the most i t f ifi l ith i i appropriate for specific algorithmic processing To obtain an efficient algorithm you may need to turn it into a specially To obtain an efficient algorithm, you may need to turn it into a specially suited form ll “ l We may call this “compiling” the data Often the “compilation” (initialization) is as costly as the actual Often, the compilation (initialization) is as costly as the actual processing, or more, but that’s not a problem if justified by the overall cost decrease

• Intro. to Programming, lecture 20: Topological sort algorithm

54

Another lesson

It may be OK to duplicate information in our data structures:

successors: ARRAY [LINKED_LIST [INTEGER]]

• - Items that must appear after any given one

4 2 3

4 4

predecessor_count: ARRAY [INTEGER]

• - Number of items that must appear before

1

2 4

• - any given one

3 4

2 1

This is a simple space-time tradeoff

2 1

1

• Intro. to Programming, lecture 20: Topological sort algorithm

55

Key concepts

A very interesting algorithm, useful in many

applications

Mathematical basis: binary relations Remember binary relations & their properties Transitive closure, Reflexive transitive closure Algorithm: adapting the data structure is the key “Compilation”strategy Initialization can be as costly as processing Algorithm not enough: need API (convenient,

extendible, reusable)

This is the difference between algorithms and

f i i

• Intro. to Programming, lecture 20: Topological sort algorithm

56

software engineering

Software engineering lessons

Great algorithms are not enough We must provide a solution with a clear interface (API), easy to use Turn patterns into components

• Intro. to Programming, lecture 20: Topological sort algorithm

57

End of lecture 22

• Intro. to Programming, lecture 20: Topological sort algorithm

58