A general definition of the big oh notation for algorithm analysis - - PowerPoint PPT Presentation

▶
a general definition of the big oh notation for algorithm
SMART_READER_LITE
LIVE PREVIEW

A general definition of the big oh notation for algorithm analysis - - PowerPoint PPT Presentation

Sep 27, 2022 229 likes •1.25k views

Algorithm analysis O -notation Prevailing definition Implied properties A general definition of the big oh notation for algorithm analysis Kalle Rutanen Department of Mathematics Tampere University of Technology June 3, 2014 1 / 101

slide-1
SLIDE 1

Algorithm analysis O-notation Prevailing definition Implied properties

A general definition of the big oh notation for algorithm analysis

Kalle Rutanen

Department of Mathematics Tampere University of Technology

June 3, 2014

1 / 101

slide-2
SLIDE 2

Algorithm analysis O-notation Prevailing definition Implied properties Correctness analysis Complexity analysis

Algorithm analysis

2 / 101

slide-3
SLIDE 3

Algorithm analysis O-notation Prevailing definition Implied properties Correctness analysis Complexity analysis

Algorithm analysis

Algorithm An algorithm is a finite sequence of instructions for transforming data to another form, a process which can be followed with pen and paper.

3 / 101

slide-4
SLIDE 4

Algorithm analysis O-notation Prevailing definition Implied properties Correctness analysis Complexity analysis

Algorithm analysis

Algorithm An algorithm is a finite sequence of instructions for transforming data to another form, a process which can be followed with pen and paper. Example An algorithm could provide a way to sort a sequence of integers in increasing order. Then (0, 4, 2, 7) would be transformed to (0, 2, 4, 7).

4 / 101

slide-5
SLIDE 5

Algorithm analysis O-notation Prevailing definition Implied properties Correctness analysis Complexity analysis

Algorithm analysis

Algorithm An algorithm is a finite sequence of instructions for transforming data to another form, a process which can be followed with pen and paper. Example An algorithm could provide a way to sort a sequence of integers in increasing order. Then (0, 4, 2, 7) would be transformed to (0, 2, 4, 7). Algorithm analysis Algorithm analysis studies the correctness and complexity of a given algorithm.

5 / 101

slide-6
SLIDE 6

Algorithm analysis O-notation Prevailing definition Implied properties Correctness analysis Complexity analysis

Correctness analysis

6 / 101

slide-7
SLIDE 7

Algorithm analysis O-notation Prevailing definition Implied properties Correctness analysis Complexity analysis

Correctness analysis

Correctness analysis Does the algorithm do what it is claimed to do?

7 / 101

slide-8
SLIDE 8

Algorithm analysis O-notation Prevailing definition Implied properties Correctness analysis Complexity analysis

Correctness analysis

Correctness analysis Does the algorithm do what it is claimed to do? Example of correctness Prove that a given algorithm sorts any sequence of integers in increasing order, and does so in finite time for a finite sequence.

8 / 101

slide-9
SLIDE 9

Algorithm analysis O-notation Prevailing definition Implied properties Correctness analysis Complexity analysis

Complexity analysis

9 / 101

slide-10
SLIDE 10

Algorithm analysis O-notation Prevailing definition Implied properties Correctness analysis Complexity analysis

Complexity analysis

Complexity analysis How much time / memory does it take to follow the algorithm on a given input in the worst case / best case / average case (etc.)?

10 / 101

slide-11
SLIDE 11

Algorithm analysis O-notation Prevailing definition Implied properties Correctness analysis Complexity analysis

Complexity analysis

Complexity analysis How much time / memory does it take to follow the algorithm on a given input in the worst case / best case / average case (etc.)? Example of complexity Prove that a given algorithm never uses more than n(n − 1)/2 number of order-comparisons to sort any sequence of n integers.

11 / 101

slide-12
SLIDE 12

Algorithm analysis O-notation Prevailing definition Implied properties Definition Primitive properties Uniqueness and existence

O-notation

12 / 101

slide-13
SLIDE 13

Algorithm analysis O-notation Prevailing definition Implied properties Definition Primitive properties Uniqueness and existence

O-notation

Motivation Detailed complexity analysis is not interesting; big-picture scaling behaviour is.

13 / 101

slide-14
SLIDE 14

Algorithm analysis O-notation Prevailing definition Implied properties Definition Primitive properties Uniqueness and existence

O-notation

Motivation Detailed complexity analysis is not interesting; big-picture scaling behaviour is. Example A given algorithm to sort a sequence of n integers is analyzed to take 6n2 + 5n + 37 comparisons in the worst case.

14 / 101

slide-15
SLIDE 15

Algorithm analysis O-notation Prevailing definition Implied properties Definition Primitive properties Uniqueness and existence

O-notation

Motivation Detailed complexity analysis is not interesting; big-picture scaling behaviour is. Example A given algorithm to sort a sequence of n integers is analyzed to take 6n2 + 5n + 37 comparisons in the worst case. Scaling behavior This is sometimes interesting. However, more interesting is that the algorithm’s complexity scales like n2.

15 / 101

slide-16
SLIDE 16

Algorithm analysis O-notation Prevailing definition Implied properties Definition Primitive properties Uniqueness and existence

O-notation

Motivation Detailed complexity analysis is not interesting; big-picture scaling behaviour is. Example A given algorithm to sort a sequence of n integers is analyzed to take 6n2 + 5n + 37 comparisons in the worst case. Scaling behavior This is sometimes interesting. However, more interesting is that the algorithm’s complexity scales like n2. Simplification The O-notation formalizes this scales-like simplification.

16 / 101

slide-17
SLIDE 17

Algorithm analysis O-notation Prevailing definition Implied properties Definition Primitive properties Uniqueness and existence

Definition

17 / 101

slide-18
SLIDE 18

Algorithm analysis O-notation Prevailing definition Implied properties Definition Primitive properties Uniqueness and existence

Definition

Definition An O-notation in a set X is a function OX : FX → P(FX), where FX = X → R≥0, such that it fulfills the primitive properties.

18 / 101

slide-19
SLIDE 19

Algorithm analysis O-notation Prevailing definition Implied properties Definition Primitive properties Uniqueness and existence

Definition

Definition An O-notation in a set X is a function OX : FX → P(FX), where FX = X → R≥0, such that it fulfills the primitive properties. Example To formalize our earlier example, (6n2 + 5n + 37) ∈ ON

  • n2

.

19 / 101

slide-20
SLIDE 20

Algorithm analysis O-notation Prevailing definition Implied properties Definition Primitive properties Uniqueness and existence

Definition

Definition An O-notation in a set X is a function OX : FX → P(FX), where FX = X → R≥0, such that it fulfills the primitive properties. Example To formalize our earlier example, (6n2 + 5n + 37) ∈ ON

  • n2

. Intuition The set OX(f ) contains those functions in FX which do not scale worse than f .

20 / 101

slide-21
SLIDE 21

Algorithm analysis O-notation Prevailing definition Implied properties Definition Primitive properties Uniqueness and existence

Separate concepts The O-notation used in pure mathematics is a concept separate from the O-notation in algorithm analysis; they have different properties.

21 / 101

slide-22
SLIDE 22

Algorithm analysis O-notation Prevailing definition Implied properties Definition Primitive properties Uniqueness and existence

Positive scale-invariance

22 / 101

slide-23
SLIDE 23

Algorithm analysis O-notation Prevailing definition Implied properties Definition Primitive properties Uniqueness and existence

Positive scale-invariance

Definition OX(αf ) = OX(f ) (∀α ∈ R>0, f ∈ FX)

23 / 101

slide-24
SLIDE 24

Algorithm analysis O-notation Prevailing definition Implied properties Definition Primitive properties Uniqueness and existence

Positive scale-invariance

Definition OX(αf ) = OX(f ) (∀α ∈ R>0, f ∈ FX) Intuition Positive constant factors are not interesting when comparing scaling behaviour.

24 / 101

slide-25
SLIDE 25

Algorithm analysis O-notation Prevailing definition Implied properties Definition Primitive properties Uniqueness and existence

Positive scale-invariance

Definition OX(αf ) = OX(f ) (∀α ∈ R>0, f ∈ FX) Intuition Positive constant factors are not interesting when comparing scaling behaviour. Example ON

  • 6n2

= ON

  • n2

25 / 101

slide-26
SLIDE 26

Algorithm analysis O-notation Prevailing definition Implied properties Definition Primitive properties Uniqueness and existence

Reflexivity

26 / 101

slide-27
SLIDE 27

Algorithm analysis O-notation Prevailing definition Implied properties Definition Primitive properties Uniqueness and existence

Reflexivity

Definition f ∈ OX(f ) ∀f ∈ FX

27 / 101

slide-28
SLIDE 28

Algorithm analysis O-notation Prevailing definition Implied properties Definition Primitive properties Uniqueness and existence

Reflexivity

Definition f ∈ OX(f ) ∀f ∈ FX Intuition A function does not scale worse than itself.

28 / 101

slide-29
SLIDE 29

Algorithm analysis O-notation Prevailing definition Implied properties Definition Primitive properties Uniqueness and existence

Reflexivity

Definition f ∈ OX(f ) ∀f ∈ FX Intuition A function does not scale worse than itself. Example n ∈ ON(n)

29 / 101

slide-30
SLIDE 30

Algorithm analysis O-notation Prevailing definition Implied properties Definition Primitive properties Uniqueness and existence

Transitivity

30 / 101

slide-31
SLIDE 31

Algorithm analysis O-notation Prevailing definition Implied properties Definition Primitive properties Uniqueness and existence

Transitivity

Definition f ∈ OX(g) and g ∈ OX(h) ⇒ f ∈ OX(h) ∀f , g, h ∈ FX

31 / 101

slide-32
SLIDE 32

Algorithm analysis O-notation Prevailing definition Implied properties Definition Primitive properties Uniqueness and existence

Transitivity

Definition f ∈ OX(g) and g ∈ OX(h) ⇒ f ∈ OX(h) ∀f , g, h ∈ FX Intuition If f does not scale worse than g, and g does not scale worse than h, then f does not scale worse than h.

32 / 101

slide-33
SLIDE 33

Algorithm analysis O-notation Prevailing definition Implied properties Definition Primitive properties Uniqueness and existence

Transitivity

Definition f ∈ OX(g) and g ∈ OX(h) ⇒ f ∈ OX(h) ∀f , g, h ∈ FX Intuition If f does not scale worse than g, and g does not scale worse than h, then f does not scale worse than h. Example n ∈ ON

  • n2

and n2 ∈ ON

  • n3

⇒ n ∈ ON

  • n3

.

33 / 101

slide-34
SLIDE 34

Algorithm analysis O-notation Prevailing definition Implied properties Definition Primitive properties Uniqueness and existence

Order-consistency

34 / 101

slide-35
SLIDE 35

Algorithm analysis O-notation Prevailing definition Implied properties Definition Primitive properties Uniqueness and existence

Order-consistency

Definition f ≤ g ⇒ OX(f ) ⊂ OX(g) ∀f , g ∈ FX

35 / 101

slide-36
SLIDE 36

Algorithm analysis O-notation Prevailing definition Implied properties Definition Primitive properties Uniqueness and existence

Order-consistency

Definition f ≤ g ⇒ OX(f ) ⊂ OX(g) ∀f , g ∈ FX Intuition If all values of f are at most that of g, then those functions which do not scale worse than f do not scale worse than g either

36 / 101

slide-37
SLIDE 37

Algorithm analysis O-notation Prevailing definition Implied properties Definition Primitive properties Uniqueness and existence

Order-consistency

Definition f ≤ g ⇒ OX(f ) ⊂ OX(g) ∀f , g ∈ FX Intuition If all values of f are at most that of g, then those functions which do not scale worse than f do not scale worse than g either Examples

37 / 101

slide-38
SLIDE 38

Algorithm analysis O-notation Prevailing definition Implied properties Definition Primitive properties Uniqueness and existence

Order-consistency

Definition f ≤ g ⇒ OX(f ) ⊂ OX(g) ∀f , g ∈ FX Intuition If all values of f are at most that of g, then those functions which do not scale worse than f do not scale worse than g either Examples OX(0) ⊂ OX(f ) ∀f ∈ FX

38 / 101

slide-39
SLIDE 39

Algorithm analysis O-notation Prevailing definition Implied properties Definition Primitive properties Uniqueness and existence

Order-consistency

Definition f ≤ g ⇒ OX(f ) ⊂ OX(g) ∀f , g ∈ FX Intuition If all values of f are at most that of g, then those functions which do not scale worse than f do not scale worse than g either Examples OX(0) ⊂ OX(f ) ∀f ∈ FX OR≥1(xα) ⊂ OR≥1

  • xβ

∀α, β ∈ R≥0 : α ≤ β

39 / 101

slide-40
SLIDE 40

Algorithm analysis O-notation Prevailing definition Implied properties Definition Primitive properties Uniqueness and existence

Order-consistency

Definition f ≤ g ⇒ OX(f ) ⊂ OX(g) ∀f , g ∈ FX Intuition If all values of f are at most that of g, then those functions which do not scale worse than f do not scale worse than g either Examples OX(0) ⊂ OX(f ) ∀f ∈ FX OR≥1(xα) ⊂ OR≥1

  • xβ

∀α, β ∈ R≥0 : α ≤ β OR≥1

  • 1

αe loge(β) logβ(x)

  • ⊂ OR≥1(xα)

∀α, β ∈ R>0

40 / 101

slide-41
SLIDE 41

Algorithm analysis O-notation Prevailing definition Implied properties Definition Primitive properties Uniqueness and existence

Multiplicativity

41 / 101

slide-42
SLIDE 42

Algorithm analysis O-notation Prevailing definition Implied properties Definition Primitive properties Uniqueness and existence

Multiplicativity

Definition OX(f )OX(g) = OX(fg) ∀f , g ∈ FX

42 / 101

slide-43
SLIDE 43

Algorithm analysis O-notation Prevailing definition Implied properties Definition Primitive properties Uniqueness and existence

Multiplicativity

Definition OX(f )OX(g) = OX(fg) ∀f , g ∈ FX Intuition The product of a function which does not scale worse than f and a function which does not scale worse than g does not scale worse than fg. Every function in OX(fg) is a product of such functions.

43 / 101

slide-44
SLIDE 44

Algorithm analysis O-notation Prevailing definition Implied properties Definition Primitive properties Uniqueness and existence

Multiplicativity

Definition OX(f )OX(g) = OX(fg) ∀f , g ∈ FX Intuition The product of a function which does not scale worse than f and a function which does not scale worse than g does not scale worse than fg. Every function in OX(fg) is a product of such functions. Examples

44 / 101

slide-45
SLIDE 45

Algorithm analysis O-notation Prevailing definition Implied properties Definition Primitive properties Uniqueness and existence

Multiplicativity

Definition OX(f )OX(g) = OX(fg) ∀f , g ∈ FX Intuition The product of a function which does not scale worse than f and a function which does not scale worse than g does not scale worse than fg. Every function in OX(fg) is a product of such functions. Examples ON(n)ON

  • n2

= ON

  • n3

45 / 101

slide-46
SLIDE 46

Algorithm analysis O-notation Prevailing definition Implied properties Definition Primitive properties Uniqueness and existence

Multiplicativity

Definition OX(f )OX(g) = OX(fg) ∀f , g ∈ FX Intuition The product of a function which does not scale worse than f and a function which does not scale worse than g does not scale worse than fg. Every function in OX(fg) is a product of such functions. Examples ON(n)ON

  • n2

= ON

  • n3

OR>0(1/x)OR>0(x) = OR>0(1)

46 / 101

slide-47
SLIDE 47

Algorithm analysis O-notation Prevailing definition Implied properties Definition Primitive properties Uniqueness and existence

Locality

47 / 101

slide-48
SLIDE 48

Algorithm analysis O-notation Prevailing definition Implied properties Definition Primitive properties Uniqueness and existence

Locality

Definition f ∈ OX(g) ⇔ ∀D ∈ C : (f |D) ∈ OD(g|D) ∀f , g ∈ FX, C ⊂ P(X) : C is a finite cover of X.

48 / 101

slide-49
SLIDE 49

Algorithm analysis O-notation Prevailing definition Implied properties Definition Primitive properties Uniqueness and existence

Locality

Definition f ∈ OX(g) ⇔ ∀D ∈ C : (f |D) ∈ OD(g|D) ∀f , g ∈ FX, C ⊂ P(X) : C is a finite cover of X. Intuition An f does not scale worse than g if and only if that holds when restricted to a set of a finite cover, for all such sets.

49 / 101

slide-50
SLIDE 50

Algorithm analysis O-notation Prevailing definition Implied properties Definition Primitive properties Uniqueness and existence

Zero-separation

50 / 101

slide-51
SLIDE 51

Algorithm analysis O-notation Prevailing definition Implied properties Definition Primitive properties Uniqueness and existence

Zero-separation

Definition OX(1) ⊂ OX(0)

51 / 101

slide-52
SLIDE 52

Algorithm analysis O-notation Prevailing definition Implied properties Definition Primitive properties Uniqueness and existence

Zero-separation

Definition OX(1) ⊂ OX(0) Intuition There exists a function which scales worse than 0, but does not scale worse than 1.

52 / 101

slide-53
SLIDE 53

Algorithm analysis O-notation Prevailing definition Implied properties Definition Primitive properties Uniqueness and existence

One-separation

53 / 101

slide-54
SLIDE 54

Algorithm analysis O-notation Prevailing definition Implied properties Definition Primitive properties Uniqueness and existence

One-separation

Definition ON>0(n) ⊂ ON>0(1)

54 / 101

slide-55
SLIDE 55

Algorithm analysis O-notation Prevailing definition Implied properties Definition Primitive properties Uniqueness and existence

One-separation

Definition ON>0(n) ⊂ ON>0(1) Intuition There exists a function which scales worse than 1, but does not scale worse than n.

55 / 101

slide-56
SLIDE 56

Algorithm analysis O-notation Prevailing definition Implied properties Definition Primitive properties Uniqueness and existence

Composition rule

56 / 101

slide-57
SLIDE 57

Algorithm analysis O-notation Prevailing definition Implied properties Definition Primitive properties Uniqueness and existence

Composition rule

Definition OX(f ) ◦ s ⊂ OY (f ◦ s) ∀f ∈ FX, s : Y → X.

57 / 101

slide-58
SLIDE 58

Algorithm analysis O-notation Prevailing definition Implied properties Definition Primitive properties Uniqueness and existence

Composition rule

Definition OX(f ) ◦ s ⊂ OY (f ◦ s) ∀f ∈ FX, s : Y → X. Intuition A function which does not scale worse than f , mapped through s, does not scale worse than f mapped through s.

58 / 101

slide-59
SLIDE 59

Algorithm analysis O-notation Prevailing definition Implied properties Definition Primitive properties Uniqueness and existence

Uniqueness and existence

59 / 101

slide-60
SLIDE 60

Algorithm analysis O-notation Prevailing definition Implied properties Definition Primitive properties Uniqueness and existence

Uniqueness and existence

Primitive properties The given properties are called the primitive properties.

60 / 101

slide-61
SLIDE 61

Algorithm analysis O-notation Prevailing definition Implied properties Definition Primitive properties Uniqueness and existence

Uniqueness and existence

Primitive properties The given properties are called the primitive properties. Existence There exists a function O with the primitive properties.

61 / 101

slide-62
SLIDE 62

Algorithm analysis O-notation Prevailing definition Implied properties Definition Primitive properties Uniqueness and existence

Uniqueness and existence

Primitive properties The given properties are called the primitive properties. Existence There exists a function O with the primitive properties. Uniqueness There exists at most one function O with the primitive properties.

62 / 101

slide-63
SLIDE 63

Algorithm analysis O-notation Prevailing definition Implied properties Definition Primitive properties Uniqueness and existence

Uniqueness and existence

Primitive properties The given properties are called the primitive properties. Existence There exists a function O with the primitive properties. Uniqueness There exists at most one function O with the primitive properties. Explicit definition f ∈ OX(g) :⇔ ∃c ∈ R>0 : f ≤ cg

63 / 101

slide-64
SLIDE 64

Algorithm analysis O-notation Prevailing definition Implied properties Definition An example of failure Characterization of failure

Prevailing definition

64 / 101

slide-65
SLIDE 65

Algorithm analysis O-notation Prevailing definition Implied properties Definition An example of failure Characterization of failure

Prevailing definition

Definition f ∈ OX(g) :⇔ ∃c ∈ R>0, y ∈ X : (f |X ≥y) ≤ c(g|X ≥y) where X ⊂ Rd and d ∈ N.

65 / 101

slide-66
SLIDE 66

Algorithm analysis O-notation Prevailing definition Implied properties Definition An example of failure Characterization of failure

Prevailing definition

Definition f ∈ OX(g) :⇔ ∃c ∈ R>0, y ∈ X : (f |X ≥y) ≤ c(g|X ≥y) where X ⊂ Rd and d ∈ N. Problem Our definition is different to the prevailing definition, which has been used for decades. Is there something wrong with the prevailing definition?

66 / 101

slide-67
SLIDE 67

Algorithm analysis O-notation Prevailing definition Implied properties Definition An example of failure Characterization of failure

Prevailing definition

Definition f ∈ OX(g) :⇔ ∃c ∈ R>0, y ∈ X : (f |X ≥y) ≤ c(g|X ≥y) where X ⊂ Rd and d ∈ N. Problem Our definition is different to the prevailing definition, which has been used for decades. Is there something wrong with the prevailing definition? Solution The prevailing definition fulfills all of the primitive properties, except for the composition rule!

67 / 101

slide-68
SLIDE 68

Algorithm analysis O-notation Prevailing definition Implied properties Definition An example of failure Characterization of failure

Fundamental The composition rule is fundamental; without it the complexity analysis of an algorithm cannot be approached by dividing it into parts, and studying the complexity of each part.

68 / 101

slide-69
SLIDE 69

Algorithm analysis O-notation Prevailing definition Implied properties Definition An example of failure Characterization of failure

An example of failure

69 / 101

slide-70
SLIDE 70

Algorithm analysis O-notation Prevailing definition Implied properties Definition An example of failure Characterization of failure

An example of failure

Algorithm 2 An algorithm which takes as input (m, n) ∈ N2, and has time-complexity ON2(1) according to the prevailing definition.

1: procedure constantComplexity(m, n) 2:

j ← 0

3:

if m = 0 then

4:

for i ∈ [0, n] do

5:

j ← j + 1

6:

end for

7:

end if

8:

return j

9: end procedure

70 / 101

slide-71
SLIDE 71

Algorithm analysis O-notation Prevailing definition Implied properties Definition An example of failure Characterization of failure

Algorithm 3 An algorithm which takes as input n ∈ N, and calls another ON2(1) algorithm n times with varying arguments.

1: procedure basicAnalysis(n) 2:

for i ∈ [0, n) do

3:

constantComplexity(0, i)

4:

end for

5: end procedure

71 / 101

slide-72
SLIDE 72

Algorithm analysis O-notation Prevailing definition Implied properties Definition An example of failure Characterization of failure

Algorithm 4 An algorithm which takes as input n ∈ N, and calls another ON2(1) algorithm n times with varying arguments.

1: procedure basicAnalysis(n) 2:

for i ∈ [0, n) do

3:

constantComplexity(0, i)

4:

end for

5: end procedure

Composition Computed via the composition rule, the complexity of this algorithm is ON(n).

72 / 101

slide-73
SLIDE 73

Algorithm analysis O-notation Prevailing definition Implied properties Definition An example of failure Characterization of failure

Algorithm 5 An algorithm which takes as input n ∈ N, and calls another ON2(1) algorithm n times with varying arguments.

1: procedure basicAnalysis(n) 2:

for i ∈ [0, n) do

3:

constantComplexity(0, i)

4:

end for

5: end procedure

Composition Computed via the composition rule, the complexity of this algorithm is ON(n). Substitution Computed via substitution, the complexity of this algorithm is ON

  • n2

. A contradiction!

73 / 101

slide-74
SLIDE 74

Algorithm analysis O-notation Prevailing definition Implied properties Definition An example of failure Characterization of failure

Characterization of failure

74 / 101

slide-75
SLIDE 75

Algorithm analysis O-notation Prevailing definition Implied properties Definition An example of failure Characterization of failure

Characterization of failure

Theorem (Asymptotic composition rule) Let X ⊂ Rd1, Y ⊂ Rd2, and s : Y → X. The composition rule holds for s under the prevailing definition if and only if ∀x∗ ∈ X : ∃y∗ ∈ Y : s(Y ≥y∗) ⊂ X ≥x∗. (1)

75 / 101

slide-76
SLIDE 76

Algorithm analysis O-notation Prevailing definition Implied properties Definition An example of failure Characterization of failure

Characterization of failure

Theorem (Asymptotic composition rule) Let X ⊂ Rd1, Y ⊂ Rd2, and s : Y → X. The composition rule holds for s under the prevailing definition if and only if ∀x∗ ∈ X : ∃y∗ ∈ Y : s(Y ≥y∗) ⊂ X ≥x∗. (1) Subset-sum Since the subset-sum rule implies the composition rule, the former does not hold for the prevailing definition either.

76 / 101

slide-77
SLIDE 77

Algorithm analysis O-notation Prevailing definition Implied properties Definition An example of failure Characterization of failure

Characterization of failure

Theorem (Asymptotic composition rule) Let X ⊂ Rd1, Y ⊂ Rd2, and s : Y → X. The composition rule holds for s under the prevailing definition if and only if ∀x∗ ∈ X : ∃y∗ ∈ Y : s(Y ≥y∗) ⊂ X ≥x∗. (1) Subset-sum Since the subset-sum rule implies the composition rule, the former does not hold for the prevailing definition either. Subset-sum or composition Actually, assuming the other properties, the composition rule and the subset-sum rule are equivalent.

77 / 101

slide-78
SLIDE 78

Algorithm analysis O-notation Prevailing definition Implied properties

Implied properties

78 / 101

slide-79
SLIDE 79

Algorithm analysis O-notation Prevailing definition Implied properties

Implied properties

More properties The following properties are implied by the primitive properties.

79 / 101

slide-80
SLIDE 80

Algorithm analysis O-notation Prevailing definition Implied properties

Implied properties

More properties The following properties are implied by the primitive properties. Implied properties

80 / 101

slide-81
SLIDE 81

Algorithm analysis O-notation Prevailing definition Implied properties

Implied properties

More properties The following properties are implied by the primitive properties. Implied properties Monotonicity: OX(OX(f )) ⊃ OX(f )

81 / 101

slide-82
SLIDE 82

Algorithm analysis O-notation Prevailing definition Implied properties

Implied properties

More properties The following properties are implied by the primitive properties. Implied properties Monotonicity: OX(OX(f )) ⊃ OX(f ) Idempotence: OX(OX(f )) = OX(f )

82 / 101

slide-83
SLIDE 83

Algorithm analysis O-notation Prevailing definition Implied properties

Implied properties

More properties The following properties are implied by the primitive properties. Implied properties Monotonicity: OX(OX(f )) ⊃ OX(f ) Idempotence: OX(OX(f )) = OX(f ) Membership rule: f ∈ OX(g) ⇔ OX(f ) ⊂ OX(g)

83 / 101

slide-84
SLIDE 84

Algorithm analysis O-notation Prevailing definition Implied properties

Implied properties

More properties The following properties are implied by the primitive properties. Implied properties Monotonicity: OX(OX(f )) ⊃ OX(f ) Idempotence: OX(OX(f )) = OX(f ) Membership rule: f ∈ OX(g) ⇔ OX(f ) ⊂ OX(g) Bounded translation-invariance: (∃β ∈ R>0 : f ≥ β) ⇒ OX(f + α) = OX(f )

84 / 101

slide-85
SLIDE 85

Algorithm analysis O-notation Prevailing definition Implied properties

Implied properties

More properties The following properties are implied by the primitive properties. Implied properties Monotonicity: OX(OX(f )) ⊃ OX(f ) Idempotence: OX(OX(f )) = OX(f ) Membership rule: f ∈ OX(g) ⇔ OX(f ) ⊂ OX(g) Bounded translation-invariance: (∃β ∈ R>0 : f ≥ β) ⇒ OX(f + α) = OX(f ) Positive homogenuity: αOX(f ) = OX(αf )

85 / 101

slide-86
SLIDE 86

Algorithm analysis O-notation Prevailing definition Implied properties

Implied properties

More properties The following properties are implied by the primitive properties. Implied properties Monotonicity: OX(OX(f )) ⊃ OX(f ) Idempotence: OX(OX(f )) = OX(f ) Membership rule: f ∈ OX(g) ⇔ OX(f ) ⊂ OX(g) Bounded translation-invariance: (∃β ∈ R>0 : f ≥ β) ⇒ OX(f + α) = OX(f ) Positive homogenuity: αOX(f ) = OX(αf ) Positive power-homogenuity: OX(f )α = OX(f α)

86 / 101

slide-87
SLIDE 87

Algorithm analysis O-notation Prevailing definition Implied properties

Implied properties

More properties The following properties are implied by the primitive properties. Implied properties Monotonicity: OX(OX(f )) ⊃ OX(f ) Idempotence: OX(OX(f )) = OX(f ) Membership rule: f ∈ OX(g) ⇔ OX(f ) ⊂ OX(g) Bounded translation-invariance: (∃β ∈ R>0 : f ≥ β) ⇒ OX(f + α) = OX(f ) Positive homogenuity: αOX(f ) = OX(αf ) Positive power-homogenuity: OX(f )α = OX(f α) Additive consistency: uOX(f ) + vOX(f ) = (u + v)OX(f )

87 / 101

slide-88
SLIDE 88

Algorithm analysis O-notation Prevailing definition Implied properties

Implied properties

More properties The following properties are implied by the primitive properties. Implied properties Monotonicity: OX(OX(f )) ⊃ OX(f ) Idempotence: OX(OX(f )) = OX(f ) Membership rule: f ∈ OX(g) ⇔ OX(f ) ⊂ OX(g) Bounded translation-invariance: (∃β ∈ R>0 : f ≥ β) ⇒ OX(f + α) = OX(f ) Positive homogenuity: αOX(f ) = OX(αf ) Positive power-homogenuity: OX(f )α = OX(f α) Additive consistency: uOX(f ) + vOX(f ) = (u + v)OX(f ) Multiplicative consistency: OX(f )uOX(f )v = OX(f )u+v

88 / 101

slide-89
SLIDE 89

Algorithm analysis O-notation Prevailing definition Implied properties

Implied properties

More implied properties

89 / 101

slide-90
SLIDE 90

Algorithm analysis O-notation Prevailing definition Implied properties

Implied properties

More implied properties Maximum-consistency: max(OX(f ), OX(f )) = OX(f )

90 / 101

slide-91
SLIDE 91

Algorithm analysis O-notation Prevailing definition Implied properties

Implied properties

More implied properties Maximum-consistency: max(OX(f ), OX(f )) = OX(f ) Restriction rule: (OX(f )|D) = OD(f |D)

91 / 101

slide-92
SLIDE 92

Algorithm analysis O-notation Prevailing definition Implied properties

Implied properties

More implied properties Maximum-consistency: max(OX(f ), OX(f )) = OX(f ) Restriction rule: (OX(f )|D) = OD(f |D) Additivity: OX(f ) + OX(g) = OX(f + g)

92 / 101

slide-93
SLIDE 93

Algorithm analysis O-notation Prevailing definition Implied properties

Implied properties

More implied properties Maximum-consistency: max(OX(f ), OX(f )) = OX(f ) Restriction rule: (OX(f )|D) = OD(f |D) Additivity: OX(f ) + OX(g) = OX(f + g) Maximum rule: max(OX(f ), OX(g)) = OX(max(f , g))

93 / 101

slide-94
SLIDE 94

Algorithm analysis O-notation Prevailing definition Implied properties

Implied properties

More implied properties Maximum-consistency: max(OX(f ), OX(f )) = OX(f ) Restriction rule: (OX(f )|D) = OD(f |D) Additivity: OX(f ) + OX(g) = OX(f + g) Maximum rule: max(OX(f ), OX(g)) = OX(max(f , g)) Summation rule: OX(f + g) = OX(max(f , g))

94 / 101

slide-95
SLIDE 95

Algorithm analysis O-notation Prevailing definition Implied properties

Implied properties

More implied properties Maximum-consistency: max(OX(f ), OX(f )) = OX(f ) Restriction rule: (OX(f )|D) = OD(f |D) Additivity: OX(f ) + OX(g) = OX(f + g) Maximum rule: max(OX(f ), OX(g)) = OX(max(f , g)) Summation rule: OX(f + g) = OX(max(f , g)) Maximum-sum rule: max(OX(f ), OX(g)) = OX(f ) + OX(g)

95 / 101

slide-96
SLIDE 96

Algorithm analysis O-notation Prevailing definition Implied properties

Implied properties

More implied properties Maximum-consistency: max(OX(f ), OX(f )) = OX(f ) Restriction rule: (OX(f )|D) = OD(f |D) Additivity: OX(f ) + OX(g) = OX(f + g) Maximum rule: max(OX(f ), OX(g)) = OX(max(f , g)) Summation rule: OX(f + g) = OX(max(f , g)) Maximum-sum rule: max(OX(f ), OX(g)) = OX(f ) + OX(g) Injective composition rule: OX(f ) ◦ s = OY (f ◦ s) (s injective)

96 / 101

slide-97
SLIDE 97

Algorithm analysis O-notation Prevailing definition Implied properties

Implied properties

More implied properties Maximum-consistency: max(OX(f ), OX(f )) = OX(f ) Restriction rule: (OX(f )|D) = OD(f |D) Additivity: OX(f ) + OX(g) = OX(f + g) Maximum rule: max(OX(f ), OX(g)) = OX(max(f , g)) Summation rule: OX(f + g) = OX(max(f , g)) Maximum-sum rule: max(OX(f ), OX(g)) = OX(f ) + OX(g) Injective composition rule: OX(f ) ◦ s = OY (f ◦ s) (s injective) Subset-sum rule: f ∈ OY (g) ⇒

  • x →

y∈S(x) f (y)

  • ∈ OX
  • y∈S(x) g(y)
  • 97 / 101
slide-98
SLIDE 98

Algorithm analysis O-notation Prevailing definition Implied properties

Prevailing definition The prevailing definition fulfills all of the implied properties, except for the subset-sum rule, and the injective composition rule.

98 / 101

slide-99
SLIDE 99

Algorithm analysis O-notation Prevailing definition Implied properties

Relation definitions

Given the O-notation OX : FX → P(FX), we define the relations , , ≺, ≻, ≈ ⊂ (FX × FX) such that g f ⇔ g ∈ OX(f ), g f ⇔ f ∈ OX(g), g ≺ f ⇔ f ∈ OX(g) and g ∈ OX(f ), g ≻ f ⇔ f ∈ OX(g) and g ∈ OX(f ), g ≈ f ⇔ f ∈ OX(g) and g ∈ OX(f ), (2) for all f , g ∈ FX.

99 / 101

slide-100
SLIDE 100

Algorithm analysis O-notation Prevailing definition Implied properties

Relation definitions

Using these relations, we define the traditional related notations ΩX, oX, ωX, ΘX : FX → P(FX) such that ΩX(f ) := {g ∈ FX : g f },

  • X(f ) := {g ∈ FX : g ≺ f },

ωX(f ) := {g ∈ FX : g ≻ f }, ΘX(f ) := {g ∈ FX : g ≈ f }, (3) for all f ∈ FX.

100 / 101

slide-101
SLIDE 101

Algorithm analysis O-notation Prevailing definition Implied properties

The end. Questions? :)

101 / 101