Lecture 3 Big-O notation, more recurrences!! Announcements! HW1 is - - PowerPoint PPT Presentation
Lecture 3 Big-O notation, more recurrences!! Announcements! HW1 is posted! (Due Friday) See Piazza for a list of HW clarifications First recitation section was this morning, theres another tomorrow (same material). (These are
Big-O notation, more recurrences!!
another tomorrow (same material). (These are
examples than we can get to in class).
You should be able to give a friend your solution and they should be able to turn it into a rigorous proof without much thought.
than “correctly” solving a math problem.
make mistakes sometimes. In this class, I’m trying to do:
(not python notation).
INSERTION SORT and MERGESORT.
correct!
time about n2
about n log(n).
200000 400000 600000 800000 1000000 200 400 600 800 1000
n log(n) n^2 nlog(n) is way better!!!
should take zero steps…why nlog(n)??
1 2 3 4 5 6 7 8
T(n) = T(n/2) + T(n/2) + T(MERGE)
Let’s say T(MERGE of size n/2) ≤ 6n
We will see later how to analyze recurrence relations like these automagically…but today we’ll do it from first principles.
T(n) = time to run MERGESORT on a list of size n T(MERGE two lists of size n/2) is the time to do:
So that’s 2T(assign) + n T(compare) + n T(assign) + 2n T(increment)
Plucky the pedantic penguin Lucky the lackadaisical lemur
= 2T(n/2) + 6n
This is called a recurrence relation: it describes the running time of a problem
smaller problems. Or 4n + 3…
should take zero steps…why nlog(n)??
make sense. Different operations take different amounts of time!
but that seems pretty dumb.
1 2 3 4 5 6 7 8
1 2 3 4 5 6 7 8
Sorting a sorted list should be fast!! Algorithm designer
Algorithm: Do the thing Do the stuff Return the answer
Here is my algorithm! Here is an input!
to solve the problem, in seconds or minutes or hours?
language, architecture, etc.
point of this class.
algorithm, independent of these considerations.
How long does an
we being so sloppy about that “6”?
n n log(n) n^2 8 24 64 16 64 256 32 160 1024 64 384 4096 128 896 16384 256 2048 65536 512 4608 262144 1024 10240 1048576
200000 400000 600000 800000 1000000 200 400 600 800 1000 1200
n log(n) n^2
n 5n log(n) n^2 8 120 64 16 320 256 32 800 1024 64 1920 4096 128 4480 16384 256 10240 65536 512 23040 262144 1024 51200 1048576
200000 400000 600000 800000 1000000 200 400 600 800 1000 1200
5n log(n) n^2
How does the running time scale as n gets large?
hardware- and language- specific issues.
much more tractable.
large (compared to the constant factors).
Pros: Cons:
One algorithm is “faster” than another if its runtime grows more “slowly” as n gets large.
2100000000000000 n is “better” than n2 ?!?!
This is especially relevant now, as data get bigger and bigger and bigger… This will provide a formal way of saying that n2 is “worse” than 100 n log(n).
that really excites the student.
that, ∀ students in CS161, the student is excited by the problem.
T(n) as n gets large.
𝑈 𝑜 = 𝑃 𝑔 𝑜 ⟺ ∃𝑑, 𝑜. > 0 𝑡. 𝑢. ∀𝑜 ≥ 𝑜., 0 ≤ 𝑈 𝑜 ≤ 𝑑 ⋅ 𝑔(𝑜)
pronounced “big-oh of …” or sometimes “oh of …”
𝑈 𝑜 = 𝑃 𝑔 𝑜 ⟺ ∃𝑑, 𝑜. > 0 𝑡. 𝑢. ∀𝑜 ≥ 𝑜., 0 ≤ 𝑈 𝑜 ≤ 𝑑 ⋅ 𝑔(𝑜) T(n) f(n) c f(n) n0 n T(n) = O(f(n)) means: Eventually, (for large enough n) something that grows like f(n) is always bigger than T(n).
f(n) n T(n)
c=1
n0 = 1 𝑈 𝑜 = 𝑃 𝑔 𝑜 ⟺ ∃𝑑, 𝑜. > 0 𝑡. 𝑢. ∀𝑜 ≥ 𝑜., 0 ≤ 𝑈 𝑜 ≤ 𝑑 ⋅ 𝑔(𝑜)
(formal proof
c f(n) =
leading coefficients are O(nk).
(On the board)
and n0 so that the definition is satisfied.
contradiction:
the definition is satisfied.
a contradiction.
O(…) means an upper bound, and
as T(n) as n gets large.
𝑈 𝑜 = Ω 𝑔 𝑜 ⟺ ∃𝑑, 𝑜. > 0 𝑡. 𝑢. ∀𝑜 ≥ 𝑜., 0 ≤ 𝑑 ⋅ 𝑔 𝑜 ≤ 𝑈 𝑜
Switched these!!
𝑈 𝑜 = Ω 𝑔 𝑜 ⟺ ∃𝑑, 𝑜. > 0 𝑡. 𝑢. ∀𝑜 ≥ 𝑜., 0 ≤ 𝑑 ⋅ 𝑔 𝑜 ≤ 𝑈 𝑜 T(n) c f(n) f(n) n0 n
Fun exercise: check all of these carefully!!
is a precise definition.
don’t have to pay close attention to all those pesky constant factors like “4” or “6”.
will be actually practical, without needing to take 𝑜 ≥ 𝑜. = 2:........
This is my happy face!
Questions about asymptotic notation?
We’ve seen three recursive algorithms so far.
T(n) = time to solve a problem of size n.
What’s the pattern?!?!?!?!
(Reminders on board)
recurrences when all of the sub-problems are the same size.
Wednesday when not all problems are the same size).
Jedi master Yoda
A useful formula it is. Know why it works you should.
= > + 𝑃 𝑜@ . Then
Many symbols those are….
Three parameters: a : number of subproblems b : factor by which input size shrinks d : need to do nd work to create all the subproblems and combine their solutions.
(details on board)
𝑈 𝑜 = 𝑏 ⋅ 𝑈
= > + 𝑃 𝑜@ .
a = 4 b = 2 d = 1 a = 3 b = 2 d = 1 a = 2 b = 2 d = 1
a > bd a > bd a = bd
MergeSort, but be more careful.
= > + 𝑑 ⋅ 𝑜@.
Plucky the Pedantic Penguin Hang on! The hypothesis of the Master Theorem was the the extra work at each level was O(nd). That’s NOT the same as work <= cn for some constant c. Lucky the lackadaisical lemur That’s true … we’ll actually prove a weaker statement that uses this hypothesis instead of the hypothesis that 𝑈 𝑜 = 𝑏 ⋅ 𝑈
= > + 𝑃 𝑜@ .
It’s a good exercise try to make this proof work rigorously with the O() notation.
Size n n/b n/b
(Size 1)
n/b2 n/bt n/bt n/bt n/bt n/bt n/bt
Level
Amount of work at this level # problems
1 2 t logb(n) 1 a a2 at alog_b(n)
Size of each problem
n n/b n/b2 n/bt 1
n/b n/b2 n/b2 n/b2 n/b2 n/b2 n/b2 𝑈 𝑜 = 𝑏 ⋅ 𝑈 𝑜 𝑐 + 𝑑 ⋅ 𝑜@
𝑑 ⋅ 𝑜𝑒
𝑏C𝑑 𝑜 𝑐C
@
𝑏𝑑 𝑜 𝑐
@
𝑏D𝑑 𝑜 𝑐D
@
𝑏EFG _>(=)𝑑
Size n n/b n/b
(Size 1)
n/b2 n/bt n/bt n/bt n/bt n/bt n/bt
Level
Amount of work at this level # problems
1 2 t logb(n) 1 a a2 at alog_b(n)
Size of each problem
n n/b n/b2 n/bt 1
n/b n/b2 n/b2 n/b2 n/b2 n/b2 n/b2 𝑈 𝑜 = 𝑏 ⋅ 𝑈 𝑜 𝑐 + 𝑑 ⋅ 𝑜@
𝑑 ⋅ 𝑜𝑒
𝑏C𝑑 𝑜 𝑐C
@
𝑏𝑑 𝑜 𝑐
@
𝑏D𝑑 𝑜 𝑐D
@
𝑏EFG _>(=)𝑑
Total work (derivation on board) is at most: 𝑑 ⋅ 𝑜@ ⋅ I 𝑏 𝑐@
D JKLM(=) DN.
problems are different sizes?
that happen?
nor f = Ω(g)?
the above is true?
F(1) = 1, F(n) = F(n-1) + F(n-2) for n > 2.
True or false:
𝑈 𝑜 = 𝑃 𝑔 𝑜 ⟺ ∃𝑑, 𝑜. > 0 𝑡. 𝑢. ∀𝑜 ≥ 𝑜., 0 ≤ 𝑈 𝑜 ≤ 𝑑 ⋅ 𝑔(𝑜) f(n) n0 = 1 n g(n) 2.1⋅ f(n)
1 2 2.1
𝑈 𝑜 = 𝑃 𝑔 𝑜 ⟺ ∃𝑑, 𝑜. > 0 𝑡. 𝑢. ∀𝑜 ≥ 𝑜., 0 ≤ 𝑈 𝑜 ≤ 𝑑 ⋅ 𝑔(𝑜) f(n) n g(n) 2.1⋅ f(n)
1 2 2.1
n0