Sorting Sorting - re - arranging elements of a sequence 5 st So I - - PowerPoint PPT Presentation

SLIDE 1

Sorting

SLIDE 2

Sorting

• re - arranging

elements of

a sequence

5

st

.

So I S , I Sz E

• I

Sn- I

• We

will

look at

5 sorting algorithms

:

• 3

iterative

• 2 recursive

SLIDE 3

Theiteratirealgorithn

• maintain a partition:

"unsortedt

" t 'short

"

• Sort

a sequence of

n

elements in

n-1

stages

. at each stage, move 1

element from the

unsorted

part to the sorted part :

" sorted " "unsorted "

• 1-

sort(A) {

✓

1 stage

moves 1 element

• initialize
• repeat

n- i times

}

move 1 element from unsorted to sorted part

• the algorithms differ in how they

:

• select an element to remove from the unsorted part
• insert

it into the sorted part

SLIDE 4

Insertion Sort

n-1 elements

• in unsorted part
• initially : sorted part is just AH
• repeat at times :

~

Sorted unsorted

• remove the first element

←

from the unsorted part

TI

• insert it into the sorted

t

part (shifting elements

,

to the right

as needed)

1¥

• -

sorted

unsorted insertion

• sort(A)

forli

• - 1 ton
• 1){

pivot

• _ Afilflfirst element in unsorted part

j=i

• I

whileljtoAUDALjtspir.DE/gssIIteaoYepakrIfthsat

"

A-Ljti]

• ALJIXshiftg.tt

are larger than pivot

"

"

}

J=j

• I

e- to the right .

} ALjttj-pivotkmorepirot.info position

.

SLIDE 5

insertiohnstExamol.es ④

• 1. start .

②

⑥

b stages

.

①

'II::

.

③

] stage 5-

I

SLIDE 6

Selection Sort

unsorted

• initially, sorted partempty

1-

• repeat n - 1 times

Existed

• findthe smallest element

in the unsorted part

¥

,

swap

• move it to the first position

i1

which becomes the new

• last position of sorted part

.

sorted b)

unsorted

this is its selection-sort (A)1

final location

for li=1 ton-1){

¥nnd [

I

• 1 "I

isindexofminfounds-fa.ie/emewtwhileCkan

){

in

if( ALKIAALJJ)j=k;

unsorted

k=ktt

{wap Ali-13 and ACJI

} }

SLIDE 7

selectionsortExamplel.SI#&)stase1

a) stash

• 5- 1 stage 3

##④ 1st

.

4

#¥s

g

, I st.sn

.

SLIDE 8

• Selections Heaps

takes 0.cn) time

Initiated partempty ]

• make unsorted part into a heap
• repeat n - 1 times
• findthe smallest element } heap extract

( vs . OLD for the

in the unsorted part

s takes

Logan)

scan in selection

• move it to the first position

time sort ) .

which becomes the new

last position of sorted part

.

Consider the organization of array contents :

①

¥4

1-

I

Z if this is the root of the heap , then

it is also the smallest element

in

the usorted

part , so is in its

correct final position

.

To

usethis

arrangement , the root of the

heap keeps moving ,

so we have

lots of shifting

to do

.

SLIDE 9

¥4

② T If this is the root of the heap , then everything

works :

• we extract
• ;

move the

last leaf

D to

the

root t do

a percolate

• down ;

store

• where

is

way ,

which is

now free , and

is the correct final

location to • , after

which

we

have:

E→←¥

÷#÷

Bet :

we must

re - code our heap implementation

s.tv the root

is at

A Ln

• D , with the result

. that the indexing

is now

less intuitive

.

③ Instead

, we use a

max - heap , and

this arrangement

:

unsorted

sorted

←

root of

heap I

Elas teat .

Now :

. the heap root is at

A Lot

• heap extraction removes •

, moves D to A Lot ,

freeing

up the spot where

• belongs

.

Leaving

us : -77

I

Re -coding

a min Heap

into

a

man heap

is just replacing

<

with

>

and vice

versa .

SLIDE 10

Sextons

Heapsort

unsorted

• initially, sorted partempty
• make unsorted part into a max heap

7- heap with max here

• repeat n - 1 times

largest

• findthe'ls#f"element

!!"

in the unsorted part

I

u

last

is

unsortejd-hsfted-move.it

to the first position

'

take last leaf

which becomes the new

from

' end' of heap

test position of sorted part

.

×

newest element

first

7¥

sorted

part

this is its ¥-4

new root of

heapsort(A){

final location

heap ( which then

buildmaxheap.LA)

gets percolated

for LE1 to

n-1) E

down)

ALn.is

• extract maxll

}

1-

• }

unsorted

heap , -

• f size 1 J

sorted

has smallest element .

SLIDE 11

Heapsort with in line

percolate

• down
• heapsort (A)I

makeMax Heap (A)

forCi= I to

n -1) {

A more last leaf to root

swap

AL03 and Ahn - i] 4 and old root to where K last leaf was .

gite ← n - it IN size of heap

= size of unsorted part

I ← 0

while ( 2J

t t

a site ){

• e. hied ← 2J

+ I

percolate

if ( 2J t 2 a site

AND A

jt

2] L

A L2J "3){

down

g.

child

← 2J t 2

it

In

,

asians

}

} else

} j

← size X terminate the while .

}

}

}

SLIDE 12

HeapsortExampj.FI#fpdmYniEp

g) stage '

I

z¥

↳

stages

[

* ' "

s

stages

I

• staged

¥¥#I

stages

SLIDE 13

Heapsortexamptef

.

Dma

"

stage

stages

÷÷÷÷:÷:÷±÷

.

12%

staged stages

SLIDE 14

TmelompkxityofIterativgAgithms

• each algorithm does exactly

n-1 stages

• the work done at the

ith stage varies with the algorithm (

k input)

. . we take

# of item comparisons

as a measure of

workAim

Selection Sort

• exactly

n-i

'

comparisons to find

min.

• element in unsorted part

Insertion sort

• between 1 and i comparisons to find
• location for pivot

H¥t

:

• between

land 26g. Ln- it ' )

comparisons

for percolate

• down

SLIDE 15

*

Number of comparisons

• We must verify

# comparisons for

some

constant

times # comparisons)

is an upper bound

• n

work done

by

each algorithm

.

• #of assignments (4 swaps )

also matters

in actual

run

time.

SLIDE 16

Selection

On input of size n , # of comparisons

is always

(regardless of input) :

Etait

= Iii = Sln

• i)

= In -1) (n)

2-

= n

2

= Etna)

SLIDE 17

Insertionsort-worstcesellpper-B.vn

d

:

# comparisons

E

i

= MI- = 0 (n') .

trowed :

Worst case : initial sequence is in reuerder

.

Eg

.

I

In the

the stage

we have

• -
• kiT

This takes

I

comparisons , because the

sorted part is of size i

.

So , # Comparisons

>

=D(ne ) So

, Insertion sort worst case

is

①( n')

SLIDE 18

Insertion Sort Best Case

• Best

case : initial sequence

is fully

• rdered

.

Then : In each stage exactly 1 comparison is made

.

So : # comparisons

= n - I

= ①(n)

.

SLIDE 19

tteapsortworstcaseupp.es

Bound :

# Comparisons I

. 52 log. Ln- it 1) =

2 log.Citi)

I 2

login

I 2h Logan

= Oln log n)

Lower Bound ?

• BestCas (what

input would

lead

to

movement

during percolate

• down?

What if we exclude this

case ? )

SLIDE 20

Recorsiredividekconguersorting.pe

rtiti onthe sequence A into two parts AI , Az

. Recursively sort each of A , and Ac

• Combine the sorted versions of A , and

Az to

• btain

a sorted version of A

Partitiony y \

T

• 1-

I

'

I

sort →

\

1=1

Combine h -\

• I

The algorithms differ in how they choose the

partition ,

and how they combine the

sorted parts

SLIDE 21

Mergesortn :

. . Uses the fact that merging two

sorted lists is easy

* "

TT

¥t

I

4

P

p

p

• Takes ON time

, where

n

is the total

size

SLIDE 22

Merge sort

• :
• partition

: first half

4 second

half

.

• Combine i merge the

parts

• ran;!q⇐¥¥¥¥*

merge 2 parts

If

3HH

• Works with linked - list

⇐ array implementations

. in array implementations,

uses

A Ln) extra space

SLIDE 23

MergesortmergesortfA.to

, hill

if (lol hi){ 1 there are 72 items , so work to do

mid ← Ll lo thi)121

merge sort ( A

, Io , mid)

merge sort

( A

, midt 1 , hi)

}

merge ( A , low , mid

, hi)

}

SLIDE 24

mergeformergesortme.ge/A,loimidihi)l After *

, the sorted

( ← to r←midt1

sequence

is

in

n ←

Lo

while ( llmid

AND

re hi)l

it AG3A AG3H

BIO]

. . . Bfhi] .

BLIND

:*:*.am

too

.

htt

3

swap

• A. B :

while( Lamia)E

B43 ← ALL] Htsntt

temp ← A

{hilecrahi)E

A A- B

'

BED ← AH

rttsntt

B * temp

3*3

SLIDE 25

tmeLomplexityofHergeSort_ viatreeofrecursireca.tl#

• n -

→ nf \

041

• N12-0LN)

us.

Noo

noo

Hoo

Hoo .

Yoo

Yoo Yoo

Yoo

'

: :

:

:

:

:

:

:

:

2 I

i.A

041

If

n is a power of 2 , the tree of recursive calls

is

a perfect binary

• tree with

n

leaves

,

and height login

. . At depth i there

are

2' calls to merge , each to merge two

lists

• f

site 4241 into

• ne
• f

site "hi

.

T

• tal work at depth i is 246

i)

= 0( n 'Ii ) = ON .

T

• tal work is #depths
• OLD

= login . OH =0(n log n) .

SLIDE 26

Quiiks

• Uses a pivot

to partition sequence into

"small

" and

" large

" etemeuts :

small elements

< p < large elements

• combining

sorted

versions

is trivial

→

• choose

a

# partition

r-

4

If

←

my

two parts

.

Tip

p

values IP values > p

in order.

in

• rder

.

• choosing pivots

is

key to

performance

.

SLIDE 27

Quickso

quicksort(A

, lo , hi){

if ( lo chi){ 4there are 72 items

pivot position ← partition (A. to ,hi) A partition

quick sort ( A , Io , pivotposition

• 1)

}

quick sort 1A , pivotposition tt , hi )

}

Quicksort

is correct

as long

as every call

to partitionD

returns

and

leaves the

variables satisfying the following

:

I.

to

E pivotposition ± hi

• 2. for every i.j with

lo ti E pivot position Eje hi

Ali Is A (pivot position] I A- Chi]

Howeve , efficiency relies critically

• n choice of pivot

.

SLIDE 28

E±PerfectPijt

• Suppose all

, elements

are distinct, and the pivot is chosen to be the

median element

in

Alloy

. . . Achi) .

Then

, every call to

Quicksort

• n

sequence of

size K 72

makes

two

recursive calls

• n sequences of

site EK12

:p

median

K

• -

I K12

4KK

• By essentially the

same argument

as used

for

Merge sort

, this

gives us running time of

log In)

. fan) , where fans

is the time

to

run partition

• n

a

sequence of

size n .

• Assuming

01in) time for partition , this would give

us

0cm log n)

time for Quick sort

. . But : finding

medians

is too

slow in practice

.

• Optional exercise : Can the median be found in %) time ?

SLIDE 29

Exiworstcasepirots.fi

Suppose all elements are distinct, and the

max is always chosen

as pirot .

• Then every call to Quicksort
• n a sequence of

K72

elements

makes

• ne recursive call
• n

a sequence of

size K- I ,

and

• ne
• n

a sequence of

site 0 :

K

• if i

f-

.

• The recursion tree

looks like this :

n -I

✓ to

• This tree

is of height ⑦n)

, n-2

/

T

• n-3

giving

us a running time -

• f 0h)

. fins ,

• r

T

②( h2)

assuming

.

• * for partition

.

I ' °

SLIDE 30

Partition

Partition must choose

a pivot p . and efficiently

re -arrange elements

partition( A.lu , hi){

pivot index ← choosePirot (A. to

, hi) 11 choose

pirot

swap

AL pivot index)

and

A Chi] 4 move pivot out of the way

.

p ← A Chi) Xp

is the

pivot

i ← Io

T known

"small " values will be at indices < i

too ( j

= to ; ja hi ; jtt){ A. "already inspected " values will

Xbe at

indices

< y

'

if I ALj3Ep)E

µ if we

are inspecting a

"small

"

swap

Ali)

and

A Cj] 4 swap it with first

"

non small

"

}

is it't

4

increase size of

"smalls " part .

{ wap

Ali )

and

A

Amore pivot where it belongs

.

}

return i 4 this is pivot position

A :.in?nownotinsejetp..j

Is

currently being inspected -

pivot

SLIDE 31

Partition Example

• known small : -3

known large:-.

,

• - -
• i ,

⑥ swap

pivot into hi ¥V_

' ÷

. VE?li4E

j

y sT--oTTTg_ y

④

Ali 's 44

,

AL5H 't

It

I =

%

# Afraid

i

• '

Afj3c4 I

i

• 5
• .

÷

.fi?94E

'

④ swap

ALDKALhilo-elfflj344IF-o.IE

Ij94

I

soso.oe-ooo.to

If Afj]c4

1

IT

=

,

TMATIT

large

iE

,

pivot

'

n

• '

SLIDE 32

• Partition

Time complexity of

partition is

①Ln) tgln)

,

where gln

)

is time taken by

choosePivot .

• Q

: How

can we choose

"

good

" pivots "fast

" ?

. Quick sort

is the most - used

sorting algorithm

in practise ,

so

there must

be

a way

. . But .

• what qualifies

as

"

fast

"

?

probably

a very small constant

• what qualifies

as

"

good

"

?

(given that it must be fast)

SLIDE 33

Consider :

IA.mg/lnumberofbadpirots

:

• -
• 5-
• -

tfh I

in dhEtIEh dhE

His

Its

isis

disks

HAITI

makes a small difference in height.tk#DDtYDB

• Perfect pivots are not needed for 04 login ) time

.

=

F- 1-

Eg

. if pivots are all

T1-1II

better than I:¥ ,

IIDIIDDD

then depth

is log%n D DD

DD

ND Fln

Dam

hmm

n n '

so we still get

DB

.

" "

0(

nlogn )

.

SLIDE 34

somebimpfeichoosepi-votoptions.tt

thi)

• fast , but performs badly
• n many inputs .

. Median

• perfect pivots,

but too slow to compute

• random

: . If pivots

are chosen uniformly at random , then

Quick sort

runs in time 0 (nlogn) with

probability

I

• Yan
• ie almost always

. . BI : good

random

numbers are not fast to make

. . median{ AID , Alni ] , A knit LD12] }

fast

. not very easy to

come up with a "

very

bad

" input .

SLIDE 35

(ompIexityofQuicso

• Depends critically
• n how pivots are chosen
• Choosing

" perfect " pivots is too slow

for

a

practical sorting algorithm

• Fortunately ,

choosing pivots that are

"

good enough

"

for mo-stinp-utso.am be done fast

.

• Quicksort
• with practical pivot choice strategies
• is this

,

but

is often described as

" like

0In login)

in practice

"

.

SLIDE 36

In pract

e

x.

There are settings where Merge sort

% Insertion sort are preferred

. In most settings , the preferred algorithm is

Quicksort

• For small sets , selection sort

is faster

• Often ,this variant for similar )

is faster :

quicksort (A

, lo , hi){

if ( lo chi){ 4there are 72 items

if( lot 15 a hi){ A less than 15 items

selectionsort(A.to

, hi)

} else {

pivot position ← partition (A , Io , hi) A partition

quick sort ( A , Io , pivotposition

• 1)

quick sort LA , pivotposition tt , hi )

}

} }

SLIDE 37

F-not