Adaptive Software Speculation for Enhancing the Cost-Efficiency of - - PowerPoint PPT Presentation

▶
adaptive software speculation
SMART_READER_LITE
LIVE PREVIEW

Adaptive Software Speculation for Enhancing the Cost-Efficiency of - - PowerPoint PPT Presentation

Oct 19, 2022 161 likes •639 views

Adaptive Software Speculation for Enhancing the Cost-Efficiency of Behavior-Oriented Parallelization Yunlian Jiang Xipeng Shen The College of William and Mary High-level Parallelism Parallel computing is becoming ubiquitous High-level

slide-1
SLIDE 1

Adaptive Software Speculation

for Enhancing the Cost-Efficiency of Behavior-Oriented Parallelization Yunlian Jiang

Xipeng Shen The College of William and Mary

slide-2
SLIDE 2

2

High-level Parallelism

 Parallel computing is becoming ubiquitous  High-level parallelism exists in many

programs

 E.g. utilities, interpreters, scientific computations  Difficult to parallelize

Bit-level operations, unrestricted pointers, exception handling, custom mem. management, third-party libraries Example*: while ( s=nextSentence() ) { parse(s); if ( isCommand(s) ) updateParsingEnv(s); } Complexity in the code Uncertain parallelism

slide-3
SLIDE 3

3

Software Behavior-Oriented Parallelization [Ding+:PLDI07]

 Speculatively execute programs in parallel  Efficiently detect dependence during runtime  But, blind speculation causes

cost-inefficiency.

slide-4
SLIDE 4

4

Cost and Speedup 1 2 3

1 0.9 0.42 0.12

Speculation Success Rate Cost SpeedUp

slide-5
SLIDE 5

5

Cost and Speedup

0.5 1 1.5 2

1 0.9 0.42 0.12 Speculation Success Rate

Cost SpeedUp

slide-6
SLIDE 6

6

Outline

 Introduction to BOP  Adaptive BOP  Experimental Result  Conclusion

slide-7
SLIDE 7

7

Outline

 Introduction to BOP  Adaptive BOP  Experimental Result  Conclusion

slide-8
SLIDE 8

8

Behavior-Oriented Parallelization (BOP)

 A tool for parallelizing sequential programs  Need no parallel programming or debugging  Basic scheme: software speculation  Correctness protected through runtime system

slide-9
SLIDE 9

9

Basic Scheme of BOP

. . . . . . While (S= ReadSentence()) { Parse(S); } . . . . . . Ctrl Main Spec

Possibly Parallel Re Region ( (PPR) PPR)

Two reason for failed speculation:

  • 1. Dependence violation
  • 2. Spec runs too slow

Parser BeginPPR(); EndPPR();

slide-10
SLIDE 10

10

Problem to tackle

 Cost inefficiency

 BOP blindly speculates every PPR instance  Failed speculation may

 Cause slowdown to applications

  • Protection overhead
  • Resource (cache, bus) contention

 Waste computing resources

  • CPU --- multi-programming environment
  • Power --- Mobile computing
slide-11
SLIDE 11

11

Outline

 Introduction to BOP  Adaptive BOP  Experimental Result  Conclusion

slide-12
SLIDE 12

12

Solution: Adaptive Speculation

 Basic strategy

 Predict profitability of PPR  Speculate only likely profitable ones

 Prediction approaches

 Extended last-value-based  Decayed-history-based

slide-13
SLIDE 13

13

Extended last-value-based prediction

 Speculate or not?

 Speculate only if PPRsToSkip == 0.

 Adjust PPRsToSkip

 If this PPR is not speculated

 PPRsToSkip - -

 On a failed speculation

 PPRsToSkip = NextPenalty;  NextPenalty *= α; (increase penalty exponentially)

 On a successful speculation

 NextPenalty = 1; (reset the penalty on the next failure)

slide-14
SLIDE 14

14

Extended last-value-based prediction

 | | |

| | | | | | | | | | | | | | |

  • PPRsToSkip=0
  • NextPenalty=1
  • α=2

Speculation! Success NextPenalty = 1

| : Profitable PPR | : Unprofitable PPR

slide-15
SLIDE 15

15

 | | |

| | | | | | | | | | | | | | |

  • PPRsToSkip=0
  • NextPenalty=1
  • α=2

Speculation! Failed PPRsToSkip = 1 NextPenalty = 2

Extended last-value-based prediction

| : Profitable PPR | : Unprofitable PPR

slide-16
SLIDE 16

16

 | | |

| | | | | | | | | | | | | | |

  • PPRsToSkip=1
  • NextPenalty=2
  • α=2

No Speculation! PPRsToSkip = 0

Extended last-value-based prediction

| : Profitable PPR | : Unprofitable PPR

slide-17
SLIDE 17

17

 | | |

| | | | | | | | | | | | | | |

  • PPRsToSkip=0
  • NextPenalty=2
  • α=2

Speculation! Failed PPRsToSkip = 2 NextPenalty = 4

Extended last-value-based prediction

| : Profitable PPR | : Unprofitable PPR

slide-18
SLIDE 18

18

 | | |

| | | | | | | | | | | | | | |

  • PPRsToSkip=2
  • NextPenalty=4
  • α=2

No Speculation! PPRsToSkip = 1

Extended last-value-based prediction

| : Profitable PPR | : Unprofitable PPR

slide-19
SLIDE 19

19

 | | |

| | | | | | | | | | | | | | |

  • PPRsToSkip=1
  • NextPenalty=4
  • α=2

No Speculation! PPRsToSkip = 0

Extended last-value-based prediction

| : Profitable PPR | : Unprofitable PPR

slide-20
SLIDE 20

20

 | | |

| | | | | | | | | | | | | | |

  • PPRsToSkip=0
  • NextPenalty=4
  • α=2

Speculation! Success NextPenalty = 1

Extended last-value-based prediction

| : Profitable PPR | : Unprofitable PPR

slide-21
SLIDE 21

21

 | | |

| | | | | | | | | | | | | | |

  • PPRsToSkip=0
  • NextPenalty=1
  • α=2

Speculation! Success NextPenalty = 1

...

Extended last-value-based prediction

| : Profitable PPR | : Unprofitable PPR

slide-22
SLIDE 22

22

Extended last-value-based prediction

 Limitations

 Can not keep history well

 Successful speculation

clean history

 Phase changes

|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||

slide-23
SLIDE 23

23

Decayed-history-based prediction

 Cumulative gain (CG)  cg = γ*g + (1-γ) * cg  Expected Profitability (EP)  EP = cg + SkippedPPRs *β  Speculate only if EP > THEP

g=

1 : success 0: failed

* THEP : threshold of speculation; * SkippedPPRs: reset to 0 on a success increase by 1 on a non-speculated PPR

slide-24
SLIDE 24

24

 | | |

| | | | | | | | | | | | | | |

  • THEP=0.6
  • β=0.2
  • γ = 0.5
  • SkippedPPRs=0
  • cg=1
  • EP =1

Speculation! Success cg = 1 EP = 1

Decayed-history-based prediction

| : Profitable PPR | : Unprofitable PPR

slide-25
SLIDE 25

25

 | | |

| | | | | | | | | | | | | | |

  • THEP=0.6
  • β=0.2
  • γ = 0.5
  • SkippedPPRs=0
  • cg=1
  • EP =1

Speculation! Failed cg = 0.5 EP = 0.5

Decayed-history-based prediction

| : Profitable PPR | : Unprofitable PPR

slide-26
SLIDE 26

26

 | | |

| | | | | | | | | | | | | | |

  • THEP=0.6
  • β=0.2
  • γ = 0.5
  • SkippedPPRs=0
  • cg=0.5
  • EP =0.5

No Speculation! SkippedPPRs=1 EP = 0.7

Decayed-history-based prediction

| : Profitable PPR | : Unprofitable PPR

slide-27
SLIDE 27

27

 | | |

| | | | | | | | | | | | | | |

  • THEP=0.6
  • β=0.2
  • γ = 0.5
  • SkippedPPRs=1
  • cg=0.6
  • EP =0.7

Speculation! Success SkippedPPRs=0 cg = 0.8 EP = 0.8

Decayed-history-based prediction

| : Profitable PPR | : Unprofitable PPR

slide-28
SLIDE 28

28

 | | |

| | | | | | | | | | | | | | |

  • THEP=0.6
  • β=0.2
  • γ = 0.5
  • SkippedPPRs=0
  • cg=0.8
  • EP =0.8

Speculation! Failed cg = 0.4 EP = 0.4

Decayed-history-based prediction

| : Profitable PPR | : Unprofitable PPR

slide-29
SLIDE 29

29

 | | |

| | | | | | | | | | | | | | |

  • THEP=0.6
  • β=0.2
  • γ = 0.5
  • SkippedPPRs=0
  • cg=0.4
  • EP =0.4

No Speculation! SkippedPPRs = 1 EP = 0.6

Decayed-history-based prediction

| : Profitable PPR | : Unprofitable PPR

slide-30
SLIDE 30

30

 | | |

| | | | | | | | | | | | | | |

  • THEP=0.6
  • β=0.2
  • γ = 0.5
  • SkippedPPRs=1
  • cg=0.4
  • EP =0.6

Speculation! Success SkippedPPRs = 0 EP = 0.7

Decayed-history-based prediction

| : Profitable PPR | : Unprofitable PPR

...

slide-31
SLIDE 31

31

Outline

 Introduction  Adaptive-Algorithms  Experimental Result  Conclusion

slide-32
SLIDE 32

32

Experimental Result

 Prediction Accuracy

 Choose the best parameters for the algorithms  Evaluate two algorithms

 Computation Efficiency

 Finishing time  Time spent on all CPUs (Cost)

slide-33
SLIDE 33

33

Accuracy for Last-value-based

 Adjust non-speculate numbers

 Non-speculation

 PPRsToSkip -1

 Success Speculation

 NextPenalty= 1

 Failed Speculation

 PPRsToSkip = NextPenalty  NextPenalty *= α

slide-34
SLIDE 34

34

Accuracy for Last-value-based

α=1.4 Accuracy=81.6

slide-35
SLIDE 35

35

Accuracy for Decayed history based

 G_TH : gain threshold  Current state weight

 gain+ quota*β

 Non-speculative execution

 quota++

 Speculative execution

 gain = γ*g + (1-γ) * gain  quota  0

slide-36
SLIDE 36

36

Accuracy for Decayed history based

 Cumulative gain (CG)  cg = γ*g + (1-γ) * cg  Expected Profitability (EP)  EP = cg + SkippedPPRs *β  Speculate only if EP > THEP

g=

1 : success 0: failed

* THEP : threshold of speculation; * SkippedPPRs: reset to 0 on a success increase by 1 on a non-speculated PPR

slide-37
SLIDE 37

37

Accuracy for Decayed history based

 THEP = 0.25  β= 0.0075  γ = 0.4

Accuracy = 85.6%

slide-38
SLIDE 38

38

Computation Efficiency

 Machine: Intel Pentium-D dual-core processors  Compiler: gcc4.1  Benchmarks

 Gzip, Parser, Reduction

 Metrics

 Cost

 Total running time of all the processes

 Time

 Finishing time of a program

slide-39
SLIDE 39

39

Efficiency comparison on gzip

10 20 30 40

1.6MB 320KB 192KB

Buffer size Cost(s) seq

  • rg-bop

adapt-bop

slide-40
SLIDE 40

40

Efficiency comparison on gzip

5 10 15 20

1.6MB 320KB 192KB

Buffer size Time(s) seq

  • rg-bop

adapt-bop

slide-41
SLIDE 41

41

Efficiency comparison on parser

10 20 30 40

50 10 2

Num of Sentences Cost(s)

seq

  • rg-bop

adapt-bop

slide-42
SLIDE 42

42

Efficiency comparison on parser

5 10 15 20

50 10 2

Num of Sentences Time(S)

seq

  • rg-bop

adapt-bop

slide-43
SLIDE 43

43

Efficiency comparison on Reduction

10 20 30 40 50

10% 50% 90%

Denpendence Cost(s)

seq

  • rg-bop

adapt-bop

slide-44
SLIDE 44

44

Efficiency comparison on Reduction 10 20 30

10% 50% 90%

Denpendence Time(s)

seq

  • rg-bop

adapt-bop

slide-45
SLIDE 45

45

Outline

 Introduction to BOP  Adaptive Algorithms  Experimental Result  Conclusion

slide-46
SLIDE 46

46

Conclusions

 Failed Speculation is a problem  Two adaptive algorithms

 Last-value-based prediction  Decayed-history-based prediction

 Performance

 High accurate prediction  Keep fast running speed  Reduce cost

slide-47
SLIDE 47

47

Questions?

Thanks!