APPROXIMATE ORACLES AND SYNERGY IN SOFTWARE ENERGY SEARCH SPACES - - PowerPoint PPT Presentation

▶
approximate oracles and synergy in software energy search
SMART_READER_LITE
LIVE PREVIEW

APPROXIMATE ORACLES AND SYNERGY IN SOFTWARE ENERGY SEARCH SPACES - - PowerPoint PPT Presentation

Nov 07, 2023 131 likes •716 views

APPROXIMATE ORACLES AND SYNERGY IN SOFTWARE ENERGY SEARCH SPACES Bobby R. Bruce, Justyna Petke, Mark Harman, and Earl T. Barr IN THE BEGINNING Published at GECCO 2015 Used Genetic Improvement to reduce the energy consumption of MiniSAT

slide-1
SLIDE 1

APPROXIMATE ORACLES AND SYNERGY IN SOFTWARE ENERGY SEARCH SPACES

Bobby R. Bruce, Justyna Petke, Mark Harman, and Earl T. Barr

slide-2
SLIDE 2

IN THE BEGINNING

➤ Published at GECCO 2015 ➤ Used Genetic Improvement to reduce

the energy consumption of MiniSAT Some “Threats to Validity”

➤ Only for 1 (very small) piece of

software

➤ Limited number of runs ➤ Evidence our current approach to

Genetic Improvement was not ideal

➤ Indirect energy measurement (Intel

Power Gadget)

Bobby R. Bruce

slide-3
SLIDE 3

FOLLOWUP: LET’S DO THE SAME, BUT BIGGER!

Bobby R. Bruce

slide-4
SLIDE 4

FOLLOWUP: LET’S DO THE SAME, BUT BIGGER!

Real, direct energy measurements!

Bobby R. Bruce

slide-5
SLIDE 5

FOLLOWUP: LET’S DO THE SAME, BUT BIGGER!

Real, direct energy measurements! More applications!

Bobby R. Bruce

slide-6
SLIDE 6

FOLLOWUP: LET’S DO THE SAME, BUT BIGGER!

Real, direct energy measurements! More applications! Larger applications!

Bobby R. Bruce

slide-7
SLIDE 7

FOLLOWUP: LET’S DO THE SAME, BUT BIGGER!

Real, direct energy measurements! More applications! Larger applications! More evaluations!

Bobby R. Bruce

slide-8
SLIDE 8

THE FRAMEWORK WE DEVELOPED

Software + Input Output + Energy Master Node Node Measurement Board Controller Energy Measurement Board

Bobby R. Bruce

slide-9
SLIDE 9

THE COPY, DELETE, AND REPLACE OPERATORS

Bobby R. Bruce

slide-10
SLIDE 10

THE COPY, DELETE, AND REPLACE OPERATORS

Bobby R. Bruce

<IF_fileA_40>

slide-11
SLIDE 11

THE COPY, DELETE, AND REPLACE OPERATORS

Bobby R. Bruce

<IF_fileA_40> <for1_fileB_6><for1_fileB_78>

slide-12
SLIDE 12

THE COPY, DELETE, AND REPLACE OPERATORS

Bobby R. Bruce

<IF_fileA_40> <for1_fileB_6><for1_fileB_78> <fileA_101>+<fileA_15>

slide-13
SLIDE 13

THE COPY, DELETE, AND REPLACE OPERATORS

Bobby R. Bruce

<IF_fileA_40> <for1_fileB_6><for1_fileB_78> <fileA_101>+<fileA_15>

slide-14
SLIDE 14

THE COPY, DELETE, AND REPLACE OPERATORS

Bobby R. Bruce

<IF_fileA_40> <for1_fileB_6><for1_fileB_78> <fileA_101>+<fileA_15>

slide-15
SLIDE 15

THE GREAT GENETIC IMPROVEMENT CYCLE

Source Code Representation Population Altered Representation Selection Crossover Mutation Evaluation Better Source Code

Bobby R. Bruce

slide-16
SLIDE 16

APPLICATIONS STUDIED

Bobby R. Bruce

Bodytrack Ferret OMXplayer

slide-17
SLIDE 17

INITIAL INVESTIGATION

App Pop Size Num Gen Improvement? 7zip 100 20 Nope Bodytrack 100 20 Nope Bodytrack 100 50 Nope

Bobby R. Bruce

slide-18
SLIDE 18

Bobby R. Bruce

slide-19
SLIDE 19

Bobby R. Bruce

slide-20
SLIDE 20

Maybe 7zip and Bodytrack can’t be optimised very much?

Bobby R. Bruce

slide-21
SLIDE 21

Maybe 7zip and Bodytrack can’t be optimised very much? Maybe our search algorithm isn’t suitable?

Bobby R. Bruce

slide-22
SLIDE 22

Maybe 7zip and Bodytrack can’t be optimised very much? Maybe our search algorithm isn’t suitable? How effective can an individual modification be?

Bobby R. Bruce

slide-23
SLIDE 23

Maybe 7zip and Bodytrack can’t be optimised very much? Maybe our search algorithm isn’t suitable? How effective can an individual modification be? If an individual modification’s effectiveness is small, can it be detected?

Bobby R. Bruce

slide-24
SLIDE 24

Bobby R. Bruce

What does the search space look like?

slide-25
SLIDE 25

RESEARCH QUESTIONS

RQ1: Measurement: What variance occurs when measuring energy consumption? RQ1a: What is the variance when measuring using a single energy measurement device? RQ1b: What is the variance in direct energy measurements across multiple devices? RQ1c: What is the variance in proportional energy changes across multiple devices? RQ2: Improvement: What additional energy improvement can be achieved when using approximate oracles in place of exact oracles? RQ3: Synergy: How frequently do synergistic or antagonistic effects occur when combining known effective modifications?

Bobby R. Bruce

slide-26
SLIDE 26

THE SOFTWARE TARGETS

App Modifiable Lines of code What does “passed” mean? What can be approximated? 7zip 2,524

Creates a valid .7z file Compression Rate

Bodytrack 1,030

Creates a non- empty, readable

  • utput

The accuracy of the tracking

Ferret 5,032

Creates a non- null, readable list

  • f images

The ranking

OMXPlayer 5,184

Outputs data to the HDMI port The video quality

Bobby R. Bruce

slide-27
SLIDE 27

RQ1, MEASUREMENT WHAT VARIANCE OCCURS WHEN MEASURING ENERGY CONSUMPTION?

RQ1a: What is the variance when measuring using a single measurement device?

48.0 48.2 48.4 48.6 48.8 49.0 Energy Measurement (J) 361 362 363 364 365 366 Energy Measurement (J) 99 100 101 102 103 104 105 106 Energy Measurement (J) 57.0 57.2 57.4 57.6 57.8 58.0

Energy Measurement (J)

Bobby R. Bruce

7zip Ferret Bodytrack OMXPlayer

slide-28
SLIDE 28

RQ1, MEASUREMENT WHAT VARIANCE OCCURS WHEN MEASURING ENERGY CONSUMPTION?

RQ1b: What is the variance in direct energy measurements across multiple devices?

Bobby R. Bruce

slide-29
SLIDE 29

RQ1, MEASUREMENT WHAT VARIANCE OCCURS WHEN MEASURING ENERGY CONSUMPTION?

RQ1c: What is the variance in proportional energy changes across multiple devices?

Bobby R. Bruce

Small-Medium Medium-Large 10 20 30 40 50 60 Increase in energy consumption (%)

slide-30
SLIDE 30

RQ1 CONCLUSIONS

➤ The MAGEEC Energy Measurement boards have provided us

with a framework capable of acceptable precision but low accuracy.

➤ While the readings, when reported in Joules, vary wildly, the

proportional changes are relatively stable. Thus proportional change (i.e. a percentage increase or decrease) is recorded.

➤ The variation of energy readings within devices appear

acceptably small to detect meaningful changes when they

  • ccur

Bobby R. Bruce

slide-31
SLIDE 31

Bobby R. Bruce

Software Candidate Modification list Test Set

RQ2, IMPROVEMENT WHAT ADDITIONAL ENERGY IMPROVEMENT CAN BE ACHIEVED WHEN USING APPROXIMATE TEST ORACLES IN PLACE OF EXACT TEST ORACLES?

slide-32
SLIDE 32

Bobby R. Bruce

Software Candidate Modification list Test Set For each test Unmodified energy data Measure Energy x30

RQ2, IMPROVEMENT WHAT ADDITIONAL ENERGY IMPROVEMENT CAN BE ACHIEVED WHEN USING APPROXIMATE TEST ORACLES IN PLACE OF EXACT TEST ORACLES?

slide-33
SLIDE 33

Bobby R. Bruce

Software Candidate Modification list Test Set For each test Unmodified energy data Measure Energy x30

RQ2, IMPROVEMENT WHAT ADDITIONAL ENERGY IMPROVEMENT CAN BE ACHIEVED WHEN USING APPROXIMATE TEST ORACLES IN PLACE OF EXACT TEST ORACLES?

slide-34
SLIDE 34

Bobby R. Bruce

Software Candidate Modification list Per Modification Test Set For each test Mod, Energy, Approximation Unmodified energy data Measure Energy x30 Calculate Approx. Value

RQ2, IMPROVEMENT WHAT ADDITIONAL ENERGY IMPROVEMENT CAN BE ACHIEVED WHEN USING APPROXIMATE TEST ORACLES IN PLACE OF EXACT TEST ORACLES?

slide-35
SLIDE 35

RQ2B, IMPROVEMENT WHAT ADDITIONAL ENERGY IMPROVEMENT CAN BE ACHIEVED WHEN USING APPROXIMATE TEST ORACLES IN PLACE OF EXACT TEST ORACLES?

Original 33% Energy Reduction

Bobby R. Bruce

Bodytrack

slide-36
SLIDE 36

RQ2B, IMPROVEMENT WHAT ADDITIONAL ENERGY IMPROVEMENT CAN BE ACHIEVED WHEN USING APPROXIMATE TEST ORACLES IN PLACE OF EXACT TEST ORACLES?

Bobby R. Bruce

Bodytrack Energy reduction Approximation Value 2.69% 0.000 19.26% 0.131 27.97% 0.170 29.13% 0.192 33.69% 0.452

slide-37
SLIDE 37

RQ2B, IMPROVEMENT WHAT ADDITIONAL ENERGY IMPROVEMENT CAN BE ACHIEVED WHEN USING APPROXIMATE TEST ORACLES IN PLACE OF EXACT TEST ORACLES?

Bobby R. Bruce

7zip Energy reduction Approximation Value 5.08% 3.93x10-4 12.29% 0.072 13.17% 0.102 48.30% 0.741

slide-38
SLIDE 38

RQ2B, IMPROVEMENT WHAT ADDITIONAL ENERGY IMPROVEMENT CAN BE ACHIEVED WHEN USING APPROXIMATE TEST ORACLES IN PLACE OF EXACT TEST ORACLES?

Bobby R. Bruce

Ferret Energy reduction Approximation Value 43.19% 0.154 60.79% 39.873 75.53% 78.800 75.55% 1550.200 76.21% 2669.710 79.88% 6221.220

slide-39
SLIDE 39

RQ2B, IMPROVEMENT WHAT ADDITIONAL ENERGY IMPROVEMENT CAN BE ACHIEVED WHEN USING APPROXIMATE TEST ORACLES IN PLACE OF EXACT TEST ORACLES?

Bobby R. Bruce

Ferret Energy reduction Approximation Value 43.19% 0.154 60.79% 39.873 75.53% 78.800 75.55% 1550.200 76.21% 2669.710 79.88% 6221.220

slide-40
SLIDE 40

RQ2B, IMPROVEMENT WHAT ADDITIONAL ENERGY IMPROVEMENT CAN BE ACHIEVED WHEN USING APPROXIMATE TEST ORACLES IN PLACE OF EXACT TEST ORACLES?

Bobby R. Bruce

OMXPlayer Energy reduction Approximation Value 2.23% 0.000 78.45% 0.003 92.70% 0.637 95.53% 1.002 95.60% 1.043

slide-41
SLIDE 41

RQ2B, IMPROVEMENT WHAT ADDITIONAL ENERGY IMPROVEMENT CAN BE ACHIEVED WHEN USING APPROXIMATE TEST ORACLES IN PLACE OF EXACT TEST ORACLES?

Bobby R. Bruce

OMXPlayer Energy reduction Approximation Value 2.23% 0.000 78.45% 0.003 92.70% 0.637 95.53% 1.002 95.60% 1.043

slide-42
SLIDE 42

RQ2B, IMPROVEMENT WHAT ADDITIONAL ENERGY IMPROVEMENT CAN BE ACHIEVED WHEN USING APPROXIMATE TEST ORACLES IN PLACE OF EXACT TEST ORACLES?

Bobby R. Bruce

App Line deletion Line copy Line replace 7zip 5 3 Ferret 81 7 69 Bodytrack 44 1 27 OMXPlayer 8 3 13 Percentage 52.9% 4.2% 42.9% Distribution of operators

slide-43
SLIDE 43

RQ2B, IMPROVEMENT WHAT ADDITIONAL ENERGY IMPROVEMENT CAN BE ACHIEVED WHEN USING APPROXIMATE TEST ORACLES IN PLACE OF EXACT TEST ORACLES?

Bobby R. Bruce

App Line deletion Line copy Line replace 7zip 16.60% 0.00% 7.42% Ferret 56.45% 64.74% 43.50% Bodytrack 8.27% 0.16% 8.31% OMXPlayer 57.01% 71.86% 64.92% Percentage 34.58% 46.59% 31.04% Average impact of the operators

slide-44
SLIDE 44

RQ2 CONCLUSIONS

➤ When approximation is permitted the number of

modifications that reduce energy consumption increased

➤ The modifications are also capable of reducing energy

consumption by a greater extent

➤ Some applications produced better Pareto frontiers than

  • thers

➤ Copy operations are rarely effective though all have roughly

the same impact when they are.

Bobby R. Bruce

slide-45
SLIDE 45

RQ3, SYNERGY HOW FREQUENTLY DO SYNERGISTIC AND ANTAGONISTIC EFFECTS OCCUR WHEN COMBINING KNOWN EFFECTIVE MODIFICATIONS?

Bobby R. Bruce

slide-46
SLIDE 46

RQ3, SYNERGY HOW FREQUENTLY DO SYNERGISTIC AND ANTAGONISTIC EFFECTS OCCUR WHEN COMBINING KNOWN EFFECTIVE MODIFICATIONS?

Bobby R. Bruce

Effective Modification List

slide-47
SLIDE 47

RQ3, SYNERGY HOW FREQUENTLY DO SYNERGISTIC AND ANTAGONISTIC EFFECTS OCCUR WHEN COMBINING KNOWN EFFECTIVE MODIFICATIONS?

Bobby R. Bruce

Effective Modification List Create all possible pairs

slide-48
SLIDE 48

RQ3, SYNERGY HOW FREQUENTLY DO SYNERGISTIC AND ANTAGONISTIC EFFECTS OCCUR WHEN COMBINING KNOWN EFFECTIVE MODIFICATIONS?

Bobby R. Bruce

Effective Modification List Create all possible pairs Select Random 15%

slide-49
SLIDE 49

RQ3, SYNERGY HOW FREQUENTLY DO SYNERGISTIC AND ANTAGONISTIC EFFECTS OCCUR WHEN COMBINING KNOWN EFFECTIVE MODIFICATIONS?

Bobby R. Bruce

Effective Modification List Create all possible pairs Select Random 15% For each modification pair

slide-50
SLIDE 50

RQ3, SYNERGY HOW FREQUENTLY DO SYNERGISTIC AND ANTAGONISTIC EFFECTS OCCUR WHEN COMBINING KNOWN EFFECTIVE MODIFICATIONS?

Bobby R. Bruce

Effective Modification List Create all possible pairs Select Random 15% Measure in Framework For each modification pair

slide-51
SLIDE 51

RQ3, SYNERGY HOW FREQUENTLY DO SYNERGISTIC AND ANTAGONISTIC EFFECTS OCCUR WHEN COMBINING KNOWN EFFECTIVE MODIFICATIONS?

Bobby R. Bruce

Effective Modification List Aver(Mod1) Aver(Mod2) Aver(Mod1 + Mod2) Aver(Original) Create all possible pairs Select Random 15% Measure in Framework For each modification pair

slide-52
SLIDE 52

RQ3, SYNERGY HOW FREQUENTLY DO SYNERGISTIC AND ANTAGONISTIC EFFECTS OCCUR WHEN COMBINING KNOWN EFFECTIVE MODIFICATIONS?

Bobby R. Bruce

Effective Modification List Aver(Mod1) Aver(Mod2) Aver(Mod1 + Mod2) Aver(Original) Create all possible pairs Select Random 15% Measure in Framework For each modification pair

slide-53
SLIDE 53

RQ3, SYNERGY HOW FREQUENTLY DO SYNERGISTIC AND ANTAGONISTIC EFFECTS OCCUR WHEN COMBINING KNOWN EFFECTIVE MODIFICATIONS?

Bobby R. Bruce

Not worth combining Worth combining

Effective Modification List Aver(Mod1) Aver(Mod2) Aver(Mod1 + Mod2) Aver(Original) Create all possible pairs Select Random 15% Measure in Framework For each modification pair

slide-54
SLIDE 54

RQ3, SYNERGY HOW FREQUENTLY DO SYNERGISTIC AND ANTAGONISTIC EFFECTS OCCUR WHEN COMBINING KNOWN EFFECTIVE MODIFICATIONS?

Bobby R. Bruce

App Synergy Weak Antagonism Antagonism 7zip 0.9% 60.4% 38.7% Ferret 9.2% 48.8% 42.0% Bodytrak 35.3% 40.1% 24.6% OMXPlayer 2.6% 48.7% 48.7% Average 12.0% 49.5% 38.5%

slide-55
SLIDE 55

RQ3 CONCLUSIONS

➤ The majority of modifications are worth combining ➤ However, a significant minority exhibit antagonism, meaning

they shouldn’t be combined

➤ This means a greedy approach is unlikely to produce an

  • ptimal solution; more advanced search is required

Bobby R. Bruce

slide-56
SLIDE 56

ANY QUESTIONS?

Bobby R. Bruce

slide-57
SLIDE 57

ANY QUESTIONS?

Bobby R. Bruce