Conserving Fitness Evaluations by Marking Used Transitions Daniil - - PowerPoint PPT Presentation

▶
conserving fitness evaluations by
SMART_READER_LITE
LIVE PREVIEW

Conserving Fitness Evaluations by Marking Used Transitions Daniil - - PowerPoint PPT Presentation

Aug 29, 2023 164 likes •744 views

Learning Finite-State Machines: Conserving Fitness Evaluations by Marking Used Transitions Daniil Chivilikhin and Vladimir Ulyantsev National Research University of IT, Mechanics and Optics St. Petersburg, Russia Machine Learning Applications

slide-1
SLIDE 1

Learning Finite-State Machines: Conserving Fitness Evaluations by Marking Used Transitions

Daniil Chivilikhin and Vladimir Ulyantsev National Research University of IT, Mechanics and Optics

  • St. Petersburg, Russia

Machine Learning Applications in Software Engineering @ ICMLA’13 December 6, 2013

slide-2
SLIDE 2

Outline

  • Motivation and scope
  • Proposed idea
  • Experiments

– Artificial Ant problem – Test-based EFSM induction

  • Conclusion

Learning FSMs: Conserving Fitness Evaluations 2

slide-3
SLIDE 3

Motivation and scope

3

slide-4
SLIDE 4

Motivation: Reliable software

  • Systems with high cost of failure

– Energy industry – Aircraft industry – Space industry – …

  • We want to have reliable software

– Testing is not enough – Verification is needed

Learning FSMs: Conserving Fitness Evaluations 4

slide-5
SLIDE 5

Verification

  • Checking temporal rules (e.g. LTL)
  • Software verification can be harder than

software development

  • Need to make software that satisfies LTL-

specification by design

  • How?

Learning FSMs: Conserving Fitness Evaluations 5

slide-6
SLIDE 6

Model-driven development

  • Automated software engineering
  • Model-driven development

Software specification Model Code

Learning FSMs: Conserving Fitness Evaluations 6

slide-7
SLIDE 7

Model-driven development

Software specification Model Code

Finite-state machine

Learning FSMs: Conserving Fitness Evaluations 7

slide-8
SLIDE 8

Finite-State Machine

  • Σ – set of input events
  • Δ – set of output actions
  • δ: S×Σ→S – transition function
  • λ: S×Σ→Δ – actions function

2 A/z2 T/z1 1 A/z3 T/z3

Learning FSMs: Conserving Fitness Evaluations 8

slide-9
SLIDE 9

Automated-controlled object Finite-state machine Controlled

  • bject

Actions Events

Automata-based programming

Design programs with complex behavior as automated-controlled

  • bjects

e1 e2 z1 z3 Events Output actions z2 z4 z2

Learning FSMs: Conserving Fitness Evaluations 9

slide-10
SLIDE 10

Automata-based programming: advantages

  • Model before programming code, not vice

versa

  • Possibility of program verification using

Model Checking

  • You can check temporal properties (LTL)

Model Code

Finite-state machine

Learning FSMs: Conserving Fitness Evaluations 10

slide-11
SLIDE 11

Issues

  • Hard to build an FSM with desired

structure and behavior

  • One of approaches – mutation-based

metaheuristics

Learning FSMs: Conserving Fitness Evaluations 11

slide-12
SLIDE 12

Mutation-based FSM learning

  • Evolution Strategies (ES)
  • Genetic Algorithms (GA)
  • Mutation-based Ant Colony Optimization

(MuACO) All these algorithms use FSM mutations

Learning FSMs: Conserving Fitness Evaluations 12

slide-13
SLIDE 13

Mutation-Based Ant Colony Optimization

  • Proposed by the authors of this work in

2012

  • Based on Ant Colony Optimization
  • Can be thought of as an ES “with

memory”

  • No time to go into detail 

Learning FSMs: Conserving Fitness Evaluations 13

slide-14
SLIDE 14

Solution representation

Transition table Output table δ Event λ Event State A T State A T 1 1 2 1 z1 z2 2 2 1 2 z2 z3

Learning FSMs: Conserving Fitness Evaluations 14

slide-15
SLIDE 15

FSM Mutations

2

A/z2 T/z1

1

A/z3 T/z3

2

A/z2 T/z1

1

A/z1 T/z3

2

A/z2 T/z1

1

A/z3 T/z3 Change transition action Change transition end state

Learning FSMs: Conserving Fitness Evaluations 15

slide-16
SLIDE 16

16

Proposed idea

slide-17
SLIDE 17

Fitness evaluation: used transitions

Learning FSMs: Conserving Fitness Evaluations 17

Calculated fitness Used transitions

slide-18
SLIDE 18

Mutation

Learning FSMs: Conserving Fitness Evaluations 18

Mutate this transition

slide-19
SLIDE 19

Observation

Learning FSMs: Conserving Fitness Evaluations 19

Mutation Used transition Unused transition Fitness has not changed Compute fitness

slide-20
SLIDE 20

Challenge

OK, found a way not to calculate fitness of FSMs in some cases

1.Can we design an efficient implementation? 2.Will it make a difference in performance? 3.Limits of applicability?

Learning FSMs: Conserving Fitness Evaluations 20

slide-21
SLIDE 21

Implementation

  • Store an array of transition usage marks

for each FSM

  • Mark used transitions during fitness

evaluation

  • Copy marks when not calculating fitness

Learning FSMs: Conserving Fitness Evaluations 21

slide-22
SLIDE 22

Domain knowledge

  • Black box – no domain knowledge
  • General domain knowledge about FSMs
  • Problem-specific domain knowledge

Learning FSMs: Conserving Fitness Evaluations 22

slide-23
SLIDE 23

Experiments

Learning FSMs: Conserving Fitness Evaluations 23

slide-24
SLIDE 24

Experiments: Purpose

  • Does it make a difference?
  • How much resources does it require?

Learning FSMs: Conserving Fitness Evaluations 24

slide-25
SLIDE 25

Experiments: Algorithms

  • Evolutionary strategy (ES)
  • Genetic algorithm (GA)
  • Mutation-Based Ant Colony Optimization

for learning FSMs (MuACOsm)

Learning FSMs: Conserving Fitness Evaluations 25

slide-26
SLIDE 26

Experiments: Problems

  • 1. Artificial Ant Problem
  • 2. Learning EFSMs from tests

Learning FSMs: Conserving Fitness Evaluations 26

slide-27
SLIDE 27

General experimental setup

  • 1. Tune each algorithm for time ttune

– Using full factorial design of experiment

  • 2. Run each algorithm with tuned

parameters

Learning FSMs: Conserving Fitness Evaluations 27

slide-28
SLIDE 28

Artificial Ant Problem

  • Toroidal field N×N
  • M pieces of food
  • K time steps
  • Fixed position of food

and the ant

  • Goal – build an FSM,

such that the ant will eat all food in K steps

Field example: John Muir Trail

Learning FSMs: Conserving Fitness Evaluations 28

slide-29
SLIDE 29

Artificial Ant: Fitness function

K s K n f 1

last food

   

  • nfood – number of eaten food pieces
  • K – max number of allotted steps
  • slast – number of used steps

f

eaten food used time steps

Learning FSMs: Conserving Fitness Evaluations 29

slide-30
SLIDE 30

Success rate

  • Successful run: fitness ≥ 89
  • Success rate = Nsuccesful runs / Nruns

Learning FSMs: Conserving Fitness Evaluations 30

slide-31
SLIDE 31

Experiment design

  • Vary number of states
  • Limited number of fitness evaluations
  • Measure:

– Success rate – Time

Learning FSMs: Conserving Fitness Evaluations 31

slide-32
SLIDE 32

Success rate

32

slide-33
SLIDE 33

ES median time

33

slide-34
SLIDE 34

GA median time

34

slide-35
SLIDE 35

MuACO median time

35

slide-36
SLIDE 36

ES Fitness

36

slide-37
SLIDE 37

GA Fitness

37

slide-38
SLIDE 38

MuACO Fitness

38

slide-39
SLIDE 39

Ratio of lazy evaluations

39

slide-40
SLIDE 40

Fitness evaluation time: plain algorithms

40

slide-41
SLIDE 41

Fitness evaluation time: with marking

41

slide-42
SLIDE 42

Statistical Significance

  • ANOVA test
  • Fitness distributions significantly different

for ES and MuACOsm

  • Insignificant for GA

Learning FSMs: Conserving Fitness Evaluations 42

slide-43
SLIDE 43

Learning Extended Finite-State Machines from tests (1)

Learning FSMs: Conserving Fitness Evaluations 43

slide-44
SLIDE 44

Input data:

  • Number of states C and sets Σ and Δ
  • Set of test examples T
  • Ti =<input sequence Ij, output sequence Oj>

NP-hard problem: build an EFSM with C states compliant with tests T

Learning Extended Finite-State Machines from tests (2)

Learning FSMs: Conserving Fitness Evaluations 44

slide-45
SLIDE 45

Example of a test

A H T[x1] T[x1 & x2] → z1 z1 z2 z5 z7

Learning FSMs: Conserving Fitness Evaluations 45

slide-46
SLIDE 46

Learning EFSMs: Fitness function

  • Pass inputs to EFSM, record outputs
  • Compare generated outputs with references
  • Fitness = string similarity measure (edit

distance)

Learning FSMs: Conserving Fitness Evaluations 46

slide-47
SLIDE 47

Experimental setup

  • 1. Generate random EFSM with C states
  • 2. Generate set of tests of total length C×150
  • 3. Learn EFSM from tests
  • 4. Experiment for each C repeated 100 times
  • 5. Limited number of fitness evaluations

Learning FSMs: Conserving Fitness Evaluations 47

slide-48
SLIDE 48

Success rate

48

slide-49
SLIDE 49

Mean fitness

49

slide-50
SLIDE 50

Fitness evaluations

50

slide-51
SLIDE 51

Median time

51

slide-52
SLIDE 52

Time for fitness evaluation

52

slide-53
SLIDE 53

Conclusion

Developed approach

– Applicable to all FSM learning algorithms that use mutations – Allows to explore more FSMs with the same number of fitness evaluations – Effectively improves fitness and time

Learning FSMs: Conserving Fitness Evaluations 53

slide-54
SLIDE 54

Limitations

  • Makes sense to use if cost of fitness

computation is relatively high

Learning FSMs: Conserving Fitness Evaluations 54

slide-55
SLIDE 55

Future work

Explore more ways of using domain knowledge in FSM learning algorithms

Learning FSMs: Conserving Fitness Evaluations 55

slide-56
SLIDE 56

Acknowledgements

  • University ITMO research project 610455
  • Ministry of Education and Science of Russian

Federation in the framework of the federal program “Scientific and scientific-pedagogical personnel of innovative Russia in 2009-2013” (contract 16.740.11.0455, agreement 14.B37.21.0397)

Learning FSMs: Conserving Fitness Evaluations 56

slide-57
SLIDE 57

Thank you for your attention!

Daniil Chivilikhin Vladimir Ulyantsev This presentation online: http://rain.ifmo.ru/~chivdan/papers/2013/icmla-2013- presentation.pdf

Learning FSMs: Conserving Fitness Evaluations by Marking Used Transitions

{chivdan,ulyantsev}@rain.ifmo.ru

Learning FSMs: Conserving Fitness Evaluations 57