Hybrid Soft Computing: Soft Computing Overview Where Are We Going? - - PDF document

hybrid soft computing
SMART_READER_LITE
LIVE PREVIEW

Hybrid Soft Computing: Soft Computing Overview Where Are We Going? - - PDF document

4/21/2005 Hybrid SC and EA - Outline Hybrid Soft Computing: Soft Computing Overview Where Are We Going? -SC Components: PR, FL, NN, EA Modeling with FL and EA Hybrid SC Systems -FLC Parameter Tuning by EA Piero P. Bonissone -EA


slide-1
SLIDE 1

4/21/2005 1 Page 1

Hybrid Soft Computing: Where Are We Going?

Piero P. Bonissone GE

  • Soft Computing Overview
  • SC Components: PR, FL, NN, EA
  • Modeling with FL and EA
  • Hybrid SC Systems
  • FLC Parameter Tuning by EA
  • EA Parameter Setting
  • Conclusions

Hybrid SC and EA - Outline

  • Soft Computing Overview
  • SC Components: PR, FL, NN, EA
  • Modeling with FL and EA
  • Hybrid SC Systems
  • FLC Parameter Tuning by EA
  • EA Parameter Setting
  • Conclusions

Hybrid SC and EA - Outline Soft Computing

  • Soft Computing (SC): the symbiotic use of

many emerging problem-solving disciplines.

  • According to Prof. Zadeh:

"...in contrast to traditional hard computing, soft computing exploits the tolerance for imprecision, uncertainty, and partial truth to achieve tractability, robustness, low solution-cost, and better rapport with reality”

  • Soft Computing Main Components:
  • Approximate Reasoning:

» Probabilistic Reasoning, Fuzzy Logic

  • Search & Optimization:

» Neural Networks, Evolutionary Algorithms

Problem Solving Techniques

Symbolic Logic Reasoning Traditional Numerical Modeling and Search Approximate Reasoning Functional Approximation and Randomized Search

HARD COMPUTING SOFT COMPUTING Precise Models Approximate Models

Soft Computing: Hybrid Probabilistic Systems

Functional Approximation/ Randomized Search

Neural Networks Bayesian Belief Nets Evolutionary Algorithms Multivalued & Fuzzy Logics Dempster- Shafer Probabilistic Models

Approximate Reasoning

slide-2
SLIDE 2

4/21/2005 2 Page 2

Soft Computing: Hybrid Probabilistic Systems

Functional Approximation/ Randomized Search

Neural Networks Bayesian Belief Nets Fuzzy Influence Diagrams Belief of Fuzzy Events Evolutionary Algorithms Multivalued & Fuzzy Logics Probability of Fuzzy Events HYBRID PROBABILISTIC SYSTEMS Dempster- Shafer Probabilistic Models

Approximate Reasoning

Soft Computing: Hybrid Probabilistic Systems

Functional Approximation/ Randomized Search

Neural Networks Bayesian Belief Nets Fuzzy Influence Diagrams Belief of Fuzzy Events Evolutionary Algorithms Multivalued & Fuzzy Logics Probability of Fuzzy Events HYBRID PROBABILISTIC SYSTEMS Dempster- Shafer Probabilistic Models

Approximate Reasoning

Soft Computing: Hybrid Probabilistic Systems

Functional Approximation/ Randomized Search

Neural Networks Bayesian Belief Nets Fuzzy Influence Diagrams Belief of Fuzzy Events Evolutionary Algorithms Multivalued & Fuzzy Logics Probability of Fuzzy Events HYBRID PROBABILISTIC SYSTEMS Dempster- Shafer Probabilistic Models

Approximate Reasoning

Soft Computing: Hybrid FL Systems

Functional Approximation/ Randomized Search

Probabilistic Models Neural Networks Fuzzy Systems Evolutionary Algorithms Multivalued & Fuzzy Logics Multivalued Algebras Fuzzy Logic Controllers

Approximate Reasoning

Soft Computing: Hybrid FL Systems

Functional Approximation/ Randomized Search

Probabilistic Models Neural Networks Fuzzy Systems FLC Generated and Tuned by EA FLC Tuned by NN (Neural Fuzzy Systems) Evolutionary Algorithms Multivalued & Fuzzy Logics NN modified by FS (Fuzzy Neural Systems) Multivalued Algebras Fuzzy Logic Controllers HYBRID FL SYSTEMS

Approximate Reasoning

Soft Computing: Hybrid FL Systems

Functional Approximation/ Randomized Search

Probabilistic Models Neural Networks Fuzzy Systems FLC Generated and Tuned by EA FLC Tuned by NN (Neural Fuzzy Systems) Evolutionary Algorithms Multivalued & Fuzzy Logics NN modified by FS (Fuzzy Neural Systems) Multivalued Algebras Fuzzy Logic Controllers HYBRID FL SYSTEMS

Approximate Reasoning

slide-3
SLIDE 3

4/21/2005 3 Page 3 Soft Computing: Hybrid FL Systems

Functional Approximation/ Randomized Search

Probabilistic Models Neural Networks Fuzzy Systems FLC Generated and Tuned by EA FLC Tuned by NN (Neural Fuzzy Systems) Evolutionary Algorithms Multivalued & Fuzzy Logics NN modified by FS (Fuzzy Neural Systems) Multivalued Algebras Fuzzy Logic Controllers HYBRID FL SYSTEMS

Approximate Reasoning

Soft Computing: Hybrid NN Systems

Probabilistic Models Multivalued & Fuzzy Logics Feedforward NN RBF Recurrent NN Neural Networks Hopfield SOM ART

Functional Approximation/ Randomized Search Approximate Reasoning

Evolutionary Algorithms

Single/Multiple Layer Perceptron

Soft Computing: Hybrid NN Systems

Probabilistic Models Multivalued & Fuzzy Logics Feedforward NN RBF Recurrent NN HYBRID NN SYSTEMS NN topology &/or weights generated by EAs controlled by FLC NN parameters

(learning rate η momentum α )

Neural Networks Hopfield SOM ART

Functional Approximation/ Randomized Search Approximate Reasoning

Evolutionary Algorithms

Single/Multiple Layer Perceptron

Soft Computing: Hybrid NN Systems

Probabilistic Models Multivalued & Fuzzy Logics Feedforward NN

Single/Multiple Layer Perceptron

RBF Recurrent NN HYBRID NN SYSTEMS NN topology &/or weights generated by EAs controlled by FLC NN parameters

(learning rate η momentum α )

Neural Networks Hopfield SOM ART

Functional Approximation/ Randomized Search Approximate Reasoning

Evolutionary Algorithms

Soft Computing: Hybrid EA Systems

Probabilistic Models Multivalued & Fuzzy Logics Neural Networks Evolution Strategies Evolutionary Programs Genetic Progr. Genetic Algorithms Evolutionary Algorithms

Approximate Reasoning Functional Approximation/ Randomized Search

Soft Computing: Hybrid EA Systems

Probabilistic Models Multivalued & Fuzzy Logics Neural Networks Evolution Strategies Evolutionary Programs Genetic Progr. EA parameters (Pop size, select.) controlledby EA Genetic Algorithms EA parameters (N, P

cr, P mu)

controlled by FLC EA-based search inter-twined with hill-climbing HYBRID EA SYSTEMS Evolutionary Algorithms

Approximate Reasoning Functional Approximation/ Randomized Search

slide-4
SLIDE 4

4/21/2005 4 Page 4 Soft Computing: Hybrid EA Systems

Probabilistic Models Multivalued & Fuzzy Logics Neural Networks Evolution Strategies Evolutionary Programs Genetic Progr. EA parameters (Pop size, select.) controlledby EA Genetic Algorithms EA parameters (N, P

cr, P mu)

controlled by FLC EA-based search inter-twined with hill-climbing HYBRID EA SYSTEMS Evolutionary Algorithms

Approximate Reasoning Functional Approximation/ Randomized Search

Soft Computing: Hybrid EA Systems

Probabilistic Models Multivalued & Fuzzy Logics Neural Networks Evolution Strategies Evolutionary Programs Genetic Progr. EA parameters (Pop size, select.) controlledby EA Genetic Algorithms EA parameters (N, P

cr, P mu)

controlled by FLC EA-based search inter-twined with hill-climbing HYBRID EA SYSTEMS Evolutionary Algorithms

Approximate Reasoning Functional Approximation/ Randomized Search

  • Soft Computing Overview
  • SC Components: PR, FL, NN, EA
  • Modeling with FL and EA
  • Hybrid SC Systems
  • FLC Parameter Tuning by EA
  • EA Parameter Setting
  • Conclusions

Hybrid SC and EA – Outline (2)

Fuzzy Logic Genealogy

  • Origins: MVL for treatment of imprecision

and vagueness

  • 1930s: Post, Kleene, and Lukasiewicz

attempted to represent undetermined, unknown, and other possible intermediate truth-values.

  • 1937: Max Black suggested the use of a

consistency profile to represent vague (ambiguous) concepts

  • 1965: Zadeh proposed a complete theory of

fuzzy sets (and its isomorphic fuzzy logic), to represent and manipulate ill-defined concepts

Fuzzy Logic : Linguistic Variables

  • Fuzzy logic give us a language (with syntax and

local semantics), in which we can translate our qualitative domain knowledge.

  • Linguistic variables to model dynamic systems
  • These variables take linguistic values that are

characterized by:

  • a label - a sentence generated from the syntax
  • a meaning - a membership function

determined by a local semantic procedure

Fuzzy Logic : Reasoning Methods

  • The meaning of a linguistic variable may be

interpreted as a elastic constraint on its value.

  • These constraints are propagated by fuzzy

inference operations, based on the generalized modus-ponens.

  • A FL Controller (FLC) applies this reasoning

system to a Knowledge Base (KB) containing the problem domain heuristics.

  • The inference is the result of interpolating

among the outputs of all relevant rules.

  • The outcome is a membership distribution on

the output space, which is defuzzified to produce a crisp output.

slide-5
SLIDE 5

4/21/2005 5 Page 5

S S L L S L S L LN SN SP LP Inputs

Fuzzy Logic Control : Inference Method

State Variables Output Variable Rules

Defuzzification Inter- polation

FLC Inference Method (cont.)

  • A FLC (KB + Reasoning Mechanism)

defines a deterministic response surface in the cross product of state and output spaces, which approximates the original relationship.

  • The FLC leverages the interpolation

properties of this reasoning mechanism, to exhibit robustness with respect to parameter variations, disturbances, etc.

Example (MISO): Max-min Composition with Centroid Defuzzification

  • If X is SMALL and Y is SMALL then Z is NEG. LARGE
  • If X is SMALL and Y is LARGE the Z is NEG. SMALL
  • If X is LARGE and Y is SMALL the Z is POS. SMALL
  • If X is LARGE and Y is LARGE then Z is POS. LARGE

Response Surface

Term set Rules set

Evolutionary Algorithms (EA)

EA are part of the Derivative-Free Optimization and Search Methods:

  • Evolutionary Algorithms
  • Simulated annealing (SA)
  • Random search
  • Downhill simplex search
  • Tabu search

EA consists of:

  • Evolution Strategies (ES)
  • Evolutionary Programming (EP)
  • Genetic Algorithms (GA)
  • Genetic Programming (GP)

Evolutionary Algorithms Characteristics

  • Most Evolutionary Algorithms can be

described by

  • x[t] : the population at time t under

representation x

  • v : is the variation operator(s)
  • s : is the selection operator

x[t + 1] = s(v(x[t]))

Evolutionary Algorithms Characteristics

  • EA exhibit an adaptive behavior that allows

them to handle non-linear, high dimensional problems without requiring differentiability or explicit knowledge of the problem structure.

  • EA are very robust to time-varying behavior,

even though they may exhibit low speed of convergence.

slide-6
SLIDE 6

4/21/2005 6 Page 6

Modeling

  • Model =

Structure + Parameters + Search Method

  • Classical control theory:
  • Structure: order of the differential equations
  • Parameters: coefficients of differential

equation.

  • Search method: LMSE, Pole-placement, etc.

Modeling Using FLC (Mamdani type)

  • A Mamdani- type FLC approximates a

relationship between a state X and an

  • utput Y by using a KB and a reasoning

mechanism (generalized modus-ponens).

  • The Knowledge Base (KB) is defined by:
  • Scaling factors (SF): ranges of values of state

and output variables

  • Termset (TS): membership functions of values
  • Ruleset (RS): a syntactic mapping of symbols

from X to Y

Modeling Using FLC (Mamdani type)

  • The structure of the model is the ruleset.
  • The parameters of the model are the

scaling factors and termsets.

  • The search method is initialized by

knowledge engineering and refined with some other external methods (SOFC, error minimization, etc.)

Modeling Using EA

  • Similarly, for EA:
  • The structure of the model is the

representation of an individual in the population (e.g., binary string, vector, parse tree, Finite State Machine).

  • The parameters of the model are the

Population Size, Probability of Mutation, Prob.

  • f Recombination, Generation Gap, etc.
  • The search method is a global search based
  • n maximization of population fitness function
  • Soft Computing Overview
  • SC Components: PR, FL, NN, EA
  • Modeling with FL and EA
  • Hybrid SC Systems
  • FLC Parameter Tuning by EA
  • EA Parameter Setting
  • Conclusions

Hybrid SC and EA – Outline (3a)

Hybrid Soft Computing: FLC Tuned by EAs

Probabilistic Models Neural Networks FLC Generated and Tuned by EA Multivalued Algebras HYBRID FLC/EA SYSTEMS Evolution Strategies Evolutionary Programs Genetic Progr. Fuzzy Logic Controllers Fuzzy Systems

Approximate Reasoning

Multivalued & Fuzzy Logics Genetic Algorithms Evolutionary Algorithms

Functional Approximation/ Randomized Search

slide-7
SLIDE 7

4/21/2005 7 Page 7 FLC Tuned by EA - Outline

  • Components & Historical Approaches
  • Application to Automatic Train

Handling (ATH)

  • Solution Architecture
  • Analysis of Results
  • Remarks

FL Controllers Tuned by EAs

  • FLC
  • FLC = KB + Inference Engine (with Defuzz.)
  • KB parameters:

»Scaling factors (SF) »Membership Functions (MF) »Rule set (RS)

  • EA
  • Encoding: binary or real-valued
  • Chromosome: string or table
  • Fitness function: Sum quadratic errors, entropy
  • Operators: one-point crossover, max-min

arithmetical crossover, point-radius crossover.

FL Controllers tuned by EAs (cont.)

  • Historical Approaches:
  • Karr 91-93:

» Chromosome = concatenation of all termsets. » Each value in a termset was represented by 3 binary- encoded parameters.

  • Lee & Takagi 93:

» Chromosome = 1 TSK rule (LHS: memb. fnct. RHS pol.) » Binary encoding of 3-parameter repr. of each term

  • Surman et al: 93:

» Fitness function with added entropy term describing number of activated rules

SC in Train Handling: An Example

  • Problem Description
  • Develop an automated train handler to control a massive,

distributed system with little sensor information

  • Freight trains consist of several hundred heavy railcars

connected by couplers (train length up to two miles)

  • Each coupler typically has a dead zone and a

hydraulically damped spring

  • Railcars can move relative to each other while in motion,

leading to a train that can change its length by 50 – 100 ft.

  • The position of the cars and couplers cannot be

electronically sensed

SC in Train Handling: An Example

  • Solution Requirements
  • An automated system has to satisfy multiple goals:
  • Tracking a velocity reference (defined over distance)

to enforce speed limits and respect the train schedule

Multi-body regulation problem, subject to proper slack management, without sensors for most of the state

  • Providing a degree of train-handling uniformity across

all crews

  • Operating the train in fuel-efficient regimes
  • Maintaining a smooth ride by avoiding sudden

accelerations or brake applications (slack control)

SC in Train Handling: An Example

  • Description of Our Approach
  • Use a Velocity Profile externally generated

(using classical optimization or Evolutionary Algorithms)

  • Use a Fuzzy Logic Control (FLC) to track the

velocity reference (Fuzzy PI Control)

  • Use an Evolutionary Algorithms to tune the

FLC parameters to minimize velocity tracking error and number of throttle changes

  • Implement control actions with fuzzy rule set

to maintain slack control

slide-8
SLIDE 8

4/21/2005 8 Page 8

FLC tuned by EAs: Our Approach

  • Chromosome (real-valued encoding)
  • Chr. 1 = Scaling factors;
  • Chr. 2 = Termsets;
  • Chr. 3 = Rules (not used)
  • Order of tuning (as in Zheng '92):
  • Initialize rulebase with standard PI structure and

termsets with uniformly distributed terms

  • Apply EAs to find best scaling factors
  • Apply EAs to find best termsets
  • Apply EAs to find best rule set (not used)
  • Transition from large to small granularity

FLC Sensitivity to Parameter Changes

X1 X2 Very Low Low Medium High Very High Very Low PH PH PM PL ZE Low PH PM PL ZE NL Medium PM PL ZE NL NM High PL ZE NL NM NH Very High ZE NL NM NH NH

Changing a Scaling Factor

X1 X2 Very Low Low Medium High Very High Very Low PH PH PM PL ZE Low PH PM PL ZE NL Medium PM PL ZE NL NM High PL ZE NL NM NH Very High ZE NL NM NH NH

Changing a Term in X1

X1 X2 Very Low Low Medium High Very High Very Low PH PH PM PL ZE Low PH PM PL ZE NL Medium PM PL ZE NL NM High PL ZE NL NM NH Very High ZE NL NM NH NH

Changing a Rule

Architecture: Modules, Fitness Funct.

  • Architecture
  • EA: pop.size=50; P(cross)=.6; P(mut)=.001
  • Three Types of fitness functions
  • Train Simulator: NSTD (STD+TEM)
  • Fuzzy PI (Ke, Kedot, K∆u)
  • Fitness functions (f1, f2, f3)

f1 = min( |notch

i − i

notch(i−1)|+|dynbrakei − dynbrake(i−1)|) f2 = min( |vi

i

− vi

d|)

f3 = min(w1 |notchi −

i

notch

(i−1)|

K1 + w2 ( |vi

i

− vi

d|)

K2

FLC tuned by GAs

Train Simulator FLC (PI) Fitness Function GA (GENESIS) SF or MF

Experiment Design

  • 12 test (4 for each fitness function)
  • Initial SF with initial MF;
  • EA tuned SF with Initial MF
  • Initial SF with EA tuned MF;
  • EA tuned SF with EA tuned MF
  • Train Simulation:
  • 14 miles long flat track
  • 1 uniformly heavy train with 100 cars and 4

locomotives

  • Analytically computed velocity profile

Experiment Design

  • Representation:
  • SF: 3 floating point values for Ke, Kedot, K∆u
  • MF (21-9) = 12 values

» 21 parameters: [(Lefti ,Centeri , Righti ) for i=1, ..., 7] » 9 dependent values: [(Lefti = Right(i+1)) for i=1, ..., 6] + [Center1= Center7]+[Right1 = Left7 = 0]

  • Constraints to maintain 0.5 terms overlap, for

best interpolation

slide-9
SLIDE 9

4/21/2005 9 Page 9

Experiments Results

  • Experiment Results with f1

Description Time Journey Fuel Fitness Gen.

Initial SF; Initial MF 26.5 14.26 878 73.2 EA tuned SF; Initial MF 27.8 14.21 857 15.15 4 Initial SF; EA tuned MF 26.00 14.18 879 70.93 20 EA tuned SF; EA tuned MF 28.3 14.12 829 14.64 10

  • Experiment Results with f3

Description Time Journey Fuel Fitness Gen.

Initial SF; Initial MF 26.5 14.26 878 1 EA tuned SF; Initial MF 27.2 14.35 871 0.817 4 Initial SF; EA tuned MF 26.26 14.18 871 0.942 20 EA tuned SF; EA tuned MF 27.3 14.35 872 0.817 10

Tuning of FLC with EA: Remarks

  • Verified tuning order proposed by Zheng (92)

»SF tuning: major impact »MF tuning: minor impact »RS tuning: almost no impact

  • For both f1 and f3, fuel minimization is implicitly

derived from throttle jockeying minimization

  • Complex fitness function (requiring simulation

run - 23 sec for each chromosome evaluation) limited trials number - with no apparent impact

  • Successfully tested on simulated 43 mile long

track with altitude excursions

»(Selkirk, NY->Framingham, MA)

MPH 60.00 50.00 40.00 30.00 20.00 10.00 0.00

NOTCH POSITION

8 4

NOTCH POSITION mile mile

Results of EA Tuned PI on 43 mile Track

MPH 60.00 50.00 40.00 30.00 20.00 10.00 0.00

NOTCH POSITION

8 4

NOTCH POSITION mile mile

Results of EA Tuned PI on 43 mile Track

MPH 60.00 50.00 40.00 30.00 20.00 10.00 0.00

NOTCH POSITION

8 4

NOTCH POSITION mile mile

Results of EA Tuned PI on 43 mile Track

MPH 60.00 50.00 40.00 30.00 20.00 10.00 0.00

NOTCH POSITION

8 4

NOTCH POSITION mile mile

Results of EA Tuned PI on 43 mile Track

slide-10
SLIDE 10

4/21/2005 10 Page 10

  • Soft Computing Overview
  • SC Components: PR, FL, NN, EA
  • Modeling with FL and EA
  • Hybrid SC Systems
  • FLC Parameter Tuning by EA
  • EA Parameter Setting
  • Conclusions

Hybrid SC and EA – Outline (3b)

  • EA Model:
  • Structure, Parameters
  • EA Parameter Setting
  • EA Parameter Tuning
  • EA Parameter Control
  • An Application to Agile Manufacturing
  • Object-level Representation and Complexity
  • Solution
  • FLC KB
  • Statistical Experiments
  • Analysis and Summary of 1200 Experiments
  • Remarks

EA Parameter Setting

Object-level Problem Object-level GA Structure & Parameters

EA Model

  • GA Structural Design Selections:
  • GA Type:

»{Simple, Steady-State, Niche,…}

  • Chromosome Encoding:

»{Binary, Integer, Real,...}

  • Constraints Representation:

»{Penalty function, data structure, filters, …}

  • Fitness Function:

»{Scalar function, Weighted aggregation of multiple functions, Vector-valued function, …}

EA Structure

  • Adjustable parameters for a GA
  • N

= Population size Large pop. prevent premature convergence

  • Pc = Crossover rate:

Pcr * N = # crossovers per generation

  • Pm

= Mutation rate: Pm * N * L = # mutations per generation

  • G

= Generation Gap Percentage of population to be replaced

  • W

= Scaling Window Size =[1, 7]

  • S

= Selection Strategy = {Elitist, Non-Elitist}

  • Other possible parameters that could be adjusted:

EA Parameters

  • EA Model:
  • Structure, Parameters
  • EA Parameter Setting
  • EA Parameter Tuning
  • EA Parameter Control
  • An Application to Agile Manufacturing
  • Object-level Representation and Complexity
  • Solution
  • FLC KB
  • Statistical Experiments
  • Analysis and Summary of 1200 Experiments
  • Remarks

EA Parameter Setting - Outline

slide-11
SLIDE 11

4/21/2005 11 Page 11

EAs Parameter Setting

Parameter Tuning Before the run Parameter Control Parameter Setting During the run

EAs Parameter Setting

Parameter Tuning Adaptive Deterministic Before the run Parameter Control Self-Adaptive Parameter Setting During the run

Parameter Tuning

EAs Parameter Setting: Parameter Tuning

Adaptive Deterministic

Parameter Control

Self-Adaptive

Parameter Setting

During the run

  • Off-line Tuning
  • Determined before

running the GAs on the object-level problem by »Studying a subset

  • f five diverse

problems (DeJong, 1975) »Running a Meta- Genetic Algorithm (Grefenstette, 1986)

Before the run

Object-level Problem Object-level GA

Population Size: 50 Crossover Rate: 0.6 Mutation Rate: 0.001 Replacement 100% Scaling Window n=inf Selection Strategy Elitist Suite of 5 problems:

  • Parabola
  • Rosenbrock’s

saddle

  • Step function
  • Quartic Noise
  • Shekel’s foxholes

Object-level GA

Off Line Tuning of GA Parameters (DeJong, 1975)

SC Hybrid Systems: EA Tuning EA

Probabilistic Models Approximate Reasoning Approaches Multivalued & Fuzzy Logics Neural Networks Evolution Strategies Evolutionary Programs Genetic Progr. EA parameters (Pop size, select.) controlledby EA HYBRID EA SYSTEMS

Genetic Algorithms Evolutionary Algorithms

Search/Optimization Approaches Off-Line Performance

Population Size: 80 Crossover Rate: 0.45 Mutation Rate: 0.01 Replacement 90% Scaling Window n = 1 Selection Strategy

NonElitist

Object-level Problem

Object-level GA On-Line Performance

Population Size: 30 Crossover Rate: 0.96 Mutation Rate: 0.01 Replacement 100% Scaling Window n = inf Selection Strategy Elitist

Object-level GA Meta- GA

Off Line Tuning of GA Parameters (Grefenstette, 1986)

Suite of 5 problems:

  • Parabola
  • Rosenbrock’s

saddle

  • Step function
  • Quartic Noise
  • Shekel’s foxholes

Object GA Parameter Set Object GA Performance

slide-12
SLIDE 12

4/21/2005 12 Page 12

GAs Parameter Setting: Deterministic Control

Adaptive

Deterministic

Parameter Tuning

Parameter Control

Self-Adaptive

During the run

  • No feedback

information is used.

  • A time-varying

schedule is used to modify a GA parameter p

  • p is replaced by p(t)
  • Correct design of p(t)

is very difficult

Before the run

Parameter Setting

Control of Population size

By decreasing Population Size toward the last part of the Evolution we are trying to improve the solution refinement (e.g., more generations with same number of trials)

EAs Parameter Setting: Deterministic Control - Example

300 500 30%45% 80% CurrGen/Ma 150 PopSize 338 500 30% 45% CurrGen/Max 150 PopSize 80%

  • Constant Population size: N = 338
  • Number of trials = 338 * MaxGen
  • Variable Population size: N(t)
  • Number of trials = 338 * MaxGen
  • Incorporate parameters

into chromosome making them subject to evolution

  • Typically used to

determine Mutation Step S: [g1 g2 ... gn S]

  • r

[g1 g2 ... gn S1 S2 ... Sn ]

EAs Parameter Setting: Self-Adaptive Control

Adaptive Deterministic

Before the run Parameter Tuning

Parameter Control Self- Adaptive Parameter Setting

Mutation Step for Entire Genome

Mutation Steps for Each Genome Value During the run

GAs Parameter Setting: Adaptive Control

Adaptive

Deterministic

Before the run Parameter Tuning

Parameter Control

Self-Adaptive

  • Feedback from the

search is used to determine the direction and/or magnitude of the change in the parameter value.

  • A Fuzzy Logic Controller

is used to obtain parameter changes in

» Population Size » Mutation Rate

as a function of

» Genotypic Diversity » Percentage Completed Trials

During the run

Parameter Setting

SC Hybrid Systems: FLC Tuning EA

Probabilistic Models Neural Networks Evolution Strategies Evolutionary Programs Genetic Progr. EA parameters controlled by FLC HYBRID SYSTEMS MV-Algebras Fuzzy Controller Fuzzy Logic Multivalued & Fuzzy Logics Approximate Reasoning Approaches

Genetic Algorithms Evolutionary Algorithms Search/Optimization Approaches

Object-level Problem

Fuzzy Logic Controlled GA (FLC-GA)

Object-level GA Fuzzy Logic Controller

  • Genotypic Diversity
  • Percentage

Completed Trials

KB

∆ Population Size ∆ Mutation Rate

State Variables describing the evolution stage Controlled GA parameters

slide-13
SLIDE 13

4/21/2005 13 Page 13

  • EA Model:
  • Structure, Parameters
  • EA Parameter Setting
  • EA Parameter Tuning
  • EA Parameter Control
  • An Application to Agile Manufacturing
  • Object-level Representation and Complexity
  • Solution
  • FLC KB
  • Statistical Experiments
  • Analysis and Summary of 1200 Experiments
  • Remarks

EA Parameter Setting

Global optimization of design, manufacturing, supplier planning decisions in a distributed manufacturing environment

Customer

Supplier

Manufacturing Design

Tools Tools Data

Marketing

Tools Data

Virtual Design Environment

EA Parameter Control: An Application

Data Tools Data Data Tools

Design Part P1 Acceptable Alternates Suppliers

P1 P2 Pk M 1 2 3 8 Genome Gene Allele Sets Crossover Operation Mutation

Parents Offspring

1 2 3 6

Part P2 Part Pk

Manufacturing Facilities Parts, suppliers, and design DB

  • Mfg. DB

3 ) 10 ( −

=

T ij

T T ij ij

e C J

min i,j

Object-level Optimization Problem

Object-level Problem Representation Search Space Size

  • For EA Statistical Analysis:

O(107)

  • For EA Performance Validation:

O(1018) and O(1021)

Object-level Problem Complexity

  • EA Model:
  • Structure, Parameters
  • EA Parameter Setting
  • EA Parameter Tuning
  • EA Parameter Control
  • An Application to Agile Manufacturing
  • Object-level Representation and Complexity
  • Solution
  • FLC KB
  • Statistical Experiments
  • Analysis and Summary of 1200 Experiments
  • Remarks

EA Parameter Setting - Outline

Fuzzy Logic Controller

Solution Architecture

Object-level GA Manufacturing Planning Module Untuned GA Fuzzy Logic Controlled GA (Online Control) Object-level GA

slide-14
SLIDE 14

4/21/2005 14 Page 14

Object-level GA Manufacturing Planning Module

Population Size: 50 Generations: 250 Crossover Rate: 0.6 Mutation Rate: 0.001

Untuned GA (U-TGA)

Guidance for Experiments

  • Minimize high-level search space size for FLC-EA by
  • Identify primary drivers (influences) of EA search

DOE determined that the two main drivers were:

Population Size (N) and Mutation Rate (Pm )

  • Control primary drivers by few simple heuristic rules

Built two FLC controllers with heuristic rule sets and SF Changed on input (state variable) to capture evolution stage

  • Determining FLC firing rate
  • Take a control action every 10 generation
  • Extensive & statistically significant empirical evidence
  • Use t-test and F-tests to analyze µ and σ

improvements

Object-level Problem

Object-level EA Fuzzy Logic Controller

  • Genotypic Diversity
  • Percentage

Completed Trials

KB

∆ Population Size ∆ Mutation Rate

I/O Scaling Factors I/O Termsets Rule Sets

Fuzzy Logic Controller for EAs: Knowledge Base

  • Inputs

GD = Genotypic Diversity: Normalized Average Hamming Distance PFE = Percentage Fitness Evaluations:

(Completed # Trials) / (Max Allocated # Trials)

∑ + = = − = n i j i Length Genome ij d n n GD 1 , 1 ) 1 ( 2

where dij is the Hamming Distance GD range is [0, 1] == [Low, High] PFE range is [0, 1] = [Low, High]

Fuzzy Controller for ∆N and ∆Pm: Inputs

Object-level Problem

Object-level EA Fuzzy Logic Controller

  • Genotypic Diversity
  • Percentage

Completed Trials

KB

∆ Population Size ∆ Mutation Rate

I/O Scaling Factors I/O Termsets Rule Sets

Fuzzy Logic Controller for EAs: Knowledge Base

  • Inputs:
  • GD:

A(Very Low), B(Low), C(Medium), D(High), E(Very High)

  • PFE: A(Very Low), B(Low), C(Medium), D(High), E(Very High)
  • Outputs (for both ∆ N and ∆ Pm):
  • A(Neg. High), B(Neg. Medium), C(No Change), D(Pos. Medium),

E(Pos. High)

0.8 1 A B C D E 0.8 1 A B C D E

Fuzzy Controller for ∆N and ∆Pm: Termsets

slide-15
SLIDE 15

4/21/2005 15 Page 15

Object-level Problem

Object-level EA Fuzzy Logic Controller

  • Genotypic Diversity
  • Percentage

Completed Trials

KB

∆ Population Size ∆ Mutation Rate

I/O Scaling Factors I/O Termsets

Rule Sets

Fuzzy Logic Controller for EAs: Knowledge Base

GD = Genotypic Diversity: Normalized Average Hamming Distance PFE = Percentage Fitness Evaluations: (Completed # Trials) / (Max Allocated # Trials) ∆N = Change in Population Size

GD, PFE−>∆N

Genotypic Percentage Fitness Evaluation (PFE) Diversity (GD) Very Low Low Medium High Very High Very Low

Pos High Pos High Pos High Pos Medium No Change

Low

Pos High Pos High Pos Medium No Change Neg Medium

Medium

Pos High Pos Medium No Change Neg Medium Neg High

High

Pos Medium No Change Neg Medium Neg High Neg High

Very High

No Change Neg Medium Neg High Neg High Neg High

Exploitation Stage

Reduce population/ Refine best

Exploration Stage

Increase population/ broaden search

Fuzzy Controller for Population Size: Rule Set

  • Data Set for Experiments
  • Seven part classes corresponding to a

complexity of O(107)

  • EA Structure:
  • Type:

{Simple, Steady- State}

  • Chromosome Encoding:

Integer

  • Fitness Function:

Three type of cost functions

  • Selection Method:

Proportional Roulette

  • Crossover Operator:

Uniform

  • Mutation Operator:

Exponentially

Statistical Experiments: EA Structure

  • Set-Up for 1200 experiments:
  • We defined 4 EA configurations

(a) Untuned Simple EA (U-SEA) (b) FL Controlled Simple EA (FLC-SEA) (c) Untuned Steady State EA (U-SSEA) (d) FL Controlled Steady State EA (FLC- SSEA)

Statistical Experiments: Set-Up

  • For each configuration we performed

300 experiments:

  • 20 runs for each pair of (Cost function, Max

number of Trials)

  • 15 different pairs of (Cost function, Max number
  • f Trials)
  • Three types of cost functions:

(1) J = C*T; (2) J = C*T2; (3) J = C*e(T-10)/3

  • Five values of maximum number of Trials (to

evaluate effect of different evolution lengths):

(i) 3,000; (ii) 5,000; (iii) 7,000; (iv) 9,000; (v) 11,000

Statistical Experiments: Set-Up (cont.)

  • For each of the four configurations (a-d) we

ran 20 experiments with the same parameters

  • Then we considered the following

measures: B = sample average over 20 experiments of Best score frequency (number of time cost function J reached its minimal value - known a priori for small size experiment) µ = average of population best σ = standard deviation of population best

B ˆ

σ ˆ

Statistical Experiments: Measures

µ ˆ

slide-16
SLIDE 16

4/21/2005 16 Page 16

  • We performed an ANOVA test (both

t and F test - with p < 0.05 ) to see if:

Cost (U-SEA) >> cost ( FLCSEA) Cost (U-SEA) >> cost ( U-SSEA) Cost (U-SSEA) >> cost ( FLC-SSEA)

  • We verified if the FLC caused the

controlled EA to perform worse than its corresponding untuned EA, i.e.:

Cost (U-SEA) << cost ( FLC-SEA) Cost (U-SSEA) << cost ( FLC-SSEA)

Statistical Experiments: Analysis Statistical Experiments: Results

  • For each cost function we ran 400 experiments (100 x EA type)
  • For each EA type we ran 20 experiments for 5 different pop. sizes
  • The entry in each cell is the number of significant changes found in

the statistics of each of these five groups of experiments

C*T C*T2 C*e(T-10)/3

J = Cost U-SEA FLC-SEA U-SSEA FLC-SSEA Function B µ σ B µ σ B µ σ B µ σ

  • 4

3

  • 1

2

  • 2

3

  • 1

3

  • 1
  • 3
  • 1
  • 1

2

Significant changes in σ

Total

7% 47% 60% 7% 20% 47%

Significant changes in µ and in σ

  • EA Model:
  • Structure, Parameters
  • EA Parameter Setting
  • EA Parameter Tuning
  • EA Parameter Control
  • An Application to Agile Manufacturing
  • Object-level Representation and Complexity
  • Solution
  • FLC KB
  • Statistical Experiments
  • Analysis and Summary of 1200 Experiments
  • Remarks

EA Parameter Setting Remarks

  • FLC State Representation: [Evolution

Stage]

  • Evolution time needs to be an explicit state

variable since we have different control goals during the EA’s stages.

  • Diversity measures the evolutionary stage:

» Percentage Fitness Evaluations (PFE) » Genotypic Diversity (GD)

  • FLC Control Variables: [EA Adaptable

Parameters]

− ∆N = Change in Population Size − ∆Pm = Change in Mutation Rate

Remarks (cont.)

  • Main Result
  • By using the FLC with the above State and

Control variables, we achieved a good improvement of the population average and an even better improvement of the population variance.

  • No major negative effects on EA performance

using FLC

  • Soft Computing Overview
  • SC Components: PR, FL, NN, EA
  • Modeling with FL and EA
  • Hybrid SC Systems
  • FLC Parameter Tuning by EA
  • EA Parameter Setting
  • Conclusions

Hybrid SC and EA – Outline (4)

slide-17
SLIDE 17

4/21/2005 17 Page 17 Synergy in SC: Reasons & Approaches

  • Hybrid Soft Computing
  • Leverages tolerance for imprecision, uncertainty, and

incompleteness - intrinsic to the problems to be solved

  • Generates tractable, low-cost, robust solutions to such

problems by integrating knowledge and data

  • Tight Hybridization
  • Data-driven Tuning of Knowledge-derived Models

» Translate domain knowledge into initial structure and parameters » Use Global or local data search to tune parameters

  • Knowledge-driven Search Control

» Use Global or local data search to derive models (Structure + Parameters) » Translate domain knowledge into an algorithm’s controller to improve/manage solution convergence and quality

Synergy in SC: Reasons & Approaches

  • Loose Hybridization (Model Fusion)
  • Does not combine features of methodologies - only their

results

  • Their outputs are compared, contrasted, and

aggregated, to increase reliability

  • Hybrid Search Methods
  • Intertwining local search within global search
  • Embedding knowledge in operators for global search
  • Future:
  • Circle of SC's related technologies will probably widen

beyond its current constituents.

  • Push for low-cost solutions and intelligent tools will result

in deployment of hybrid SC systems that efficiently integrate reasoning and search techniques.

FL Controllers tuned by EAs (cont.)

  • Historical Approaches (cont.):
  • Kinzel et al. 94:

» Chromosome = Rule Table » Point-radius crossover changing 3x3 rule window (similar to a two-point crossover for string representation) » Order of tuning:

– Initialize rulebase according to heuristics – Apply GAs to find best rule table – Tune membership function of best rule set

  • Herrera et al. 95:

» Chromosome = concatenation of all rules » Real-valued encoding, Max-min arithmetical crossover

Evolutionary Algorithms: ES

  • Evolutionary Strategies (ES)
  • Originally proposed for the optimization of

continuous functions

  • (µ , λ)-ES and (µ + λ)-ES

» A population of µ parents generate λ offspring » Best µ offspring are selected in the next generation » (µ , λ)-ES: parents are excluded from selection » (µ + λ)-ES: parents are included in selection

  • Started as (1+1)-ES (Reschenberg) and evolved

to (µ + λ)-ES (Schwefel)

  • Started with Mutation only (with individual

mutation operator) and later added a recombination operator

  • Focus on behavior of individuals

Evolutionary Algorithms: EP

  • Evolutionary Programming (EP)
  • Originally proposed for sequence predictiom

and optimal gaming strategies

  • Currently focused on continuous parameter
  • ptimization and training of NNs
  • Could be considered a special case of (µ + µ)
  • ES without recombination operator
  • Focus on behavior of species (hence no

crossover)

  • Proposed by Larry Fogel (1963)
slide-18
SLIDE 18

4/21/2005 18 Page 18

Evolutionary Algorithms: GA

  • Genetic Algorithms (GA)
  • Perform a randomized search in solution space using a

genotypic rather than a phenotypic

  • Each solution is encoded as a chromosome in a population (a

binary, integer, or real-valued string) » Each string’s element represents a particular feature of the solution

  • The string is evaluated by a fitness function to determine the

solution’s quality » Better-fit solutions survive and produce offspring » Less-fit solutions are culled from the population

  • Strings are evolved using mutation & recombination operators.
  • New individuals created by these operators form next generation of

solutions

  • Started by Holland (1962; 1975)

Evolutionary Algorithms: GP

  • Genetic Programming (GP)
  • A special case of Genetic Algorithms

»Chromosomes have a hierarchical rather than a linear structure »Their sizes are not predefined »Individuals are tree-structured programs »Modified operators are applied to sub-trees or single nodes

  • Proposed by Koza (1992)
  • GA Structural Design Selections:
  • Parent Selection Method:

»{Proportional Roulette, Tournament, Rank, Uniform, ... }

  • Crossover Operator:

»{Once-cut, Two-cuts, Uniform, BLX, Parent Weighted, ...}

  • Mutation Operator:

»Mutation Rate: {Exponentially Decreasing, Uniform, ..} »Value: {Exponentially Decreasing, Uniform, Normally Distributed, …}

GA Structure (cont.)

  • Other possible parameters that could be

adjusted:

  • T

= Number of Trials = Σ Ni where N is population size and i= 1, Max_Gen σµ = Mutation step σ in Normally distributed Mutation value

  • PS = Probability of Selection

Parametrized slope of probability distribut.

  • AS = Arity of Parents

number of parents in recombination

GA Parameters (cont.)

GD = Genotypic Diversity: Normalized Average Hamming Distance PFE = Percentage Fitness Evaluations: (Completed # Trials) / (Max Allocated # Trials) ∆Pm = Change in Mutation Rate

GD, PFE−>∆Pm

Genotypic Percentage Fitness Evaluation (PFE) Diversity (GD) Very Low Low Medium High Very High Very Low

Pos High Pos High Pos Medium Pos Medium No Change

Low

Pos High Pos Medium Pos Medium No Change No Change

Medium

Pos Medium Pos Medium No Change No Change No Change

High

Pos Medium No Change No Change No Change No Change

Very High

No Change No Change No Change No Change No Change

Fuzzy Controller for Mutation Rate: Rule Set

  • GA Parameters
  • N = Base Population size: 50
  • Pc = Crossover rate:

0.600

  • Pm = Mutation rate:

0.005

  • G = Generation Gap

100% replacement

  • Simple GA (SGA)

25% replacement

  • Steady State GA (SSGA)
  • S = Selection Strategy:

Elitist

Statistical Experiments: Set-Up (cont.)

slide-19
SLIDE 19

4/21/2005 19 Page 19

J = C*e(T-10)/3

Summary of 1200 Experiments

Max Numb U-SGA FLC-SGA U-SSGA FLC-SSGA Trials B µ σ σ/µ B µ σ σ/µ B µ σ σ/µ B µ σ σ/µ 3000 0% 1788.8 71 0.040 20% 1729 81 0.047 70% 1685 63 0.037 80% 1677 58 0.034 5000 5% 1767.9 103 0.058 35% 1705 74 0.043 75% 1682 63 0.037 80% 1673 47 0.028 7000 35% 1710.3 81 0.047 45% 1680 41 0.025 60% 1739 108 0.062 95% 1665 45 0.027 9000 20% 1748.8 102 0.058 50% 1676 46 0.027 80% 1695 82 0.048 85% 1689 70 0.041 11000 50% 1719.5 88 0.051 75% 1668 40 0.024 75% 1709 98 0.058 95% 1665 45 0.027 Max Numb U-SGA FLC-SGA U-SSGA FLC-SSGA Trials B µ σ σ/µ B µ σ σ/µ B µ σ σ/µ B µ σ σ/µ 3000 5% 352.5 20.9 0.059 15% 341.3 12.0 0.035 50% 343.8 23.0 0.067 75% 332.4 4.6 0.014 5000 30% 338.6 8.0 0.024 30% 337.7 7.9 0.023 60% 336.2 11.6 0.034 85% 331.8 4.1 0.012 7000 20% 339.7 1.9 0.005 30% 338.8 1.7 0.005 70% 341.3 5.2 0.015 70% 336.3 3.4 0.010 9000 30% 343.2 15.74 0.046 50% 333.9 5.0 0.015 80% 335.2 15.3 0.046 60% 334.6 5.6 0.017 11000 65% 337.0 15.3 0.045 60% 331.7 3.5 0.010 65% 336.9 15.3 0.045 65% 336.9 15.3 0.045 Max Numb U-SGA FLC-SGA U-SSGA FLC-SSGA Trials B µ σ σ/µ B µ σ σ/µ B µ σ σ/µ B µ σ σ/µ 3000 0% 655.05 90.2 0.138 5% 638.2 87.7 0.137 15% 592.0 53.0 0.090 80% 554.3 14.9 0.027 5000 10% 625.1 95.0 0.152 25% 600.8 47.6 0.079 35% 597.1 91.5 0.153 55% 570.8 24.7 0.043 7000 20% 606.84 97.9 0.161 20% 566.4 22.3 0.039 70% 563.6 22.8 0.040 65% 566.0 23.7 0.042 9000 30% 569.14 29.8 0.052 50% 573.9 41.8 0.073 85% 556.3 17.7 0.032 50% 573.2 24.9 0.043 11000 25% 608.35 129.4 0.213 40% 573.0 35.7 0.062 60% 568.4 24.4 0.043 70% 563.6 22.7 0.040 7% 47% 60% 7% 20% 47% Significant change in µ Significant change in σ

J = C*T J = C*T2 J = C*T2 J = C*e(T-10)/3

Next Steps: Controlling Other Parameters

  • Run-time Controlled GAs Parameters:
  • Population size:

» larger size: increase parallel search in solution space » smaller size: focus on current existing regions

  • Probability mutation:

» Higher prob. of mutation disrupts current solutions - exploration » Lower probability of mutation favors current solutions - exploitation

  • Other Possible Run-time Controllable GAs Parameters:
  • Customized mutation operators:

» Variable amount of changes

– smaller for good solutions, larger for bad ones

  • Fitness function:

» Evolving fitness function (variable weights in multi-criteria

aggregating function)

DONE DONE

GAs controlled by FL (cont.)

  • Probability of Selection:

» Parametrized slope distribution ranging from:

– Uniform probability: ignore fitness function and perform random selection of parents - extreme case of exploration, to – Proportional selection with rescaling and other intermediate strategies - compromise between exploration and exploitation cases, and – Ranking: always select the best N and ignore the rest - extreme case of exploitation

» Probability as function of fitness and genotypical distance with other solutions - enforcing diversity and favoring exploration

  • Probability of crossover:

» Constraints applicability to mostly good solutions

  • Customized-crossover operators (for real-coded GAs):

» Selection of crossovers based on T-norms and T-conorms causes

  • ffsprings to take more extreme values (exploration)

» Selection of crossovers based on aggregating operators causes

  • ffsprings to take average values (exploitation)

Frequency of Control Actions Control Action:

mutation rate changed every 10 generations population size change every generation

Mutation Rate Mutation rates drops exponentially after a control action that increases it Inference Engine Parameters Left Hand Side (LHS) evaluation: Minimum operator Rule Firing: Minimum operator Rule Output Aggregation: Maximum operator Defuzzification: Center of Gravity (COG)

Fuzzy Controller for ∆N and ∆Pm: Control Parameters

  • Outputs

∆N = Change in Population Size (Mult. Factor) ∆Pm = Change in Mutation Rate (Mult. Factor)

∆ N range is [0.5, 1.5] == [Neg High, Pos High]

  • so that NC corresponds to 100% of previous Pop Size

∆ N range is [0.5, 1.5] == [Neg High, Pos High] so that NC corresponds to 100% of previous Pop Size Population Size is clamped within [25, 150] ∆ Pm range is [0.5, 1.5] == [Neg High, Pos High] so that NC corresponds to 100% of previous Pm Mutation Rate is clamped within [0.005, 0.10]

Fuzzy Controller for ∆N and ∆Pm: Outputs

  • Develop Collection of Quasi-independent Models
  • Each Model Generates:
  • Output Value (Vi ) - Prediction
  • Confidence parameter (Ci ) derived from training stats. - Introspection
  • Intelligent Fusion Rules
  • Consider discrepancies among Output values (v)
  • Consider dynamic confidence value (c) associated with each output

Loc Val AIGEN AICOMP FUSION RULES

Living Area Address (GeoCoded ) Lot Size # Beds # Baths, ... Pool Conditions ...

eL eG eC eF

Example of Fusion for Mortgage Collateral Evaluation

ei = { Vi , Ci }

Fusion of Reasoning Models

slide-20
SLIDE 20

4/21/2005 20 Page 20 Synergy in SC: Reasons & Approaches

  • SC Leverages Knowledge and Data to Derive the Model
  • Model = Structure + Parameters (& Search)
  • Data-driven Tuning of Knowledge-derived Models
  • Translate domain knowledge to initial structure & parameters
  • Use Global or local data search to tune parameters
  • Knowledge-driven Search Control
  • Use Global or local data search to derive models (Structure +

Parameters)

  • Translate domain knowledge into an algorithm’s controller to

improve/manage solution convergence and quality

  • Hybrid Search Methods
  • Embedding local search within global search
  • Embedding knowledge in operators for global search
  • Fusion of models to increase accuracy and reliability