1 Recent Advances in Reactive Planning: Past Approaches to Planning - - PDF document

1
SMART_READER_LITE
LIVE PREVIEW

1 Recent Advances in Reactive Planning: Past Approaches to Planning - - PDF document

Massachusetts Institute of Technology Motivation for Reactive Planning Factored Symbolic Approach to Reactive Planning Seung H. Chung Brian C. Williams Reason for Planning Reason for onboard reactive planning Anomalies June 7,


slide-1
SLIDE 1

1

June 7, 2005

Massachusetts Institute of Technology

Factored Symbolic Approach to Reactive Planning

Seung H. Chung Brian C. Williams

2

Motivation for Reactive Planning

  • Reason for Planning

– Anomalies

  • Environmental
  • System

– May require repair and/or reconfiguration capabilities.

  • Reason for onboard

reactive planning

– Time-critical situations – Communication time delays – Situation in which no communication available

3

Model-base Programs Interact Directly with States

  • Embedded programs interact with

plant sensors and actuators:

– Read sensors – Set actuators

Complexity: Programmer must map between state and sensors/actuators.

  • Model-based programs interact

with plant state:

– Read state – Write state

Simplification: Model-based executive maps between state and sensors/actuators.

Embedded Program S Plant Observations Command Model-based Embedded Program S Plant Observations Command Ŝ Model-based Executive

4

Increase Robustness through Model-based Programming

  • Raised Level of Abstraction

– Code in terms of desired state evolution – Fewer lines of code – Less chance of introducing bugs

  • Executable Specification

– Increase robustness by synthesizing executable code from the verified specification – Models are the specification of the system – Model-based Executive operates

  • n the model, i.e. the

specification – The model-based embedded program is guaranteed to meet the specification Model-based Embedded Program S Plant Observations Command Ŝ Model-based Executive

5

Model-based Executive: Deductive Controller

Model-based Programming of Intelligent Embedded Systems and Robotic Explorers [Williams et al., IEEE’03]

Mode Estimation Command Configuration Goals Observation Mode Reconfiguration State Estimate System Model (CCA) System Mode Reconfiguration Command Configuration Goals

control nondeterministic system in a nondeterministic environment

6

Mode Reconfiguration

Model-based Programming of Intelligent Embedded Systems and Robotic Explorers [Williams et al., IEEE’03]

Goal Interpreter Reactive Planner

Configuration Goal Command Goal State State Estimate (Current)

Reactive Planner

Command Goal State State Estimate (Current)

slide-2
SLIDE 2

2

7

Past Approaches to Planning

  • General-purpose Planner

– Generates a sequence of commands that achieves the goal. – A sequence of commands lacks robustness within nondeterministic system and environment. – Replanning is expansive.

  • Universal Planner

– Maps all possible initial states to the appropriate actions. – State explosion problem

  • Assume:

– x components – in average n number of states per component

  • Number of system states: O(nx)

– Must replan if the goal state changes.

8

Recent Advances in Reactive Planning: BDD-based Universal Planning

  • Ordered Binary Decision Diagrams (BDD)

– Compact representation of Boolean functions – Efficient algorithms for operating on Boolean functions

  • Symbolic Model Checking

– Use of BDDs for model checking – Reduce the state explosion problem – Has been very successful [Burch et al., IEEE’90]

  • Recognized the similarity of Symbolic Model Checking and

Planning [Cimatti et al., ECP’97]

– Reduce the state explosion problem through the use of BDDs.

  • BDD-based Universal planners have been developed:

– Strong Plan, Strong Cyclic Plan, Optimistic Planning, Etc.

A = on A' = on cmdA = on 1 B = on

9

Recent Advances in Reactive Planning: Burton [Williams & Nayak, IJCAI97]

  • Goal-directed plan :

〈Current State, Goal State〉 → Action

  • Introduced a decomposition technique that enables

subgoal serialization (i.e. in essence, applies a divide-and-conquer approach to reactive planning).

– Mitigate the state space explosion problem. – Enable a compact encoding of a goal-directed plan.

  • Only applicable to a limited subset of a planning

problems (i.e. cannot generate a plan for a system with interdependent components).

10

Factored Symbolic Approach to Reactive Planning

  • Unify the two complementary approaches:

– Address the state space explosion problem at the global level through decomposition : divide-and-conquer – Address the state space explosion problem at the subproblem level though BDD-based planning

  • Extend the decomposition technique of [Williams &

Nayak, IJCAI97] to problems with interdependent components.

  • Extend the BDD-based Universal planning

technique to generate a goal-directed plan.

11

Outline

  • Spacecraft telecommunication

system

  • Model: Concurrent automata
  • Decomposing the Problem:

Transition dependency graph

  • Reactive Plan for a

Subproblem: Goal-directed plan

  • Reactive Plan for the Problem:

Decomposed goal-directed plan

  • Executing the Plan

Compute Transition Dependency Α Α Α Α Α Α Α Α Α Α Α Α Α Α Α Α Α Α Α Α Α Α Decompose Serialize (Topological Order) Α Α Α

1 Α Α Α Α 2 Α Α Α 3 Α 4

Compute DGDP GDP GDP

1 GDP GDP 2 GDP GDP 3 GDP GDP 4

Concurrent Automata (ΧΑ) Α Α Α Α Α Α Α Α Α Α Α

12

Telecommunication Subsystem Example

  • Computer

– Controls the devices and sends data to the devices.

  • Bus Controller

– Routes the commands and the data to the appropriate devices.

  • Transmitter

– Generates a signal that corresponds to the data to be transmitted.

  • Amplifier

– Amplifies the signal and transmits it to an antenna.

Bus Controller Computer Antenna Antenna Amplifier Transmitter Amplifier Transmitter Bus Controller Amplifier Transmitter Amplifier Transmitter

slide-3
SLIDE 3

3

13

Concurrent Automata (CA)

  • Synchronous

– Assume that each automaton performs a single state transition at each time step.

  • Interleaved execution within a time step

– A single main processor executes synchronous activities by interleaving. – Devices are not synchronized.

  • ff
  • n

Transmitter Amplifier B = on A = off cmdT = off B = on A = off cmdT = on

  • ff
  • n

B = on cmdA = off B = on T = on cmdA = on

  • ff
  • n

Bus Controller cmdB = off cmdB = on

14

Interdependent Components

  • Turning the transmitter on or off can

generate a noise (i.e. transient signal).

  • The transient signal may damage the

amplifier.

  • The amplified transient signal may

damage other devices down stream of the amplifier.

  • Constraint on the system:

– The amplifier must be turned off before the transmitter can be turned on or off. – The transmitter must be turned on before the amplifier can be turned on.

  • ff
  • n

Transmitter Amplifier B = on A = off cmdT = off B = on A = off cmdT = on

  • ff
  • n

B = on cmdA = off B = on T = on cmdA = on

15

BDD Encoding of a Concurrent Automaton

A = on A' = on cmdA = on 1 B = on

  • ff
  • n

B = on cmdA = off B = on T = on cmdA = on

T = on A = on A' = on cmdA = on cmdA = on A' = on 1 B = on A = on A' = on

16

Transition Dependency Graph

  • Transition Dependency Graph (TDG)

– Vertex: for each automaton – Edge (v, u): if a transition of the automaton v is conditioned on the state of automaton u.

  • Use Strongly Connected Components (SCC) algorithm to find the cyclic

components.

  • Compose SCC concurrent automata

– New TDG is acyclic. – Serialize the subgoals in the inverse topological ordering.

Computer Antenna Antenna Bus Controller Amplifier Transmitter Amplifier Transmitter

1 2 3

17

Subgoal Serialization

  • Goal:

– Bus Controller = on – Transmitter/Amplifier #1 = (on, on) – Transmitter/Amplifier #2 = (off, off)

  • Solve each subgoal sequentially in the inverse topological order

Computer Antenna Antenna Bus Controller Amplifier Transmitter Amplifier Transmitter

1 2 3

18

Composing Strongly Connected CA

  • Compose all automata into a single automaton

RSCC = ∧ Ri

Transmitter Amplifier

  • ff
  • n

B = on A = off cmdT = off B = on A = off cmdT = on

  • ff
  • n

B = on cmdA = off B = on T = on cmdA = on B = on cmdA = off

  • nT
  • nA
  • nT
  • ffA
  • ffT
  • ffA
  • ffT
  • nA

B = on cmdT = off B = on cmdT = on B = on cmdA = off B = on cmdA = on

slide-4
SLIDE 4

4

19

Interdependent Concurrent Transitions

One Transition Missing!

  • nT
  • nA
  • nT
  • ffA
  • ffT
  • ffA
  • ffT
  • nA

B = on cmdT = off B = on cmdT = on B = on cmdA = off B = on cmdA = on B = on cmdA = off

  • ff
  • n

B = on A = off cmdT = off B = on A = off cmdT = on

  • ff
  • n

B = on cmdA = off B = on T = on cmdA = on A = off A = on B = on T = on cmdA = on T = off T = on B = on A = off cmdT = off T = off A = on T = on A = off B = on cmdT = off cmdA = on

20

A = off A = on B = on T = on cmdA = on T = off T = on B = on A = off cmdT = off T = off A = on T = on A = off B = on cmdT = off cmdA = on

Hazard!

Simultaneous Commanding

  • Both the transmitter and the amplifier depend on one another for the

transition.

  • The transmitter must be commanded “off” and the amplifier must be

commanded “on” precisely at the same time.

  • Due to concurrency via interleaving, simultaneous commanding cannot

be guaranteed.

  • If the amplifier were commanded on first, and then the transmitter is

commanded off, the amplifier can be damaged.

21

Assuring Proper Execution of Interdependent Transitions

  • Enforce concurrency as interleaving:

– For a given transition, the interdependent state constraints become the pre- and post-conditions. – No change to all other automata that are not independent. Amplifier Transmitter

A = off A = on B = on T = on T = off T = on B = on A = off cmdA = on cmdT = off T = on A = off T = on A = on B = on T = off A = off T = on A = off B = on cmdA = on cmdT = off

22

Assuring Proper Execution of Interdependent Transitions

  • Inconsistencies are automatically detected when conjoining

the transition relations in BDDs. RSCC = ∧ Ri

  • Similar to the Graphplan mutual exclusion rule.

– Interference:

  • One transition deletes the precondition and/or effect of another.

– Competing Needs:

  • Inconsistent preconditions

T = on A = off T = on A = on B = on T = off A = off T = on A = off B = on cmdA = on cmdT = off T = off A = on T = on A = off B = on cmdT = off cmdA = on

23

Goal-directed Plan

  • Goal-directed Plan 〈s,a,s’〉 :

〈s, s’〉 → a

– s : current state – s’ : goal state – a : first action/intermediate subgoals in a trajectory that eventually leads to s’

B = on cmdT = on

Goal

B = on cmdT = on B = on cmdA = on idle idle B = on cmdA = off

Current

OnT, OnA OnT, OffA OffT, OffA OnT, OnA OnT, OffA

idle B = on cmdT = off B = on cmdA = off

OffT, OffA

fail fail fail

OffT, OnA

B = on cmdA = off B = on cmdA = off B = on cmdA = off idle

OffT, OnA

  • nT
  • nA
  • nT
  • ffA
  • ffT
  • ffA
  • ffT
  • nA

B = on cmdT = off B = on cmdT = on B = on cmdA = off B = on cmdA = on B = on cmdA = off

  • Executing a goal-directed plan

guarantees:

– Progress toward the goal. – Finite number of actions to achieve the goal. – Optimal (shortest) trajectory under nominal conditions.

24

n steps

Computing Goal-Directed Plan:

COMPUTEGDP(T)

  • Iteratively search backward breath-first for the goal-directed

rules.

– Find 〈s,a,s’〉 that can reach s’ within 1 step – Find 〈s,a,s’〉 that can reach s’ within 2 steps – … – Find 〈s,a,s’〉 that can reach s’ within n steps

1 step 2 steps 3 steps … n-1 steps

slide-5
SLIDE 5

5

25

Generating Goal-Directed Plan

  • 〈s,a,s’〉 that can reach s’ within 1 step

Transition Relation

B = on cmdT = on

Goal

B = on cmdT = on B = on cmdA = on idle idle B = on cmdA = off

Current

OnT, OnA OnT, OffA OffT, OffA OnT, OnA OnT, OffA

idle B = on cmdT = off B = on cmdA = off

OffT, OffA

fail fail fail

OffT, OnA

B = on cmdA = off B = on cmdA = off B = on cmdA = off idle

OffT, OnA

  • nT
  • nA
  • nT
  • ffA
  • ffT
  • ffA
  • ffT
  • nA

B = on cmdT = off B = on cmdT = on B = on cmdA = off B = on cmdA = on B = on cmdA = off

B = on cmdT = on B = on cmdA = on idle idle B = on cmdA = off idle B = on cmdT = off B = on cmdA = off idle

26

Generating Goal-Directed Plan

  • 〈s,a,s’〉 that can reach s’ within 2 steps

– To the previous GDP add 〈s,a,s’〉 that can reach s’ in 2 steps:

  • s: current state
  • s’: goal state that can be reached in 2 steps
  • a: first control action that must be commanded to eventually reach s’
  • nT
  • nA
  • nT
  • ffA
  • ffT
  • ffA
  • ffT
  • nA

B = on cmdT = off B = on cmdT = on B = on cmdA = off B = on cmdA = on B = on cmdA = off

B = on cmdT = on

Goal Current

OnT, OnA OnT, OffA OffT, OffA OnT, OnA OnT, OffA

B = on cmdA = off

OffT, OffA

fail fail fail

OffT, OnA

B = on cmdA = off B = on cmdA = off

OffT, OnA

B = on cmdT = on B = on cmdA = on idle idle B = on cmdA = off idle B = on cmdT = off B = on cmdA = off idle B = on cmdT = on B = on cmdA = off B = on cmdA = off

27

Generating Goal-Directed Plan

  • 〈s,a,s’〉 that can reach s’ within 3 steps

– To the previous GDP add 〈s,a,s’〉 that can reach s’ in 3 steps:

  • s: current state
  • s’: goal state that can be reached in 3 steps
  • a: first control action that must be commanded to eventually reach s’
  • nT
  • nA
  • nT
  • ffA
  • ffT
  • ffA
  • ffT
  • nA

B = on cmdT = off B = on cmdT = on B = on cmdA = off B = on cmdA = on B = on cmdA = off

Goal Current

OnT, OnA OnT, OffA OffT, OffA OnT, OnA OnT, OffA OffT, OffA

fail fail fail

OffT, OnA

B = on cmdA = off

OffT, OnA

B = on cmdT = on B = on cmdA = on idle idle B = on cmdA = off idle B = on cmdT = off B = on cmdA = off idle B = on cmdT = on B = on cmdA = off B = on cmdA = off B = on cmdA = off

28

Generating Goal-Directed Plan

  • 〈s,a,s’〉 that can reach s’ within 4 steps

– No new 〈s,a,s’〉 exists that can reach s’ in 4 steps. – When the fixed-point is reached, generating the plan is complete.

  • nT
  • nA
  • nT
  • ffA
  • ffT
  • ffA
  • ffT
  • nA

B = on cmdT = off B = on cmdT = on B = on cmdA = off B = on cmdA = on B = on cmdA = off

Goal Current

OnT, OnA OnT, OffA OffT, OffA OnT, OnA OnT, OffA OffT, OffA

fail fail fail

OffT, OnA

B = on cmdA = off

OffT, OnA

B = on cmdT = on B = on cmdA = on idle idle B = on cmdA = off idle B = on cmdT = off B = on cmdA = off idle B = on cmdT = on B = on cmdA = off B = on cmdA = off

29

Computing Decomposed Goal-directed Plan

  • For each automaton

compute a GDP.

Computer Antenna Antenna Bus Controller Amplifier Transmitter Amplifier Transmitter 1 2 3

Goal

cmdB = on idle idle cmdB = off

Current

On Off On Off

Bus Controller

B = on cmdT = on

Goal

B = on cmdT = on B = on cmdA = on idle idle B = on cmdA = off

Current

OnT, OnA OnT, OffA OffT, OffA OnT, OnA OnT, OffA

idle B = on cmdT = off B = on cmdA = off

OffT, OffA

fail fail fail

OffT, OnA

B = on cmdA = off B = on cmdA = off B = on cmdA = off idle

OffT, OnA

Transmitter & Amplifier

30

Size of DGDP

  • Given

– Number of concurrent automata : n – Average number of states in each automaton : m – Number of strongly connected components : l – Average number automata in a strongly connected component : w – Number of states for one composed automaton : O(mw) – Size of a GDP : O(m2w) – Size of DGDP : O(l · m2w)

  • Approximately linear in the number of components

– Assume m and w are constant. – O(l · m2w) is linear in l < n.

  • Use of BDD makes each GDP even more compact.
Computer Antenna Antenna Bus Controller Amplifier Transmitter Amplifier Transmitter 1 2 3
slide-6
SLIDE 6

6

31

Execution

  • Achieve subgoals incrementally in the inverse topological
  • rder → subgoal serialization ordering
  • 1. Transmitter/Amplifier #2
  • 2. Transmitter/Amplifier #1
  • 3. Bus Controller

Goal

cmdB = on idle idle cmdB = off

Current

On Off On Off

Bus Controller

B = on cmdT = on

Goal

B = on cmdT = on B = on cmdA = on idle idle B = on cmdA = off

Current

OnT, OnA OnT, OffA OffT, OffA OnT, OnA OnT, OffA

idle B = on cmdT = off B = on cmdA = off

OffT, OffA

fail fail fail

OffT, OnA

B = on cmdA = off B = on cmdA = off B = on cmdA = off idle

OffT, OnA

Transmitter & Amplifier

Computer Antenna Antenna Bus Controller Amplifier Transmitter Amplifier Transmitter 1 2 3

32

Execution Example

  • Current State:

– B = off – T/A #1 = (off, off) – T/A #2 = (off, off)

  • Goal State:

– B = off – T/A #1 = (on,on) – T/A #2 = (off, off) Goal

cmdB = on idle idle cmdB = off

Current

On Off On Off

Bus Controller

B = on cmdT = on

Goal

B = on cmdT = on B = on cmdA = on idle idle B = on cmdA = off

Current

OnT, OnA OnT, OffA OffT, OffA OnT, OnA OnT, OffA

idle B = on cmdT = off B = on cmdA = off

OffT, OffA

fail fail fail

OffT, OnA

B = on cmdA = off B = on cmdA = off B = on cmdA = off idle

OffT, OnA

Transmitter & Amplifier

Computer Antenna Antenna Bus Controller Amplifier Transmitter Amplifier Transmitter

1 2 3

cmdB = on

33

Execution Example

  • Current State:

– B = on – T/A #1 = (off, off) – T/A #2 = (off, off)

  • Goal State:

– B = off – T/A #1 = (on,on) – T/A #2 = (off, off) Goal

cmdB = on idle idle cmdB = off

Current

On Off On Off

Bus Controller

B = on cmdT = on

Goal

B = on cmdT = on B = on cmdA = on idle idle B = on cmdA = off

Current

OnT, OnA OnT, OffA OffT, OffA OnT, OnA OnT, OffA

idle B = on cmdT = off B = on cmdA = off

OffT, OffA

fail fail fail

OffT, OnA

B = on cmdA = off B = on cmdA = off B = on cmdA = off idle

OffT, OnA

Transmitter & Amplifier

Computer Antenna Antenna Bus Controller Amplifier Transmitter Amplifier Transmitter

1 2 3

cmdT = on

34

Execution Example

  • Current State:

– B = on – T/A #1 = (on, off) – T/A #2 = (off, off)

  • Goal State:

– B = off – T/A #1 = (on,on) – T/A #2 = (off, off) Goal

cmdB = on idle idle cmdB = off

Current

On Off On Off

Bus Controller

B = on cmdT = on

Goal

B = on cmdT = on B = on cmdA = on idle idle B = on cmdA = off

Current

OnT, OnA OnT, OffA OffT, OffA OnT, OnA OnT, OffA

idle B = on cmdT = off B = on cmdA = off

OffT, OffA

fail fail fail

OffT, OnA

B = on cmdA = off B = on cmdA = off B = on cmdA = off idle

OffT, OnA

Transmitter & Amplifier

Computer Antenna Antenna Bus Controller Amplifier Transmitter Amplifier Transmitter

1 2 3

cmdA = on

35

Execution Example

  • Current State:

– B = on – T/A #1 = (on, on) – T/A #2 = (off, off)

  • Goal State:

– B = off – T/A #1 = (on,on) – T/A #2 = (off, off) Goal

cmdB = on idle idle cmdB = off

Current

On Off On Off

Bus Controller

B = on cmdT = on

Goal

B = on cmdT = on B = on cmdA = on idle idle B = on cmdA = off

Current

OnT, OnA OnT, OffA OffT, OffA OnT, OnA OnT, OffA

idle B = on cmdT = off B = on cmdA = off

OffT, OffA

fail fail fail

OffT, OnA

B = on cmdA = off B = on cmdA = off B = on cmdA = off idle

OffT, OnA

Transmitter & Amplifier

Computer Antenna Antenna Bus Controller Amplifier Transmitter Amplifier Transmitter

1 2 3

cmdB = off

36

Execution Example

  • Current State:

– B = off – T/A #1 = (on, on) – T/A #2 = (off, off)

  • Goal State:

– B = off – T/A #1 = (on,on) – T/A #2 = (off, off) Goal

cmdB = on idle idle cmdB = off

Current

On Off On Off

Bus Controller

B = on cmdT = on

Goal

B = on cmdT = on B = on cmdA = on idle idle B = on cmdA = off

Current

OnT, OnA OnT, OffA OffT, OffA OnT, OnA OnT, OffA

idle B = on cmdT = off B = on cmdA = off

OffT, OffA

fail fail fail

OffT, OnA

B = on cmdA = off B = on cmdA = off B = on cmdA = off idle

OffT, OnA

Transmitter & Amplifier

Computer Antenna Antenna Bus Controller Amplifier Transmitter Amplifier Transmitter

1 2 3

Done!

slide-7
SLIDE 7

7

37

DGDP Execution Capability

  • Time complexity of one execution cycle.

– GDP rule lookup is polynomial execution. – DGDP execution is O(l), where l is the number of GDPs.

  • DGDP is capable of real-time repair and reconfiguration.

– Repair capability is necessary when anomalies occur during execution time (e.g. T/A #1 fails into a reparable state). – Reconfiguration capability is necessary when goal-states change quickly (e.g. turn on T/A #2 instead).

B = on cmdT = on

Goal

B = on cmdT = on B = on cmdA = on idle idle B = on cmdA = off

Current

OnT, OnA OnT, OffA OffT, OffA OnT, OnA OnT, OffA

idle B = on cmdT = off B = on cmdA = off

OffT, OffA

fail fail fail

OffT, OnA

B = on cmdA = off B = on cmdA = off B = on cmdA = off idle

OffT, OnA

Transmitter & Amplifier

Computer Antenna Antenna Bus Controller Amplifier Transmitter Amplifier Transmitter 1 2 3

38

Conclusion

  • Factored Symbolic approach to Reactive Planning

enables:

– Compact decomposed goal-directed plan compilation through:

  • Decomposition
  • BDD encoding

– Real-time execution capabilities:

  • Reactive repair
  • Reactive reconfiguration
  • Approximately linear in the number of components
  • Possible Extension

– Add probability and utility using ADD