Dynamic and Adversarial Reach- avoid Symbolic Planning Laya Shamgah - - PowerPoint PPT Presentation

Dynamic and Adversarial Reach- avoid Symbolic Planning

Laya Shamgah Advisor: Dr. Karimoddini

1

July 21st 2017

Thrust 1: Modeling, Analysis and Control of Large-scale Autonomous Vehicles (MACLAV) Sub-trust 1-2: Cooperative Localization, Navigation and Control of LSASVs

Dynamic and Adversarial Reach-avoid Symbolic Planning Laya Shamgah 2

Motivation

Reach-avoid Problem:

Traveling from an initial point to a desired location while avoiding obstacles Challenge:

Autonomous Coordination of autonomous vehicles to achieve their sophisticated goals in an dynamic and adversarial environment

Static Environment Dynamic Environment Dynamic Adversarial Environment

Objective of research: To develop a computationally effective reactive planning method for autonomous vehicles in a dynamic adversarial environment. Dynamic Adversarial Reach-avoid scenario:

Objective

3

• attacker: tries to reach the target while avoiding of

capture.

• defender: tries to capture the attacker before

reaching the defending area.

Dynamic and Adversarial Reach-avoid Symbolic Planning Laya Shamgah 3

Challenges and Gaps

4

 Solving only the avoidance problem  Assuming limitations on the vehicle’s movements  Requiring information about the opponent vehicle  High computational cost  Lack of Reactiveness Existing methods

Pursuit-evasion games

[Bhadauria et al. 2012]

Probabilistic approaches

[Vitus et sl. 2011]

Differential games

[Tomlin et al.2011,2015]

Dynamic and Adversarial Reach-avoid Symbolic Planning Laya Shamgah 4

Challenges

Proposed approach

5

Dynamic and Adversarial Reach-avoid Symbolic Planning Laya Shamgah 5

Abstraction

To reduce the complexity:

1- Using Symbolic Control Techniques for abstraction of the (infinitely) large original problem to a (finite) small abstracted environment, 2- Designing a DES supervisor to achieve a complex task over an abstract environment 3- Projecting back the solution to the original domain.

Remark: This is the first result in the literature that uses symbolic control techniques for the reach-avoid problem.

𝐐

Target defender attacker

?

Proposed Hybrid Structure

6

Dynamic and Adversarial Reach-avoid Symbolic Planning Laya Shamgah 6

DES supervisor Vehicle Dynamics Interface Abstraction of Vehicle Dynamics

Bisumulation-based abstraction Discrete Signal Continuous Signals

Proposed Implementation Approach

7

Dynamic and Adversarial Reach-avoid Symbolic Planning Laya Shamgah 7

Hierarchical Control

Supervisor, operator,… high-level Controller Low-level Controller Symbolic Planning Temporal Logic

Supervisor Planner Real-time low-level controller

Reach-avoid Problem Description

8

Problem:

Design a controller to obtain trajectory 𝑦 𝑢 ∈ P = 𝑗=1,…,𝑜

𝑘=1,…,𝑛 𝑄𝑗𝑘, which satisfies

the objective of the defender.

Assumptions:

• Defender vehicle dynamics:

𝑦 𝑢 = 𝑔(𝑦 𝑢 , 𝑣(𝑢))

• Environment (P) is a bounded convex set
• Target is in a fixed position
• The initial position of the attacker and the defender are within P
• Defender vehicle has full observability over the position of the

attacker other

Dynamic and Adversarial Reach-avoid Symbolic Planning Laya Shamgah 8

𝐐

Target defender attacker

?

Proposed Framework

9

Dynamic and Adversarial Reach-avoid Symbolic Planning Laya Shamgah 9

Design Steps:

1.Extracting decision-making strategies 2.Construction of LTL Specification 𝝌 = 𝝌𝒃 → 𝝌𝒆 3.Checking realizability of 𝝌 4.Synthesizing the supervisor automaton G which satisfies 𝝌 5.Designing the hybrid controller

Step1: Optimal Decision-making Strategies

10

• Modeled as a finite two-player zero-sum game in matrix form
• Attacker is the maximizer player and Defender is the minimizer

Objective Function

𝑀 𝑦𝑏

′ , 𝑦𝑒 ′

= 𝑗𝑔 𝑦𝑏

′ , 𝑦𝑒 ′ ∈ 𝑄𝑗𝑘

∞ 𝑗𝑔𝑦𝑏

′ , 𝑦𝑢 ∈ 𝑄𝑗𝑘

𝛽 ∥ 𝑦𝑏

′ − 𝑦𝑒 ′ ∥ +

𝛾 ∥ 𝑦𝑏

′ −𝑦𝑢 ∥ + 𝛿 ∥ 𝑦𝑒 ′ − 𝑦𝑢 ∥ 𝑝𝑢ℎ𝑓𝑠𝑥𝑗𝑡𝑓

Dynamic and Adversarial Reach-avoid Symbolic Planning Laya Shamgah 10

𝐐𝟐𝟐 𝐐𝟐𝟑 𝐐𝟐𝟒 𝐐𝟑𝟐 ○a 𝐐𝟑𝟑 𝐐𝟑𝟒 * 𝐐𝟒𝟐 𝐐𝟒𝟑 𝐐𝟒𝟒

• d

Distance between the vehicles Distance between the attacker and the target Distance between the defender and the target

Step1: Decision-making

11

Dynamic and Adversarial Reach-avoid Symbolic Planning Laya Shamgah 11

Example:

𝛽 = 1 𝛾 = 1 𝛿 = 0.5 attacker 𝑸𝟑𝟑 𝑸𝟒𝟐 𝑸𝟐𝟐 defender 𝑸𝟒𝟑 3.414 4.650 4.650 𝑸𝟑𝟒 2 3.236 3.236 Optimization Parameters:

• Defender : min 𝑛𝑏𝑦 3.414 , 4.650 , 𝑛𝑗𝑜 2 , 3.236

= 3.236 → 𝑄23

• Attacker : max 𝑛𝑗𝑜 3.414 , 2 , 4.650 , 3.236

= 3.236 → 𝑄

11

Nash Equilibrium decision : (𝒃𝟐𝟐, 𝒆𝟑𝟒) Temporal formula

⃞(𝒃𝟑𝟐∧ 𝒆𝟒𝟒 → ⃝𝒆𝟑𝟒)

𝒐𝒏(𝒐𝒏 − 𝟑) games should be solved to calculate all the temporal transition rules.

𝐐𝟐𝟐 𝐐𝟐𝟑 𝐐𝟐𝟒 𝐐𝟑𝟐 ○a 𝐐𝟑𝟑 𝐐𝟑𝟒 * 𝐐𝟒𝟐 𝐐𝟒𝟑 𝐐𝟒𝟒

• d

Step 2: Construction of LTL Specification

Temporal logic:

Linear Temporal Logic (LTL) is a formal high-level language to describe many complex missions and a wide class of properties can be expressed by LTL:

Dynamic and Adversarial Reach-avoid Symbolic Planning Laya Shamgah 12

• Coverage: eventually visit all regions
• Sequencing: visit P2 before you go to P3
• Avoidance: until you go to P2 avoid P1 and P3

Cla lassical log logic:

• “I am hungry

ry” Tem emporal log logic

• "I am always hungry

ry“

• "I will eventually be hungry

ry“

• "I will be hungry

ry until I eat something"

Step 2: Construction of LTL Specification

Dynamic and Adversarial Reach-avoid Symbolic Planning Laya Shamgah 13

The LTL formulas (𝜒) are constructed over (Σ) using Boolean operators and temporal operators.

• 𝜯 : A finite set of atomic proposition: 𝑞 ∈ 𝛵 (𝑞 𝑑𝑏𝑜 𝑐𝑓 𝑓𝑗𝑢ℎ𝑓𝑠 𝑈 𝑝𝑠 𝐺)
• Boolean operators: negation (¬), disjunction (∨), conjunction (∧), implication

(→)

• Modal temporal operators: next (𝑃), until (𝒱), eventually (◊) and always (⎕)

Step 2: Construction of LTL Specification

Dynamic and Adversarial Reach-avoid Symbolic Planning Laya Shamgah 14

Temporal Operators:

Operators Definition Diagram ○𝛘

𝜒 is true in the next moment of time

□𝛘

𝜒 is true in all future moments

◊𝛘

𝜒 is true in some future moment

𝛘𝐯𝛚

𝜒 is true until ψis true

Step 2: Construction of LTL Specification

Dynamic and Adversarial Reach-avoid Symbolic Planning Laya Shamgah 15

Static Environment Vehicle φ Dynamic Environment vehicle φ = (𝜒𝑓→ 𝜒𝑡)

Reactive to changes in Dynamic Environment

Step 2: Construction of LTL Specification

Dynamic and Adversarial Reach-avoid Symbolic Planning Laya Shamgah 16

Reach-avoid Specification:

𝝌 = 𝝌𝒃 → 𝝌𝒆 𝝌𝒘 = 𝝌𝒋𝒐𝒋𝒖

𝒘

⋀ 𝝌𝒕𝒋𝒐𝒉

𝒘

⋀ 𝝌𝒖𝒇𝒔𝒏

𝒘

⋀ 𝝌𝒔𝒗𝒎

𝒘 ⋀ 𝝌𝒑𝒄𝒌 𝒘

𝐐

Target defender attacker

?

1 𝝌𝒋𝒐𝒋𝒖

𝒘

Boolean (𝑪) Initial position of vehicle 2 𝝌𝒕𝒋𝒐𝒉

𝒘

Temporal (□𝑼) Singularity constraint: At each time the vehicle can be in only one region 3 𝝌𝒖𝒇𝒔𝒏

𝒘

Temporal (□𝑼) Termination of the game 4 𝝌𝒔𝒗𝒎

𝒘

Temporal (□𝑼) Transitions rules over the partitioned area 5 𝝌𝒑𝒄𝒌

𝒘

Temporal (□ ◊ 𝑪) Objective of the vehicle

𝝌𝒃: all assumptions on the attacker 𝝌𝒆: all assumptions on the defender and its desired behavior

Step 3-4: Discrete Design Procedure

17

Dynamic and Adversarial Reach-avoid Symbolic Planning Laya Shamgah 17

Step 3: Checking realizability of 𝝌

Check if there exists any admissible behavior of the attacker such that no behavior of adapter can satisfy 𝝌𝒆.

Step 4: Synthesis of automaton G

• If 𝝌 is realizable then G = Q, q0, A, D, 𝜀, ℎ
• Synthesis Process: 𝐻𝑇 =< 𝑊, 𝐵, 𝐸, Θ, ρ𝑏, 𝜍𝑒, 𝜒 >
• 𝐻 ⊨ 𝜒
• Every path on G is a behavior of the attacker and the

corresponding behavior of the defender, which ends when the defender will win

Step 5: Hybrid Control Design

18

Dynamic and Adversarial Reach-avoid Symbolic Planning Laya Shamgah 18

Online implementation:

Attacker’s behavior Discrete path

Interface

Continuous path 𝒚(𝒖)

𝒃𝒋 → 𝒃𝒋+𝟐 𝒆𝒋 → 𝒆𝒋+𝟐

• Heading angle(𝜾)
• Velocity(𝒗)

Example: Description

19

Problem: Design a controller to obtain trajectory 𝑦(𝑢) which satisfies 𝜒 = 𝜒𝑏 → 𝜒𝑒 Operation region Initial positions Target

𝑄 =

𝑗,𝑘=1 3

𝑄𝑗𝑘

attacker: 𝑄

11

defender: 𝑄31 𝑄23

Dynamic and Adversarial Reach-avoid Symbolic Planning Laya Shamgah 19

𝐐𝟐𝟐 𝐐𝟐𝟑 𝐐𝟐𝟒 𝐐𝟑𝟐 𝐐𝟑𝟑 𝐐𝟑𝟒 𝐐𝟒𝟐 𝐐𝟒𝟑 𝐐𝟒𝟒 attacker defender

Example: Task Specification

20

𝝌 = 𝝌𝒃 → 𝝌𝒆

Dynamic and Adversarial Reach-avoid Symbolic Planning Laya Shamgah 20

𝝌𝒃= 𝝌𝒋𝒐𝒋𝒖

𝒃

⋀ 𝝌𝒕𝒋𝒐𝒉

𝒃

⋀ 𝝌𝒖𝒇𝒔𝒏

𝒃

⋀ 𝝌𝒔𝒗𝒎

𝒃 ⋀ 𝝌𝒑𝒄𝒌 𝒃

𝝌𝒋𝒐𝒋𝒖

𝒃

𝑏11 ∧ ¬𝑏12 ∧ ¬𝑏13 ∧ ⋯ ∧ ¬𝑏33 𝝌𝒕𝒋𝒐𝒉

𝒃

□[(𝑏11 ∧ ¬𝑏12 ∧ ¬𝑏13 ∧ ⋯ ∧ ¬𝑏33) ∨ …] 𝝌𝒖𝒇𝒔𝒏

𝒃

□[(𝑏23 →○ 𝑏23) ∧ □[(𝑏11 ∧ 𝑒11) →○ 𝑏11) ∨…] 𝝌𝒔𝒗𝒎

𝒃

□[(𝑏11 → (○ 𝑏12 ∨ ○ 𝑏21 )) ∧…] 𝝌𝒑𝒄𝒌

𝒃

□ ◊ 𝑈𝑠𝑣𝑓 𝝌𝒆= 𝝌𝒋𝒐𝒋𝒖

𝒆

⋀ 𝝌𝒕𝒋𝒐𝒉

𝒆

⋀ 𝝌𝒖𝒇𝒔𝒏

𝒆

⋀ 𝝌𝒔𝒗𝒎

𝒆 ⋀ 𝝌𝒑𝒄𝒌 𝒆

𝝌𝒋𝒐𝒋𝒖

𝒆

𝑒13 ∧ ¬𝑒11 ∧ ¬𝑒12 ∧ ⋯ ∧ ¬𝑒33 𝝌𝒕𝒋𝒐𝒉

𝒆

□[(𝑒11 ∧ ¬𝑒12 ∧ ¬𝑒13 ∧ ⋯ ∧ ¬𝑒33) ∨…] 𝝌𝒖𝒇𝒔𝒏

𝒆

□[((𝑏23 ∨ 𝑏11) ∧ 𝑒11)→○ 𝑒11) ∧ …] 𝝌𝒔𝒗𝒎

𝒆

□[((𝑏11 ∧ 𝑒31 ∧○ 𝑏21) →○ 𝑒21) ∧ … ] 𝝌𝒑𝒄𝒌

𝒆

□ ◊ [ 𝑏11 ∧ 𝑒11 ∨ 𝑏12 ∧ 𝑒12 ∨ … ]

all assumptions

• n the attacker

all assumptions

• n the defender

and its desired behavior

Example: Discrete Results

21

𝝌 = 𝝌𝒃 → 𝝌𝒆 is realizable.

Dynamic and Adversarial Reach-avoid Symbolic Planning Laya Shamgah 21

• 2. Automaton G:

1.

Example: Final Results (1)

22

Dynamic and Adversarial Reach-avoid Symbolic Planning Laya Shamgah 22

Discrete path

Attacker 𝒃𝟐𝟐𝒃𝟐𝟑𝒃𝟐𝟒𝒃𝟐𝟑 Defender 𝒆𝟒𝟐𝒆𝟒𝟑𝒆𝟑𝟑𝒆𝟐𝟑

Continuous path

𝒃𝟐𝟐 𝒃𝟐𝟑 𝒃𝟐𝟒 𝒃𝟐𝟑

Example: Final Results (2)

23

Dynamic and Adversarial Reach-avoid Symbolic Planning Laya Shamgah 23

𝒃𝟐𝟐 𝒃𝟐𝟑 𝒃𝟐𝟐 𝒃𝟑𝟐

Discrete path Continuous path

Attacker 𝒃𝟐𝟐𝒃𝟐𝟑𝒃𝟑𝟑𝒃𝟑𝟐 Defender 𝒆𝟐𝟒𝒆𝟒𝟑𝒆𝟑𝟑𝒆𝟑𝟐

Example: Final Results (3)

24

Dynamic and Adversarial Reach-avoid Symbolic Planning Laya Shamgah 24

𝒃𝟐𝟐 𝒃𝟐𝟑 𝒃𝟐𝟑

Discrete path Continuous path

𝒃𝟐𝟐

Attacker 𝒃𝟐𝟐𝒃𝟐𝟑𝒃𝟐𝟐𝒃𝟐𝟑 Defender 𝒆𝟐𝟒𝒆𝟒𝟑𝒆𝟑𝟑𝒆𝟐𝟑

Conclusion

25

Conclusion:

• A novel, formal hybrid symbolic controller was developed for the vehicles involved in a

reach-avoid scenario.

Dynamic and Adversarial Reach-avoid Symbolic Planning Laya Shamgah 25

Significance of the results:

 To the best of our knowledge, this is the first work in the literature that employs symbolic motion planning for reach-avoid problem.  The proposed method is a computationally effective method that can reactively capture the changes in a dynamic and adversarial environment.  The developed approach considers less restrictions on the robot motion and requires no knowledge about the model of the opponent.

Future Work:

 The extension of the proposed framework to more complex scenarios and environments, for example reach-avoid problem with more number of players.

26

Thank You