Event-Triggered Interactive Gradient Descent for Real-Time - - PowerPoint PPT Presentation

▶
event triggered interactive gradient descent for real
SMART_READER_LITE
LIVE PREVIEW

Event-Triggered Interactive Gradient Descent for Real-Time - - PowerPoint PPT Presentation

Sep 06, 2022 321 likes •733 views

Event-Triggered Interactive Gradient Descent for Real-Time Multiobjective Optimization Pio Ong and Jorge Cort es Mechanical and Aerospace Engineering University of California, San Diego http://carmenere.ucsd.edu/jorge 56th IEEE Conference on

slide-1
SLIDE 1

Event-Triggered Interactive Gradient Descent for Real-Time Multiobjective Optimization

Pio Ong and Jorge Cort´ es

Mechanical and Aerospace Engineering University of California, San Diego http://carmenere.ucsd.edu/jorge 56th IEEE Conference on Decision and Control: Event-Triggered and Self-Triggered Control I Melbourne, Australia December 12-15, 2017

slide-2
SLIDE 2

Flying to Australia

Options Transit (Hours) Cost (Dollars) 1 10 1100 2 5 1500 3 2 1400 How would a robot know which option is the best?

  • P. Ong and J. Cort´

es (UCSD) Event-Triggered Interactive Gradient Descent for Real-Time Multiobjective Optimization December 15, 2017 2 / 17

slide-3
SLIDE 3

Flying to Australia

Options Transit (Hours) Cost (Dollars) 1 10 1100 2 5 1500 3 2 1400 How would a robot know which option is the best? Some options are obviously worse.

  • P. Ong and J. Cort´

es (UCSD) Event-Triggered Interactive Gradient Descent for Real-Time Multiobjective Optimization December 15, 2017 2 / 17

slide-4
SLIDE 4

Flying to Australia

Options Transit (Hours) Cost (Dollars) 1 10 1100 3 2 1400 How would a robot know which option is the best? Some options are obviously worse. But, we are left with mathematically ambiguous options (Pareto Solutions)

  • P. Ong and J. Cort´

es (UCSD) Event-Triggered Interactive Gradient Descent for Real-Time Multiobjective Optimization December 15, 2017 2 / 17

slide-5
SLIDE 5

Flying to Australia

Options Transit (Hours) Cost (Dollars) Happiness 1 10 1100 3 2 1400 How would a robot know which option is the best? Some options are obviously worse. But, we are left with mathematically ambiguous options (Pareto Solutions) Ask Human!

  • P. Ong and J. Cort´

es (UCSD) Event-Triggered Interactive Gradient Descent for Real-Time Multiobjective Optimization December 15, 2017 2 / 17

slide-6
SLIDE 6

My Talk in One Slide

Motivation:

1

Rise of robots that will eventually coexist with human

2

Robot solve a optimization problem to do something

3

Robot becomes more complex, can do more than one thing

Scenario: Human interacts with robot to help solve multiobjective optimization problem Robot Accommodate Human:

1

Human cannot be asked too often

2

Human needs some time to answer

Approach: Use Event-Trigger Control to minimize human interaction.

  • P. Ong and J. Cort´

es (UCSD) Event-Triggered Interactive Gradient Descent for Real-Time Multiobjective Optimization December 15, 2017 3 / 17

slide-7
SLIDE 7

Outline

Describing Scenario Problem Statement and Assumptions Our approach: Interactive Gradient Descent Modeling Humans Human needs to rest.

Designing Event Trigger

Adding Human Response Time

Limiting design parameter

Wrapping up my Talk Simulations Conclusions

  • P. Ong and J. Cort´

es (UCSD) Event-Triggered Interactive Gradient Descent for Real-Time Multiobjective Optimization December 15, 2017 4 / 17

slide-8
SLIDE 8

Problem and Assumptions

Our problem: minimize

x∈Rn

f (x) with f (x) ∈ Rm, m objective functions In general, infinite number of Pareto solutions

  • P. Ong and J. Cort´

es (UCSD) Event-Triggered Interactive Gradient Descent for Real-Time Multiobjective Optimization December 15, 2017 5 / 17

slide-9
SLIDE 9

Problem and Assumptions

Our problem: minimize

x∈Rn

f (x) with f (x) ∈ Rm, m objective functions In general, infinite number of Pareto solutions The human has an implicit cost function, c : Rm → R, that ranks them

  • P. Ong and J. Cort´

es (UCSD) Event-Triggered Interactive Gradient Descent for Real-Time Multiobjective Optimization December 15, 2017 5 / 17

slide-10
SLIDE 10

Problem and Assumptions

Our problem: minimize

x∈Rn

f (x) with f (x) ∈ Rm, m objective functions In general, infinite number of Pareto solutions The human has an implicit cost function, c : Rm → R, that ranks them

1

Implicit because the human cannot express what it is

  • P. Ong and J. Cort´

es (UCSD) Event-Triggered Interactive Gradient Descent for Real-Time Multiobjective Optimization December 15, 2017 5 / 17

slide-11
SLIDE 11

Problem and Assumptions

Our problem: minimize

x∈Rn

f (x) with f (x) ∈ Rm, m objective functions In general, infinite number of Pareto solutions The human has an implicit cost function, c : Rm → R, that ranks them

1

Implicit because the human cannot express what it is

2

Human can respond to queries; we assume he can give the gradient

  • P. Ong and J. Cort´

es (UCSD) Event-Triggered Interactive Gradient Descent for Real-Time Multiobjective Optimization December 15, 2017 5 / 17

slide-12
SLIDE 12

Problem and Assumptions

Our problem: minimize

x∈Rn

f (x) with f (x) ∈ Rm, m objective functions In general, infinite number of Pareto solutions The human has an implicit cost function, c : Rm → R, that ranks them

1

Implicit because the human cannot express what it is

2

Human can respond to queries; we assume he can give the gradient

Assumptions: To assure there is a unique solution,

1

Each objective function is strictly convex.

2

The implicit function is strictly convex, increasing w.r.t. each objective value.

3

The implicit function is bounded from below and is radially unbounded.

  • P. Ong and J. Cort´

es (UCSD) Event-Triggered Interactive Gradient Descent for Real-Time Multiobjective Optimization December 15, 2017 5 / 17

slide-13
SLIDE 13

Restate the problem

What do we mean by solving a multiobjective optimization problem?

  • P. Ong and J. Cort´

es (UCSD) Event-Triggered Interactive Gradient Descent for Real-Time Multiobjective Optimization December 15, 2017 6 / 17

slide-14
SLIDE 14

Restate the problem

What do we mean by solving a multiobjective optimization problem? Answer: Find the Pareto solution that the human likes the best.

  • P. Ong and J. Cort´

es (UCSD) Event-Triggered Interactive Gradient Descent for Real-Time Multiobjective Optimization December 15, 2017 6 / 17

slide-15
SLIDE 15

Restate the problem

What do we mean by solving a multiobjective optimization problem? Answer: Find the Pareto solution that the human likes the best. Problem that we will solve: minimize

x∈Rn

(c ◦ f )(x)

  • P. Ong and J. Cort´

es (UCSD) Event-Triggered Interactive Gradient Descent for Real-Time Multiobjective Optimization December 15, 2017 6 / 17

slide-16
SLIDE 16

Restate the problem

What do we mean by solving a multiobjective optimization problem? Answer: Find the Pareto solution that the human likes the best. Problem that we will solve: minimize

x∈Rn

(c ◦ f )(x) Scenario: human and robot working together to get the best Pareto solution.

  • P. Ong and J. Cort´

es (UCSD) Event-Triggered Interactive Gradient Descent for Real-Time Multiobjective Optimization December 15, 2017 6 / 17

slide-17
SLIDE 17

Restate the problem

What do we mean by solving a multiobjective optimization problem? Answer: Find the Pareto solution that the human likes the best. Problem that we will solve: minimize

x∈Rn

(c ◦ f )(x) Scenario: human and robot working together to get the best Pareto solution.

Single objective optimization.

  • P. Ong and J. Cort´

es (UCSD) Event-Triggered Interactive Gradient Descent for Real-Time Multiobjective Optimization December 15, 2017 6 / 17

slide-18
SLIDE 18

Restate the problem

What do we mean by solving a multiobjective optimization problem? Answer: Find the Pareto solution that the human likes the best. Problem that we will solve: minimize

x∈Rn

(c ◦ f )(x) Scenario: human and robot working together to get the best Pareto solution.

Single objective optimization. No objective function avaliable. Only gradient value available upon requests.

  • P. Ong and J. Cort´

es (UCSD) Event-Triggered Interactive Gradient Descent for Real-Time Multiobjective Optimization December 15, 2017 6 / 17

slide-19
SLIDE 19

Interactive Gradient Descent

Let’s try gradient descent! ˙ x(t) = −(∇(c ◦ f )(x(t)))T Role: What’s the human and robot role in this optimization?

  • P. Ong and J. Cort´

es (UCSD) Event-Triggered Interactive Gradient Descent for Real-Time Multiobjective Optimization December 15, 2017 7 / 17

slide-20
SLIDE 20

Interactive Gradient Descent

Let’s try gradient descent! ˙ x(t) = −(∇(c ◦ f )(x(t)))T Role: What’s the human and robot role in this optimization? ˙ x(t) = −(∇c(f (x(t)))Jf (x(t)))T

  • P. Ong and J. Cort´

es (UCSD) Event-Triggered Interactive Gradient Descent for Real-Time Multiobjective Optimization December 15, 2017 7 / 17

slide-21
SLIDE 21

Interactive Gradient Descent

Let’s try gradient descent! ˙ x(t) = −(∇(c ◦ f )(x(t)))T Role: What’s the human and robot role in this optimization? ˙ x(t) = −(∇c(f (x(t)))

  • human

Jf (x(t))

  • robot

)T

  • P. Ong and J. Cort´

es (UCSD) Event-Triggered Interactive Gradient Descent for Real-Time Multiobjective Optimization December 15, 2017 7 / 17

slide-22
SLIDE 22

Interactive Gradient Descent

Let’s try gradient descent! ˙ x(t) = −(∇(c ◦ f )(x(t)))T Role: What’s the human and robot role in this optimization? ˙ x(t) = −(∇c(f (x(t)))

  • human

Jf (x(t))

  • robot

)T Humans cannot update the value continuously!

  • P. Ong and J. Cort´

es (UCSD) Event-Triggered Interactive Gradient Descent for Real-Time Multiobjective Optimization December 15, 2017 7 / 17

slide-23
SLIDE 23

Event-Triggered Interactive Gradient Descent

Preferably, only ask for human help only when it really needs to. ˙ x(t) = −(∇c(f (x(tk)))

  • human

Jf (x(t))

  • robot

)T with tk to be determined by the robot iteratively.

  • P. Ong and J. Cort´

es (UCSD) Event-Triggered Interactive Gradient Descent for Real-Time Multiobjective Optimization December 15, 2017 8 / 17

slide-24
SLIDE 24

Event-Triggered Interactive Gradient Descent

Preferably, only ask for human help only when it really needs to. ˙ x(t) = −(∇c(f (x(tk)))

  • human

Jf (x(t))

  • robot

)T with tk to be determined by the robot iteratively. When to ask human?

  • P. Ong and J. Cort´

es (UCSD) Event-Triggered Interactive Gradient Descent for Real-Time Multiobjective Optimization December 15, 2017 8 / 17

slide-25
SLIDE 25

Event-Triggered Interactive Gradient Descent

Preferably, only ask for human help only when it really needs to. ˙ x(t) = −(∇c(f (x(tk)))

  • human

Jf (x(t))

  • robot

)T with tk to be determined by the robot iteratively. When to ask human? Our proposition: robot monitors x(t) − x(tk) σ

˙ x(t) LcJf (x(t)) with σ ∈ (0, 1)

When these two things are equal, ask human.

  • P. Ong and J. Cort´

es (UCSD) Event-Triggered Interactive Gradient Descent for Real-Time Multiobjective Optimization December 15, 2017 8 / 17

slide-26
SLIDE 26

Design Guarantees

Properly stated tk+1 = min

t

  • t ≥ tk | x(t) − x(tk) = σ ∇c(f (x(tk)))Jf (x(t))

LcJf (x(t))

  • P. Ong and J. Cort´

es (UCSD) Event-Triggered Interactive Gradient Descent for Real-Time Multiobjective Optimization December 15, 2017 9 / 17

slide-27
SLIDE 27

Design Guarantees

Properly stated tk+1 = min

t

  • t ≥ tk | x(t) − x(tk) = σ ∇c(f (x(tk)))Jf (x(t))

LcJf (x(t))

  • Guarantees:

1

Global asymptotic stability: Lyapunov Function: V (x) = c ◦ f (x) − c ◦ f (x∗) = ⇒ d dt V (x(t)) ≤ − 1 − σ (1 + σ)2 ∇c(f (x(t)))Jf (x(t))2 < 0 = ⇒ Asymptotic Stability Moreover, if c ◦ f is strongly convex with a parameter µ, the optimizer is exponentially stable with the following bound, V (x(t)) ≤ V (x0)e

− 2µ(1−σ)

(1+σ)2 t

  • P. Ong and J. Cort´

es (UCSD) Event-Triggered Interactive Gradient Descent for Real-Time Multiobjective Optimization December 15, 2017 9 / 17

slide-28
SLIDE 28

Design Guarantee - Continued

tk+1 = min

t

  • t ≥ tk | x(t) − x(tk) = σ ∇c(f (x(tk)))Jf (x(t))

LcJf (x)

  • .

2

Autonomous operation: Robot has all the information to calculate above

  • P. Ong and J. Cort´

es (UCSD) Event-Triggered Interactive Gradient Descent for Real-Time Multiobjective Optimization December 15, 2017 10 / 17

slide-29
SLIDE 29

Design Guarantee - Continued

tk+1 = min

t

  • t ≥ tk | x(t) − x(tk) = σ ∇c(f (x(tk)))Jf (x(t))

LcJf (x)

  • .

2

Autonomous operation: Robot has all the information to calculate above

3

No Zeno behavior: There exists a uniform lower bound for the inter-event times τσ ≤ tk+1 − tk for all k ∈ N ∪ {0} where τσ is a constant given by τσ = 1 β ln(1 + β σ LcJmax )

  • P. Ong and J. Cort´

es (UCSD) Event-Triggered Interactive Gradient Descent for Real-Time Multiobjective Optimization December 15, 2017 10 / 17

slide-30
SLIDE 30

Design Guarantee - Continued

tk+1 = min

t

  • t ≥ tk | x(t) − x(tk) = σ ∇c(f (x(tk)))Jf (x(t))

LcJf (x)

  • .

2

Autonomous operation: Robot has all the information to calculate above

3

No Zeno behavior: There exists a uniform lower bound for the inter-event times τσ ≤ tk+1 − tk for all k ∈ N ∪ {0} where τσ is a constant given by τσ = 1 β ln(1 + β σ LcJmax ) tk Update gradient rest tk + τσ Ready to work standby tk+1 Update gradient

  • P. Ong and J. Cort´

es (UCSD) Event-Triggered Interactive Gradient Descent for Real-Time Multiobjective Optimization December 15, 2017 10 / 17

slide-31
SLIDE 31

Delay in Human

Previous model human responds instantaneously Better model Human requires some time to work, has some response time.

  • P. Ong and J. Cort´

es (UCSD) Event-Triggered Interactive Gradient Descent for Real-Time Multiobjective Optimization December 15, 2017 11 / 17

slide-32
SLIDE 32

Delay in Human

Previous model human responds instantaneously Better model Human requires some time to work, has some response time. The more accurate gradient descent is ˙ x(t) = ∇c(f (x(tk)))

  • human

Jf (x(t))

  • robot

, t ∈ [tk + Dk, tk+1 + Dk+1) tk Start working work tk + Dk Update gradient standby tk+1 Start working work tk+1 + Dk+1 Update gradient ˙ x[k−1] ˙ x[k]

  • P. Ong and J. Cort´

es (UCSD) Event-Triggered Interactive Gradient Descent for Real-Time Multiobjective Optimization December 15, 2017 11 / 17

slide-33
SLIDE 33

Trigger Design - Delay Case

Assuming there is a maximum delay D, we propose a similar trigger design: robot monitors x(t) − x(tk) σ′ ˙

x(t) LcJmax but σ′ not (0, 1)

tk Start working work tk + Dk Update gradient standby tk+1 Start working work tk+1 + Dk+1 Update gradient ˙ x[k−1] ˙ x[k]

  • P. Ong and J. Cort´

es (UCSD) Event-Triggered Interactive Gradient Descent for Real-Time Multiobjective Optimization December 15, 2017 12 / 17

slide-34
SLIDE 34

Trigger Design - Delay Case

Assuming there is a maximum delay D, we propose a similar trigger design: robot monitors x(t) − x(tk) σ′ ˙

x(t) LcJmax but σ′ not (0, 1)

σ′ must satisfy σ′ ≤ LcJmax β (eβ(τσ−D) − 1) tk Start working work tk + Dk Update gradient standby tk+1 Start working

≤

tk+1 + Dk+1 Update gradient ˙ x[k−1] ˙ x[k]

  • P. Ong and J. Cort´

es (UCSD) Event-Triggered Interactive Gradient Descent for Real-Time Multiobjective Optimization December 15, 2017 12 / 17

slide-35
SLIDE 35

Trigger Design - Delay Case

Assuming there is a maximum delay D, we propose a similar trigger design: robot monitors x(t) − x(tk) σ′ ˙

x(t) LcJmax but σ′ not (0, 1)

σ′ must satisfy. LcJmax β 1 + σ 1 − σ 2 (eβD − 1) < σ′ ≤ LcJmax β (eβ(τσ−D) − 1) tk Start working work tk + Dk Update gradient

≤

tk+1 Start working work tk+1 + Dk+1 Update gradient ˙ x[k−1] ˙ x[k]

  • P. Ong and J. Cort´

es (UCSD) Event-Triggered Interactive Gradient Descent for Real-Time Multiobjective Optimization December 15, 2017 12 / 17

slide-36
SLIDE 36

Design Guarantees - Delay Case

tk+1 = min

t

  • t ≥ tk | x(t) − x(tk) = σ′ ∇c(f (x(tk)))Jf (x(t))

LcJmax

  • .

Guarantees:

1

Global asymptotic stability: Same

2

Autonomous Operation: Same

3

No Zeno Behavior: The uniform lower bound to the interevent times for the delay case is τσ′ = 1 β ln    1 + β

σ′ LcJmax

1 +

  • 1+σ

1−σ

2 (eβD − 1)    + D

  • P. Ong and J. Cort´

es (UCSD) Event-Triggered Interactive Gradient Descent for Real-Time Multiobjective Optimization December 15, 2017 13 / 17

slide-37
SLIDE 37

Simulations

A robot trying to get close to two objects

0.2 0.4 0.6 0.8 1 x1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 x2 Trajectory 0.5 1 1.5 2 2.5 3 time (units) 0.2 0.4 0.6 0.8 Cost value Implicit cost function value 0.5 1 1.5 2 2.5 3 time (units) 10-20 10-10 100 c(f(x))-p* Implicit cost function convergence in logarithmic scale 5 10 15 20 25 30 35 40 Human update iteration 0.045 0.05 0.055 0.06 0.065 0.07 0.075 0.08 Interexecution time (units) Interexecution time and its lower bound 0.3 0.4 0.5 0.6 0.7 0.8 4 5 6 7 8 9 10 11 Number of triggers Number of triggers to 1% convergence for each

  • P. Ong and J. Cort´

es (UCSD) Event-Triggered Interactive Gradient Descent for Real-Time Multiobjective Optimization December 15, 2017 14 / 17

slide-38
SLIDE 38

Conclusions

Event-triggered design for human-robot interaction Human works as supervisor, robot works as extension of human capability Bound on inter-event time guarantees human has time to do other things. Provably correct: achieves multiobjective optimization task Future Work Richer models for human engagement

1

Rest time and Response time

2

Human inputs with errors

Scenarios where human needs to rest for longer than interevent time Online Learning of human model using human responses

  • P. Ong and J. Cort´

es (UCSD) Event-Triggered Interactive Gradient Descent for Real-Time Multiobjective Optimization December 15, 2017 15 / 17

slide-39
SLIDE 39

Thank You!

My advisor: Jorge Cort´ es Our Funding Source: NSF award CNS-1329619

  • P. Ong and J. Cort´

es (UCSD) Event-Triggered Interactive Gradient Descent for Real-Time Multiobjective Optimization December 15, 2017 16 / 17

slide-40
SLIDE 40

Question?

Questions and feedbacks are welcome!

  • P. Ong and J. Cort´

es (UCSD) Event-Triggered Interactive Gradient Descent for Real-Time Multiobjective Optimization December 15, 2017 17 / 17