Numerical Proofs in Nonlinear Control Sicun Gao, UCSD Nonlinear - - PowerPoint PPT Presentation

Numerical Proofs in Nonlinear Control

Sicun Gao, UCSD

Nonlinear control working

Nonlinear control not working

Dynamical systems are simple loops

x = x0 t = 0 while true do x = f(x, u(x)) · dt + x t = t + dt end while

x(t) = x(0) + ∫

t

f(x, u(x))ds

M (✓) ¨ ✓ + C(✓, ˙ ✓) ˙ ✓ + ⌧(✓) = Bu, ✓ = [✓1, ✓2, . . . , ✓n]T 2 Rn, u 2 Rn M(✓) = [aij cos (✓j ✓i)] , M (✓) 2 Rn×n C(✓, ˙ ✓) = h aij ˙ ✓j sin (✓j ✓i) i , C(✓, ˙ ✓) 2 Rn×n, ⌧(✓) = [bi sin ✓i] , G(✓) 2 Rn, B = [1, 1, . . . , 1]T ⇢ aii = Ii + mi`2

ci + `2 i

Pn

k=i+1 mk, 1  i  n

aij = aji = mj`i`cj + `i`j Pn

k=j+1 mk, 1  i < j  n

bi = mi`ci + `i

n

X

k=i+1

mk ! g, 1  i  n, pendulum system our approach can find the following neural Lyapuno

Dynamical systems are simple loops

Dynamical systems are simple loops

• Safety: do not reach bad states
• Stability (Liveness): eventually reach good states

Properties we care about

∀x0∀t∀xt(xt = Fu(x0, t) → safe(xt))

• Safety: do not reach bad states
• Stability (Liveness-ish): eventually reach good states

Properties we care about

∀x0∀t∀xt(xt = Fu(x0, t) → safe(xt))

∀ε∃δ ∀x0∀t∀xt(∥x0∥ < δ ∧ xt = Fu(x0, t) → (∥xt∥ < ε ∧ lim

t→∞ xt = 0))

• Safety (core part)
• Termination (core part)

Recall: invariants for programs

(Inv(x) ∧ T(x, x′)) → Inv(x′) T(x, x′) → (Rank(x) > Rank(x′))

For a discrete loop of the transition relation T(x, x′)

• Safety: barrier functions, differential invariants

Inductive proofs over Rn

B(x) = 0 → ∇f B(x) < 0

∇fV(x) = ∑

i

∂V ∂xi dx dt = ∑

i

∂V ∂xi fi(x)

• Lie Derivative

• Stability: Lyapunov functions

Inductive proofs over Rn

Find an “energy” landscape that forces stabilization (same as ranking function for termination)

• Stability (Lyapunov functions)

Inductive proofs over Rn

V(x) > 0,∀x ∈ D∖{0} V(0) = 0, · V(0) = 0 ∇fV(x) < 0,∀x ∈ D∖{0}

• Stability: Lyapunov functions

Inductive proofs over Rn

V ∇fV

• For discrete programs, finding invariants is always

hard, but checking them is easy

Difficulty due to nonlinearity

(Inv(x) ∧ T(x, x′)) → Inv(x′)

T(x, x′) → (Rank(x) > Rank(x′))

• Just encode the negations of these as SMT and

hope for an unsat answer

• In the continuous case, even checking the inductive

conditions is very hard

• First-order theory over nonlinear real arithmetic

∇fV(x) ≤ 0, ∀x ∈ D ⊆ ℝn

𝖴𝗂(⟨ℝ, ≤ , { + , × }⟩) is decidable but doubly-exponential

𝖴𝗂Σ1(⟨ℝ, ≤ , {sin, + , × }⟩) is undecidable

Difficulty due to nonlinearity

• FOL over reals is not that scary if we can allow

some numerical errors in the decisions

• Delta-decisions over reals [Gao-Avigad-Clarke, LICS’12]
• Can deal with any formula in where

is the set of all Type 2 computable functions

ℱ

Delta-decisions

⟨ℝ, ≤ , ℱ⟩

• Manipulate real numbers through natural encodings

as functions over the integers (e.g. Cauchy sequences)

• A real function is Type 2 computable if an algorithm

can approximate it up to arbitrary finite precisions (effective continuity)

• contains polynomials, sin, cos, exp, ODEs, etc.

Type 2 Computability

ℱ

(pretty much all the functions we need in engineering)

• Delta-weakening: put a formula in a positive normal form

and relax all to where

• Example:

is relaxed to .

• We say a formula is delta-satisfiable if its delta-weakening

is satisfiable. The delta-decision problem asks if a formula is unsat or delta-sat.

f(x) ≥ 0 f(x) ≥ − δ δ ∈ ℚ+ ∃x(x = 0) ∃x(|x| ≤ δ)

Delta-decisions

• Theorem:

formulas are delta-decidable over any compact domain.

• Theorem: The complexity of delta-deciding these formulas

is the same as their Boolean counterparts.

• Complexity results for free: e.g., global multi-objective

disjunctive nonlinear optimization is

• complete (

).

ℒℝ,ℱ ΣP

2

𝖮𝖰𝖮𝖰

Delta-decisions

• In practice, delta-decisions are all we need for

many problems in verification, optimization, etc.

• Reachability/Safety questions can be encoded,

with answers “safe” or “not robustly-safe” (a delta-perturbation makes the system unsafe)

• dReal, dReach, etc.

Delta-decisions

Difficulty with induction

• dReal always gives

spurious counterexamples

B(x) = 0 → ∇f B(x) < 0

• However, induction fails under numerical errors!

• However, induction fails under numerical errors!

Difficulty with induction

V(x) > 0,∀x ∈ D∖{0} V(0) = 0, · V(0) = 0 ∇fV(x) < 0,∀x ∈ D∖{0}

˙ p = c1 @2 ˆ u1 s p c11 − ✓ p c11 ◆2 −

• c3 + c4c2p + c5c2p2 + c6c2

2p

• 1

A ˙ r = 4 ✓ c3 + c4c2p + c5c2p2 + c6c2

2p

c13(c3 + c4c2pest + c5c2p2

est + c6c2 2pest)(1 + i + c14(r − c16)) − r

◆ ˙ pest = c1 2 ˆ u1 r

p c11 −

⇣

p c11

⌘2 − c13

• c3 + c4c2pest + c5c2p2

est + c6c2 2pest

• !

˙ i = c15(r − c16)

• But again, precise checking is unrealistic (high

nonlinearity, disturbances,…)

Difficulty with induction

(Example: powertrain control system)

Our fix to this problem

Epsilon-Lyapunov and Epsilon-Barrier functions

• We redefine the inductive proof rules over

continuous domains to robustify them

[Gao et al. CAV’19]

• Three robust proof rules (epsilon-inductive

conditions) for stability and safety

• For any epsilon, there exists a bound D, such

that for any delta<D, delta-decision procedures are sound and complete for checking the epsilon-invariance conditions

Our fix to this problem

Epsilon-Stability

• In practice, we can allow the system to oscillate

within an epsilon-ball around the origin

• Relax stability to allow small perturbation

(epsilon-stability)

• Strengthen Lyapunov conditions to allow small

numerical errors (epsilon-Lyapunov)

• Prove epsilon-Lyapunov implies epsilon-stability
• Prove epsilon-delta completeness

Relaxing Stability and Strengthening LF

Epsilon-Stability

• Relaxation: allow the system to oscillate

within an epsilon-ball around the origin

Stable(f) ⌘d

f 8(0,∞)τ9(0,∞)δ8Dx08[0,∞)t

⇣ kx0k < δ ! kF(x0, t)k < τ ⌘ Stableε(f) ⌘d

f 8[ε,∞)τ9(0,∞)δ8Dx08[0,∞)t

⇣ kx0k < δ ! kF(x0, t)k < τ ⌘

the only difference

Epsilon-Lyapunov functions

• Extend point-based requirements to neighborhoods

f = 0 V = 0 f > 0 rfV  0

V  β rfV  γ V α

ε0 ε

Lyapunov Epsilon-Lyapunov

Epsilon-Lyapunov functions

• Extend point-based requirements to neighborhoods

f = 0 V = 0 f > 0 rfV  0

V  β rfV  γ V α

ε0 ε

Lyapunov Epsilon-Lyapunov

Epsilon-Lyapunov functions

• Extend point-based requirements to neighborhoods

LF(f, V ) ⌘d

f (V (0) = 0) ^ (f(0) = 0) ^ 8D\{0}x

⇣ V (x) > 0 ^ rfV (x)  0 ⌘

LFε(f, V ) ⌘d

f 9(0,ε)ε09(0,1)α9(0,α)β9(0,1)γ

8D\Bεx ⇣ V (x) α ⌘ ^ 8Bε0 x ⇣ V (x)  β ⌘ ^ 8D\Bε0 x ⇣ rfV (x)  γ ⌘

Epsilon-Lyapunov functions

Theorem 1. If there exists an ε-Lyapunov function V for a dynamical system defined by f, then the system is ε-stable. Namely, LFε(f, V ) ! Stableε(f). Theorem 3 (Relative Completeness). For any ε 2 R+, if LFε(f, V ) is true then there exists δ 2 Q+ such that any δ-complete decision procedure must return that LFε(f, V ) is true. Theorem 2 (Soundness). If a δ-complete decision procedure confirms that LFε(f, V ) is true then V is indeed an ε-Lyapunov function, and f is ε-stable.

V  β rfV  γ V α

ε0 ε

Safety and epsilon-barrier functions

• Similarly, we define two robust barrier function

conditions that are stronger, sufficient for the normal notion of safety

• Prove epsilon-delta completeness

• Ensure that the system goes back into the

invariant set “near” the boundary

B = B = ε

init

rfB  γ

(c) Type 1 ε-Barrier

B = B = ε⇤ B = ε B = ε0

init

(d) Type 2 ε-Barrier

Safety and epsilon-barrier functions

BarrierT,ε(f, init, B) ≡d

f ∀Dx

⇣ init(x) → B(x) ≤ −ε ⌘ ∧ ∃(0,ε]ε⇤∀Dx∀[0,T ]t ⇣ (B(x) = −ε) → B(F(x, t)) ≤ −ε⇤⌘ ∧ ∃(ε,1)ε0∀Dx ⇣ (B(x) = −ε) → B(F(x, T)) ≤ −ε0⌘

Barrierε(f, init, B) ⌘d

f 8Dx

⇣ init(x) ! B(x)  " ⌘ ^ 9(0,∞)8Dx ⇣ B(x) = " ! rfB(x)  ⌘

Type 1: Type 2:

Safety and epsilon-barrier functions

Theorem 6. For any T, ε ∈ R+, BarrierT,ε(f, init, B) → Safe(f, init, B). Theorem 7. For any ε ∈ R+, there exists δ ∈ Q+ such that BarrierT,ε(f, init, B) is a δ-robust formula.   Theorem 4. For any " 2 R+, Barrierε(f, init, B) ! Safe(f, init, B).

• 2
• 1.5
• 1
• 0.5

0.5 1 1.5 2

x1

• 2.5
• 2
• 1.5
• 1
• 0.5

0.5 1 1.5 2 2.5

x2

Vector Field Limit Cycle z'Pz=90 z'Pz=110

• 2
• 1.5
• 1
• 0.5

0.5 1 1.5

x1

• 2
• 1.5
• 1
• 0.5

0.5 1 1.5

x2

Unsafe

init

Vector Field B(x) = 1.0 Levelset B(x) = -0.1 Levelset B(x) = 0.0 Levelset Forward Image of B(x)=0 Levelset at t=0.14 Forward Image of B(x)=0 Levelset at t=0.28 Forward Image of B(x)=0 Levelset at t=0.42

Safety and epsilon-barrier functions

Experiments (various nonlinear systems)

Example ↵

• "

"0 Time (s) T.R. Van der Pol 2.10×1023 1.70×1023 1025 1012 5×1013 0.05

• Norm. Pend.

7.07×1023 3.97×1023 1050 1012 5×1013 0.01 Moore-Greitzer 2.95×1019 2.55×1019 1020 1010 5×1011 0.04

Table 1: Results for the "-Lyapunov functions. Each Lyapunov function is of the form zT Pz, where z is a vector of monomials over the state variables. We report the constant values satisfying the "-Lyapunov conditions, and the time that verification of each example takes (in seconds).

Example ` "

• degree(z)

size of P Time (s) T.R. Van der Pol 90 105 105 3 9 × 9 6.47

• Norm. Pend.

[0.1, 10] 102 102 1 2 × 2 0.08 Moore-Greitzer [1.0, 10] 101 101 4 5 × 5 13.80 PTC 0.01 105 105 2 14 × 14 428.75

Experiments (powertrain control)

˙ p = c1 @2 ˆ u1 s p c11 − ✓ p c11 ◆2 −

• c3 + c4c2p + c5c2p2 + c6c2

2p

• 1

A ˙ r = 4 ✓ c3 + c4c2p + c5c2p2 + c6c2

2p

c13(c3 + c4c2pest + c5c2p2

est + c6c2 2pest)(1 + i + c14(r − c16)) − r

◆ ˙ pest = c1 2 ˆ u1 r

p c11 −

⇣

p c11

⌘2 − c13

• c3 + c4c2pest + c5c2p2

est + c6c2 2pest

• !

˙ i = c15(r − c16)

× PTC 0.01 105 105 2 14 × 14 428.75 Example ` "

• degree(z)

size of P Time (s)

• Once the proof rules can be checked, we can

further automate control design.

From verification to synthesis

∃p∃q∀x Φ(f, u(p, x), V(q, x))

• Find parameters for control and proof

certificate so that the inductive conditions in are true over all states.

u(p, x) V(q, x)

Φ

From verification to synthesis

∃p∃q∀x Φ(f, u(p, x), V(q, x))

• In general we can try solving these formulas in

the delta-decision framework. [Kong et al. CAV’18]

• But it is very hard to scale, because p and

especially q can be very high-dimensional.

From verification to synthesis

∃p∃q∀x Φ(f, u(p, x), V(q, x))

• We need cheap algorithms to search for p and q.
• We can often afford full SMT solving over x.
• Also, the form of u and V matter a lot.

From verification to synthesis

∃p∃q∀x Φ(f, u(p, x), V(q, x))

• The standard approach is to assume V is a

sum-of-squares polynomial and the search can be done through semidefinite programming.

• In practice, it is very brittle. (checking rarely passes)

Crazy attempt: use neural networks

∃p∃q∀x Φ(f, u(p, x), V(q, x))

• Instead of asking V to be a polynomial, let it

be a neural network.

• Use the verifier/falsifier to enforce the

inductive conditions and produce training sets.

[Chang et al. NeurIPS’19]

Crazy attempt: use neural networks

Require the neural network V to satisfy the inductive conditions on samples and counterexamples. Just use cheap gradient descent.

Crazy attempt: use neural networks

Crazy attempt: use neural networks

Quite amazingly it worked on many hard examples.

(humanoid balancing)

ℓH ℓA x y θA θH x

Crazy attempt: use neural networks

(humanoid balancing)

Crazy attempt: use neural networks

Quite amazingly it worked on many hard examples.

Humanoid balance Vehicle path following

L θa θ u S x y de

(wheeled vehicle path following)

Crazy attempt: use neural networks

Importantly, it improves previously known RoA.

Conclusion

• For core nonlinear control problems, we can

fully automate proofs and designs through reasoning engines and formal tools.

• Improve standard control methods both in

performance and reliability guarantees.

• Numerical and probabilistic methods are

powerful when their formal basis is established.

Thank you!