Occupation measures and LMI formulation of piecewise affine optimal - PowerPoint PPT Presentation

Occupation measures and LMI formulation of piecewise affine optimal control design problems Didier Henrion 1 , 2 Luis Rodrigues 3 1 LAAS-CNRS Univ Toulouse, FR 2 Czech Tech Univ Prague, CZ 3 Concordia Univ Montr´ eal, CA Louvain-la-Neuve, 27 April 2010

Outline 1. Occupation measures and dynamical systems 2. LMI formulation of moment problem 3. Illustration: LQ optimal control 4. Application: PWA optimal control

Measures Measure = function assigning a volume or probability to a set � � � µ : X ⊂ R n �→ R + µ ( X ) = X dµ = X dµ ( x ) = X µ ( dx ) Examples: • Lebesgue measure dµ ( x ) = dx , µ ( X ) = vol( X ) • Hermite measure dµ ( x ) = e − x T x dx (density, abs. continuous) • probability measure µ ( R n ) = 1 • Dirac measure dµ ( x ) = δ x ∗ , µ ( { x ∗ } ) = 1 (atomic, discrete)

Measures Support = smallest closed set X ⊂ R n for which µ ( R n /X ) = 0 Examples: • Dirac measure supp( δ x ) = { x } • atomic measure supp( µ ) = { x 1 , . . . , x r } • Hermite measure supp( µ ) = R n • Lesbegue measure on X = [ − 1 , 1], vol( X ) = 2 • Lesbegue measure on X = { x ∈ R 2 : x T x ≤ 1 } , vol( X ) = π Indicator, or characteristic function of a set X I X ( x ) = 1 if x ∈ X = 0 otherwise

Moments i =1 x α i Multi-index notation x α = � n with x ∈ R n , α ∈ N n i The α -th moment of measure µ is the real number � X x α dµ ( x ) y α = µ is a representing measure for sequence y = ( y α ) α ∈ N n Classical problem of moments (Hausdorff, Markov, Stieltjes): characterise sequence y having representing measure µ supported on a (given) set X Conditions on y α ? Construction of µ and X , given y ?

Occupation measures Dynamical system described by ODE x = f ( x ) ˙ with solution, or flow φ t ( x 0 ) starting from initial condition x 0 Occupation measure of trajectory from φ 0 ( x 0 ) = x 0 to φ T ( x 0 ) = x T � T µ ( X ) = 0 I X ( φ t ) dt where X is a subset of R n It is the time spent in X by the solution of the ODE

Moments of occupation measure By definition, in any subset X ⊂ R n , the α -th moment of occupation measure µ is given by X x α dµ ( x ) � y α = � T 0 x α δ x ( φ t ) dtdx = � X � T X x α δ x ( φ t ) dxdt = � 0 � T 0 φ α = t dt So if sequence y is given we can find system trajectories by solving the corresponding problem of moments Given dynamics f ( x ), how can we find sequence y ?

Measure problem Consider a continuously differentiable test function v ( x ) whose time-derivative along system trajectories is given by v = ∇ v · ˙ ˙ x = ∇ v · f From the fundamental theorem of calculus � T � T 0 dv = 0 ∇ v · fdt = � X ∇ v · fdµ = v ( x T ) − v ( x 0 ) it follows that occupation measure µ satisfies (infinitely many) linear equations � X ∇ v · fdµ = v ( x T ) − v ( x 0 ) ∀ v

Measure problem Now assume x 0 and x T are not known exactly, they are modeled by probability measures µ 0 and µ T with supports X 0 and X T , respectively Our three measures satisfy the following constraints � � � X ∇ v · fdµ = vdµ T − vdµ 0 , ∀ v X T X 0 (compare with previous slide where µ 0 = δ x 0 , µ T = δ x T ) This is an infinite-dimensional linear problem in measure space Compare with weak or variational formulations of PDE problems Interesting mathematically, but not really constructive..

Polynomial optimal control Consider now the polynomial optimal control problem � T min u,T 0 h ( x, u ) dt + h T ( x ( T )) s . t . x = f ( x ) + g ( x ) u ˙ x (0) ∈ X 0 , x ( T ) ∈ X T x ∈ X, u ∈ U with f, g, h, h T polynomials and X 0 , X T , X, U compact basic semialgebraic sets (intersections of polynomial sublevel sets) Using the same ideas, this can be written as a linear but ∞ -dimensional problem on measures min µ,µ 0 ,µ T � X hdµ + � X T h T dµ T s . t . � X ∇ v · ( f ( x ) + g ( x ) u ) dµ = � X 0 vdµ 0 − � X T vdµ T , ∀ v

Generalized problem of moments To summarize, we have a linear problem involving several measures µ i respectively supported on sets X i All the data are polynomials, so we can replace measures by their α h iα x α dµ i = � X i x α dµ i ) moments (e.g. � X i h i ( x ) dµ i = � α h iα � � X i min µ � min y � X i h i dµ i � � α h iα y iα i i s . t . α a ijα y iα = b j � � s . t . � X i a ij dµ i = b j � i i measures µ i moments y i provided we can handle the representation condition � x α dµ i ( x ) y iα = X i

Outline 1. Occupation measures and dynamical systems 2. LMI formulation of moment problem 3. Illustration: LQ optimal control 4. Application: PWA optimal control

LMI conditions Given a sequence y , define the moment matrix M d ( y ) of order d with entries indexed by multi-indices β (rows) and γ (columns) [ M d ( y )] β,γ = y β + γ , | β | + | γ | ≤ 2 d which are linear in y Necessary condition: if y has a representing measure µ then M d ( y ) � 0 ∀ d Sufficient condition (Berg 1987): if | y α | ≤ 1 ∀ α and M d ( y ) � 0 ∀ d , then y has a representing measure µ with supp( µ ) ⊂ [ − 1 , 1]

LMI conditions α g α x α , define the Given a sequence y and a polynomial g ( x ) = � localising matrix M d ( gy ) of order d with entries � [ M d ( gy )] β,γ = | α | + | β | + | γ | ≤ 2 d g α y α + β + γ , α Let X = { x ∈ R n : g i ( x ) ≥ 0 , ∀ i } be compact basic semialgebraic with { x : g i ( x ) ≥ 0 } compact for some i Necessary condition: if y has a representing measure µ with support in X , then M d ( y ) � 0, M d ( g i y ) � 0 ∀ i ∀ d Sufficient condition (Putinar 1993): if M d ( y ) � 0, M d ( g i y ) � 0 ∀ i ∀ d then y has a representing measure with supp( µ ) ⊂ X

Moment LP as LMI c T y c T y min y min y s . t . Ay = b s . t . Ay = b X x α dµ y α = � M d ( y ) � 0 X = { x : g i ( x ) ≥ 0 , ∀ i } M d ( g i y ) � 0 , ∀ i infinite-dimensional finite-dim. LMI LP problem relaxation of order d GloptiPoly 3 (DH, JB. Lasserre, J. L¨ ofberg) for Matlab models generalised problems of moments as LMI problems POCP (C. Savorgnan) models polynomial optimal control problems as generalised problems of moments homepages.laas.fr/henrion/software

Dual LMI Dual LMI problem yields polynomial subsolution h ( x, u ) + ∇ v ( x ) · ( f ( x ) + g ( x ) u ) ≥ 0 , v ( x ( T )) = h T ( x ( T )) of Hamilton-Jacobi-Bellman PDE with positivity condition relaxed to polynomial sum-of-squares LMI condition Good approximation of value function along optimal trajectories If h ( x, u ) = h x ( x ) + u T u , use optimality condition ∂ ( h ( x, u ) + ∇ v ( x ) · ( f ( x ) + g ( x ) u )) = 2 u + ∇ v · g = 0 ∂u to derive state-feedback control law u ∗ ( x ) = − 1 2 ∇ v ∗ ( x ) · g ( x )

Outline 1. Occupation measures and dynamical systems 2. LMI formulation of moment problem 3. Illustration: LQ optimal control 4. Application: PWA optimal control

Linear ODE analysis Consider the scalar linear ODE x = − x ˙ 2 ) 2 ≥ 0 } with initial measure µ 0 in X 0 = { x : g 0 ( x ) = 1 − ( x − 3 4 − x 2 ≥ 0 } with final measure µ T in X T = { x : g T ( x ) = 1 with occupation measure µ in X = { x : g ( x ) = 4 − x 2 ≥ 0 } � T 0 x 2 dt We want to find trajectories minimising the energy Linear measure problem � T 0 x 2 dµ ( x ) min dv ( x ) s . t . � dx ( − x ) dµ ( x ) = � X T vdµ T − � X 0 vdµ 0 , ∀ v X

Linear ODE analysis Setting v = x α we introduce sequences y 0 , y T , y representing measures µ 0 , µ T , µ , and we obtain the linear moment problem min y 2 s . t . − αy α = y T α − y 0 α , ∀ α and the corresponding LMI relaxation of order d min y 2 s . t . − αy α = y T α − y 0 α , ∀ α, | α | ≤ 2 d M d ( y 0 ) � 0 , M d ( y T ) � 0 , M d ( y ) � 0 M d ( g 0 y 0 ) � 0 , M d ( g T y T ) � 0 , M d ( gy ) � 0 Solving LMI relaxations of increasing orders d yields a sequence of monotonically increasing lower bounds on the optimum

Linear ODE analysis This problem can be solved analytically, with optimal trajectory x ( t ) = e − t leaving X 0 at x (0) = 1 and reaching X T at x ( T ) = 1 2 for T = log 2 ≈ 0 . 6931 � log 2 e − αt dt = 1 − 2 − α Moment matrix M ( y ) has entries y α = 0 α We get with SeDuMi 1.1R3 the following sequence of valid significant digits on T : 0, 2, 4, 7, 10, 13 (fast convergence) Convergence at a finite relaxation order is impossible since the optimum is transcendental, whereas the solution of an integer coefficient LMI is algebraic

LQR design Consider the linear quadratic regulator design problem � T 0 ( x 2 + u 2 ) dt min u,T s . t . x = u ˙ x (0) = x 0 , x ( T ) = x T with given initial and terminal conditions Measure µ 0 is the Dirac δ x 0 and measure µ T is the Dirac δ x T so that only measure µ must be found Linear measure problem � T 0 ( x 2 + u 2 ) dµ ( x, u ) min µ R 2 dv ( x ) s . t . � dx udµ ( x, u ) = v ( x T ) − v ( x 0 ) , ∀ v

LQR design Moment LMI problem min y 20 + y 02 s . t . y 01 = 2 y 11 = 3 y 21 = · · · = − 1 M d ( y ) � 0 The moment matrix has the following quasi-Hankel structure   y 00 y 10 y 01 y 10 y 20 y 11   M d ( y ) =   y 01 y 11 y 02     ... with � x α 1 u α 2 dµ ( x, u ) y α =