SLIDE 1
Foundations*for*Stochastic* Systems* Sriram*Sankaranarayanan* - - PowerPoint PPT Presentation
Foundations*for*Stochastic* Systems* Sriram*Sankaranarayanan* - - PowerPoint PPT Presentation
Foundations*for*Stochastic* Systems* Sriram*Sankaranarayanan* University*of*Colorado,*Boulder* Joint*Work*with* Aleksandar*Chakarov* Stochastic*Systems* Discrete*Time,** Continuous*Time,** Finite*State* Finite*State* S 1 * 0.9* 0.9* S 1
SLIDE 2
SLIDE 3
Stochastic*Systems*
Discrete*Time,** Finite*State* S0* S2* S3* S1* 0.2* 0.8* 0.1* 0.9*
x0 = F(x, w)
Discrete*Time,** Infinite*State* Continuous*Time,** Finite*State* S0* S2* S3* S1* 0.2* 0.8* 0.9*
dx = f(x, t)dt + g(x, t)dw
Continuous*Time,** infinite*State*
SLIDE 4
Analyzing*Stochastic*Systems*
Stochastic*System*
Random* Inputs* Outputs*
Goal:*Bounds*on*probability*that*the*system* satisfies*properties.*
SLIDE 5
Insulin*Infusion*
InsulinQInfusion* Control*
Meals,*Physical* Activity**
Insulin* Blood*Glucose*
Noise*+*Delays* Delays*+**Set*Failures*
Insulin* Infusion* Pump* Glucose** Sensor* [Cameron*et*al.,*DallaQMan*et*al.,*Doyle*et*al.,*Hovorka*et*al.]* Failure*Probability*<*10Q6*
SLIDE 6
Analysis*of*Stochastic*Systems**
“MonteQCarlo”* Simulations* Statistical* Model** Checking* Model* Checking* Prob.*Abstract* Interpretation* Deductive* Techniques* [Rubenstein*+*Kroese,* Younes+Simmons,*Jha*et*al,** Clarke*+*Zuliani,*Legay*et*al,*…]*
Statistical*Guarantees* Mathematical*Guarantees*
PRISM:** Kwiatkowska*et*al.* Monniaux** Cousot*+*Monerau** McIver+Morgan* Chakarov+S*
SLIDE 7
Rest*of*the*Talk*
- An*Illustrative*(Toy)*Example*
– Dubins*Vehicle*on*a*Tarmac*
- Concentration*of*Measure*
- (Super)*Martingales*
- Synthesizing*Super*Martingales*
- Concluding*Thoughts*
SLIDE 8
Example:*Dubins*Vehicle*with* Steering*Errors*
(x, y, θ)
Dubin’s* Vehicle* Feedback*
SteeringAngle*
θ
Disturb*
y ≥ 1 ∧ θ > 0
w ∼ Uniform(−0.01, 0.01)
y0 = y + 0.1θ θ0 = 0.99θ + w
y ≤ −1 ∧ θ < 0
SLIDE 9
Intermediate*Distributions*
Monniaux,*Kwiatkowska*et* al,**Mardziel*et*al,* S*et*al.,**Abate*et*al.,* Xiu*and*Karandiakis,…*
n=95* n=500* n=1000* Histogram*
- f*y*values*
for*Dubins* Vehicle*
SLIDE 10
Deductive*Approach*
Prajna+Jadbabaie+Pappas’04* McIver+Morgan’06* Steinhardt+Tedrake’13* Chakarov+S’13,’14*
Deduce*facts*about*the*distributions.* Without*approximating*it.*
y + 10θ
Martingale*
w ∼ Uniform(−0.01, 0.01)
y0 = y + 0.1θ θ0 = 0.99θ + w
SLIDE 11
Martingale:*Background*
Stochastic*Process* Martingale* X0, X1, X2, X3, . . .
E(X4|x3, . . . , x0) = x3
x0 x1 x2 x3 X4
Time* X*
E(Xn+1 | Xn, . . . , X0) = Xn
SLIDE 12
Super*Martingales*
Martingale* E(Xn+1|Xn) = Xn E(Xn+1|Xn)≤Xn Super*Martingale*
SLIDE 13
Martingale:*Example*
E(yn+1 + 10θn+1|yn, θn) = yn + 0.1θn + 10(0.99θn + E(wn)) = yn + 10θn
y + 10θ
Martingale*
yn+1
θn+1
w ∼ Uniform(−0.01, 0.01)
y0 = y + 0.1θ θ0 = 0.99θ + w
SLIDE 14
Azuma’s*Inequality*
“Martingales*do*not*stray*too* far*from*their*starting*values”*
t*
Xn
X0
Azuma’67,*Hoeffding’63*
Pr(Xn − X0 ≥ t)
Pr(Xn − X0 ≥ t) ≤ exp ⇣ −
t2 2nC2
⌘
SLIDE 15
Application*of*Azuma*Inequality*
Safe*Set*
Initial* Dist.*
Probability*that*system*enters*failure* set*within*first*N*steps.*
- 1. Find*a*(super)*martingale*f(x)*
- 2. Bound*f(x)*for*failure*set*
- 3. Pr*(Enter*Failure*Set)*<=*Pr*(Martingale*exceeds*bound).*
SLIDE 16
Dubin’s*Car*
(x, y, θ)
y ≥ 1 ∧ θ > 0
y ≤ 1 ∧ θ < 0
M : y + 10θ failure ⇒ |M| > 1 Pr(failure) ≤ 0.013
Azuma*Inequality* Bound*
SLIDE 17
y x
Road%Width%
Azuma%Bound% MC%Estimate%(105%sims)%
[;1,1]% <=*0.013*
0*(no*failures*seen)*
[;1.5,1.5]% <=*2.7x10Q5*
0*(no*failures*seen)*
[;2,2]% <=*4.2x10Q9*
0*(no*failures*seen)*
[;2.5,2.5]% <=*5.4x10Q14*
0*(no*failures*seen)*
SLIDE 18
Discovering*Martingales*
- 1. Fix*a*desired*form*for*the*(super)*martingale.*
- 2. Encode*the*conditions*for*being*a*
martingale.*
- 1. Linear*Systems:*Farkas*Lemma*(dualization)*
- 2. Polynomial*Systems:*SumQOfQSquares*
Programming*
- 3. Bernstein*Polynomials*
- 3. Solve*to*obtain*(super)*martingales*
c1y + c2θ + c3y2 + c4yθ + c5θ2
SLIDE 19
Discovering*Super*Martingales*
2.985n + 150θ2 − −2.985x Martingale 10θ + y Martingale 2000θy − 199n + 100y2 + 1990x Martingale 49n − 500x SuperMartingale 1000θ − n SuperMartingale 10x − n SuperMartingale −n − 1000θ SuperMartingale
x := x + 0.1(1 − 1
2θ2)
y := y + 0.1θ θ := 0.99θ + 0.1w
SLIDE 20
Beyond*Martingales*
Super*Martingales*
E X1,n . . . Xm,n ≤ M X1,n−1 . . . Xm,n−1
Expectation* Invariants* Chakarov*+*S’*2014* Nonnegative* Matrix* Abstract*Interpretation** techniques*for*discovering* Expectation*Invariants*
E(Xn|Xn−1) ≤ Xn−1
SLIDE 21
Concluding*Thoughts*
- Martingales*+*Concentration*of*Measure:**
– Prove*bounds*on*extremely*rare*events.* – Depends*critically*on*finding*the*“right”*martingale.* – Promising*approaches*[*Previous*Talk*!*]*
- Continuous*Time*Systems:*
– Theory*extends*naturally.* – Different*kinds*of*concentration*of*measure* inequalities.*
– *[*Prajna+Jadbabaie+Pappas,Steinhardt+Tedrake,**Platzer]*
SLIDE 22