A SIMULATION-BASED ARCHITECTURE FOR SMART CYBER- PHYSICAL SYSTEMS - - PowerPoint PPT Presentation

A SIMULATION-BASED ARCHITECTURE FOR SMART CYBER- PHYSICAL SYSTEMS

THOMAS GABOR, LENZ BELZNER, MARIE KIERMEIER, MICHAEL TILL BECK, ALEXANDER NEITZ LMU MUNICH INSTITUTE FOR INFORMATICS

SMART CYBER-PHYSICAL SYSTEMS

2

E.g. autonomous factories, smart citjes, ... Partjally uncontrolled environment Incomplete design-tjme knowledge Complex decisions at runtjme Space of behavioral solutjons available Task: Choose adequate solutjon in tjme

CHALLENGES

3

• Challenge 1: Changing environment
• Non-statjonary optjmal behavior
• Challenge 2: Aleatoric uncertainty
• Optjmal behavior is probabilistjc
• Irreducible
• Challenge 3: Epistemic uncertainty
• “Small data”
• Reducible
• Challenge 4: Behavioral uncertainty
• Due to resource constraints

A SIMULATION-BASED ARCHITECTURE FOR SMART CYBER- PHYSICAL SYSTEMS

MOTIVATION

5

Planning

• Given an MDP
• State space S, actjon space A
• Model of domain dynamics P(S|S x A)
• Reward functjon R : S x A x S → (encoding system goals)

ℝ

• Find strategy P(A|S) that maximizes expectatjon of gathered reward

Challenge

• Partjally uncontrolled domains
• Environment changes without system taking actjon
• Necessitates effjcient decision making

ONLINE PLANNING

6

Simulatjon-Based Online Planning

• Interleaving of planning and executjon
• Concentrates planning efgort on current situatjon
• Fast planning by sampling the search space

State of the Art (excerpt)

• Discrete Bandid-Based Planners: MCTS, UCT, SHOT
• Rolling Horizon Evolutjonary Algorithms
• Contjnuous Bandid-Based Planners: HOOT, HOLOP
• Cross Entropy Open Loop Planning

Kocsis, Levente, and Csaba Szepesvári. "Bandit based monte-carlo planning." Machine Learning: ECML 2006. Chaslot, Guillaume. "Monte-carlo tree search." Diss. Universiteit Maastricht. 2010. Cazenave, Tristan. "Sequentjal halving applied to trees." Computatjonal Intelligence and AI in Games. 2015. Perez Liebana, Diego, et al. "Open Loop Search for General Video Game Playing." ACM GECCO. 2015. Mansley, Christopher R., Ari Weinstein, and Michael L. Lituman. "Sample-Based Planning for Contjnuous Actjon Markov Decision Processes." ICAPS. 2011. Weinstein, Ari, and Michael L. Lituman. "Bandit-Based Planning and Learning in Contjnuous-Actjon Markov Decision Processes." ICAPS. 2012. Weinstein, Ari, and Michael L. Lituman. "Open-loop planning in large-scale stochastjc domains." AAAI. 2013.

Planning Executjon

THE DIGITAL TWIN ARCHITECTURE

7

THE DIGITAL TWIN ARCHITECTURE

8

CONTROL TIERS

9

• Four tjers of control
• Physical necessity
• Laws of nature, domain inherent
• In partjcular important for high-fjdelity simulatjons
• Machine-environment interface
• Circuit control, sensors, actuators, computatjonal hardware
• Is a system design variable (in contrast to physical necessity)
• Fixed in hardware, not easily changed at runtjme
• Immediate reactjon
• Watchdogs, fjxed behavioral rulesets, expert systems
• Sofuware, changeable at runtjme
• Planned reactjon
• Utjlity functjons, online planning, use of the digital twin
• Sofuware, changeable at runtjme

CHALLENGES, APPROACHES & CASE STUDIES

CHALLENGES & CASE STUDIES

11

Challenges for digital twin control

• Scalability to complex models → adaptjve abstractjon
• Safe operatjon → risk awareness

Case Studies

• Smart factory agent control
• Smart grid energy commitment

TIME ADAPTIVE MOTION PLANNING

12

• Consider a smart factory
• Materials and machines move probabilistjcally
• Model based motjon planning
• Problem:
• Environment changes contjnuously
• Optjmality of behavior changes contjnuously
• Simulatjon is expensive
• Goal: Collect targets
• Constraint: Avoid statjc and dynamic obstacles
• How to actuate in order to collect targets and avoid obstacles?
• We answer this questjon using a simulatjon @ runtjme
• Evaluate potentjal actuatjons with the digital twin
• Use of tjme adaptatjon for situatjve abstractjon

TIME ADAPTIVE MOTION PLANNING

13

• Fixed temporal resolutjon is not always optjmal
• Typical hyperparameter problem: Where to fjx?
• Assumptjon: A simulatjon is available as a functjon of tjme P(S|S x A x T)
• Idea: Incrementally identjfy 'important' moments
• Optjmize actjon duratjon
• Yields adaptatjon of simulatjon efgort to current situatjon
• A similar argument can be made for simulatjon depth

Belzner, Lenz. “Time-Adaptjve Cross Entropy Planning.” ACM SAC 2016.

RESULTS

With/without temporal abstractjon for empirical worst/best case parameters → Temporal abstractjon reduces tjme needed to collect the items

14

RESULTS

Temporal abstractjon reduces simulatjon efgort → Enables efgectjve use of simulatjon

15

RISK-AWARE COMMITMENT IN THE GRID

16

• Consider a smart grid
• Highly volatjle/variable energy consumptjon
• Contjnuous reconfjguratjon of energy productjon
• Problem:
• Overproductjon induces cost
• Power supply has to be guaranteed
• Only a limited quantjty can be sold to other markets
• Goal: Minimize productjon – consumptjon
• Constraint: 0 < productjon – consumptjon < maximum oversupply
• How much power to produce on next commitment?
• We answer this questjon using a simulatjon @ runtjme
• Evaluate potentjal productjon allotments with the digital twin
• We compared risk-neutral and risk-aware strategies
• Both use simulatjons, but difger in their evaluatjon

RISK-AWARE COMMITMENT IN THE GRID

17

ABSTRACTION, SAFETY & UNCERTAINTY ?

18

Safe adaptjve abstractjon

• Adaptjve abstractjon may provide scalability
• Potentjally “abstractjng away” risks?
• How to provide (statjstjcal) guarantees?

Learning/adaptjng models @runtjme

• Machine learning enables model adaptatjon @runtjme
• However, this introduces model uncertainty
• How to treat safety issues under model uncertainty?
• E.g. behavior optjmizatjon vs. model confjdence
• Possible approach → QoS-awareness
• Optjmize “up to” QoS requirement
• Maximize confjdence
• Consequences of adaptjve abstractjon and model uncertainty?

SUMMARY & NEXT

19

Summary

• Smart simulatjon-based CPS
• Digital twin architecture
• Control tjers
• Applicatjon examples
• Challenges for simulatjon-based control

Up next

• QoS-awareness, V&V @runtjme
• Learning models @runtjme
• Combining abstractjon & safety
• Distributed simulatjon-based CPS