Using Integrative Modeling for a hammer, look out for your Advanced - - PowerPoint PPT Presentation

Arizona’s First University.

Using Integrative Modeling for Advanced Heterogeneous System Simulation

Tapasya Patki, Hussain Al-Helal, Jacob Gulotta, Jason Hansen, Jonathan Sprinkle

In collaboration with Vanderbilt University, and the University of California, Berkeley. Supported by US AFOSR. Corollary: If everyone has a hammer, then you will not use anyone else’s nails. Thm (Kelly): If all you have is a hammer, look out for your thumb, and all that.

2

11/11/08

Electrical and Computer Engineering

Patki, et al. "Integrative Modeling...Heterogeneous Simulation"

What kinds of heterogenous systems are we interested in?

3

11/11/08

Electrical and Computer Engineering

Patki, et al. "Integrative Modeling...Heterogeneous Simulation"

What motivates us to integrate?

Control Integration Framework Platform Control Integration

4

11/11/08

Electrical and Computer Engineering

Patki, et al. "Integrative Modeling...Heterogeneous Simulation"

What domain is this anyway?

• Command and Control

– Lots of decision makers, looking at tactical data, making command decisions. – Lots of monitors – Lots of data – Lots of decisions to make

• Types?

– Tactical actors (manned/unmanned components) – GUI elements (human interfaces) – Vignettes (tactical tests)

• Example vignette:

– A UAV is sent to a location to look for blue trucks. After a blue truck is spotted, the UAV reports its location. The C2 staff tell that UAV to “track” the blue truck. The UAV then stays as close as it can.

• Example problems:

– What kind of UAV is it? What kinematic/dynamic properties does it have? – What connection/network settings do I use? Am I communicating via TCP/IP, or something more primitive?

5

11/11/08

Electrical and Computer Engineering

Patki, et al. "Integrative Modeling...Heterogeneous Simulation"

Graphically?

* From an unpublished manuscript by Balogh, et al.

6

11/11/08

Electrical and Computer Engineering

Patki, et al. "Integrative Modeling...Heterogeneous Simulation"

What’s the hardest thing about integrative modeling?

CONTROLLER AND OBJECT SEARCH

(MATLAB SIMULINK)

VISUALIZATION

(DELTA -3D)

Position Update Found Target New Waypoint

GROUND STATION

(JAVA)

Search for target Track target Lost Target

VEHICLE POSITION

(JAVA)

Position Information Position Information

CONTROLLER AND OBJECT SEARCH

(MATLAB SIMULINK)

VISUALIZATION

(DELTA -3D)

Position Update Found Target New Waypoint

GROUND STATION

(JAVA)

Search for target Track target Lost Target

VEHICLE POSITION

(JAVA)

Position Information Position Information

The integrative modeling part…

7

11/11/08

Electrical and Computer Engineering

Patki, et al. "Integrative Modeling...Heterogeneous Simulation"

Defining the Message and Integration Types

R eturnT

• Base

time : double Base_ID : String SearchF

• rT

arget feature : String radius : double z : double y : double x : double U A V C

• ntrol

uav _id : String control : String T rackT arget suspected_z : double suspected_y : double suspected_t : double suspected_r : double suspected_x : double v z : double v y : double v x : double target_t : double target_r : double target_z : double target_y : double target_x : double target_id : String BestEffortR eceiv e send_interaction_handle : int V ery Low Prio M ediumPrio SimLog C

• mment :

String T ime : double F edN ame : String SimR esume SimEnd SimPause T ex tM essage message : String A utomaticBD A z : double y : double x : double D irectC

• ntrol

T O D O : String N ew W ay Point t : double y : double z : double x : double U A V C

• mmand

H ighPrio Low Prio SimpleM essage message : String N etw

• rkInteraction

BestEffortSend siz e : int sender_ip : String receiv er_ip : String PosU pdate time : double z : double y : double x : double y aw : double v y : double v z : double v x : double pitch : double roll : double SimulationC

• ntrol

InteractionR

• ot

8

11/11/08

Electrical and Computer Engineering

Patki, et al. "Integrative Modeling...Heterogeneous Simulation"

Defining the Message and Integration Types

id x y z vx vy vz nam e

Vehicle

del f ra f ra sen rec pla las f ra sen

SendStream ObjectR

• ot

sender_ip receiver_ip f ram e_rat e f ram e_siz e sending play ing delay f ram e_loss last _received_f

R eceiv eStream

9

11/11/08

Electrical and Computer Engineering

Patki, et al. "Integrative Modeling...Heterogeneous Simulation"

Defining Component Interconnectivity

UAVGSOp UAV

PosUpdate time : double z : double y : double x : double y aw : double v y : double v z : double v x : double pitch : double roll : double NewW ay Point t : double y : double z : double x : double

10

11/11/08

Electrical and Computer Engineering

Patki, et al. "Integrative Modeling...Heterogeneous Simulation"

Making our UAV look right

• A realistic model of the STARMAC was created using Blender
• Rendered model used to represent UAV during simulation
• Multiple instances of the rendered model can be used to simulate

swarms of rotorcraft

A “default” rotorcraft

11

11/11/08

Electrical and Computer Engineering

Patki, et al. "Integrative Modeling...Heterogeneous Simulation"

STARMAC

12

11/11/08

Electrical and Computer Engineering

Patki, et al. "Integrative Modeling...Heterogeneous Simulation"

How do those vehicles fly?

13

11/11/08

Electrical and Computer Engineering

Patki, et al. "Integrative Modeling...Heterogeneous Simulation"

A small wrinkle: image processing in simulation

Delta3D Reported Location

Gives GPS Location of Object If Object is In FoV Get a Pixel Location for the Object Introduce Error to Simulate Data Return to GPS Coordinates Object is Now in Some Area

Software Camera Sensor Algorithm

14

11/11/08

Electrical and Computer Engineering

Patki, et al. "Integrative Modeling...Heterogeneous Simulation"

Demos

• The GME Paradigm: what gets generated?
• Camera models: where are things executed?
• Vehicle dynamics: where does the logic live?

18

11/11/08

Electrical and Computer Engineering

Patki, et al. "Integrative Modeling...Heterogeneous Simulation"

A thought on domains (again…)

• In the “domain” of heterogeneous simulation

– Every player has their own domain – That domain has the correct tools, etc., for doing development

• There are lots of hard pieces

– Hard for domain-experts to understand middleware programming – Hard for middleware programmers to understand domain concepts – Hard for anyone to install everyone else’s tools…

• But, major benefits, if pieces can be integrated

easily

– Allows immediate work on domain-problems, deferring integration work until later – Permits domain-specific work to use the tools

• f the domain

– Showcases the power of code generation (when used by Jedi appropriately)

19

11/11/08

Electrical and Computer Engineering

Patki, et al. "Integrative Modeling...Heterogeneous Simulation"

Conclusions, and Future Work

• We were able to stand up a significant demo within 3 months of

(beginning) to install the software

• Our work concentrated on developing domain-specific pieces to

improve visualization and design-time analysis

• The modeling infrastructure supported our development in the

appropriate level of abstraction for future integration

• Future Work

– More advanced control algorithms (mesh travel/stability) – More advanced code generation (autogenerate vignette scripts, etc.)

20

11/11/08

Electrical and Computer Engineering

Patki, et al. "Integrative Modeling...Heterogeneous Simulation"

20

11/11/08

We’re always looking for good graduate students! http://www.ece.arizona.edu/~sprinkjm/research/c2wt/