Experiences in Developing An Intelligent Ground Vehicle (IGV) - - PowerPoint PPT Presentation

SLIDE 1

Experiences in Developing An Intelligent Ground Vehicle (IGV) Ontology In Protégé

Craig Schlenoff, NIST Randy Washington, DCS Corporation Tony Barbera, NIST July 8, 2004

SLIDE 2

Agenda

• Background:

– What is an Intelligent Ground Vehicle (IGV)? – NIST 4D/RCS Methodology and Architecture

• Ontology Development:

– 4D/RCS to Ontology Mapping – Interchange Formats and Upper Ontologies – IGV Military Equipment – IGV Behaviors – IGV Conditions

• Current Status
• Issues and Lessons Learned

SLIDE 3

What is an Intelligent Ground Vehicle?

SLIDE 4

4D/RCS Mapping to IGV Ontology

Commander & Vehicle Agent Instances Service Instances Environment Instances Process Model Instances Condition Instances

DOT Driving Manuals and ARMY Field Manuals

+

Domain Experts Task Decomposition Tree (Route Reconnaissance Example) Hierarchical Organization of Agent Control Modules

BEHAVIOR GENERATION WORLD MODEL KNOWLEDGE SENSORY PROCESSING

Situations World States Objects & Attributes

ColorCameras LADAR Radar Stereo FLIR Nav Segmented Groupings Features and Attributes Objects and Maps Object Groupings and Classifications

Cattails

ConductRoadMarchToAnAssemblyArea(AA) Conduct RouteRecon PrepareFor RoadMarch FollowPlatoon ToAssemblyArea Secure AssemblyArea Organize AssemblyArea Conduct MainRoute Recon DeployTo StartPoint Conduct LeftFlank RouteRecon Locate&Secure ObstacleBypass Conduct Obstacle Recon Conduct RightFlank RouteRecon MoveInto MarchFormation PrepareDetailed MovementPlans MoveTo ControlPoint Conduct DominateTerrain Recon MoveTo Cover/Concealed Position Ford Water Obstacle Secure Area Overwatch Section Perform Ford Recon Locate WaterBypass MoveTo ControlPoint Assess FordTerrain MoveTo Position Scan Path Cross Ford ScanArea ForEnemy MoveTo Water ShiftTo 4WhLo MoveTo Opposite Bank ShiftTo 4WhHi Dry Brakes

SENSORY PROCESSING KNOWLEDGE DATABASE BEHAVIOR GENERATION WORLD MODEL VALUE JUDGMENT (Executing) (ReconToRoute) (Executing)

SENSORY INPUT

(Platoon “B” Section Agent Control Module)

Select “ConductWaterObsRecon” Plan State-Table

(ConductRightFlankRecon)

S cou t P lato

• n

1st L ight C avalry Troop C

• m

m and G roup (FSO ) Q uartering P arty Scout Platoon 3rd C

• nduc

tR

• adM

a rchTo A s sem blyA rea C

• n

ductR

• uteR

ec

• n

S ec ureA ss em blyA rea C

• nduc

tA rea R e con O rganizeA sse m b ly A re a Follow P latoo nToA s se m bly A re a P re pare ForR

• adM

a rch M

• veIntoForm

ation E x ecu teR

• ad

M arch E x ecu teH alt O cc upyA ss em bly A rea P re pare ForR

• adM

a rch M

• veIntoForm

ation E x ecu teR

• ad

M arch E x ecu teH alt O cc upyA ss em bly A rea C S ection H M M W V #4 PS G H M M W V #10 H M M W V #9 C

• m

m s Prim itive/ T rajectory S urveillanc (R ST A ) G aze Co ntrol Lethality (G unner) M

• bility

W eapon C

• ntrol

T argeting Sensors P an /T ilt C

• n

trol FLIR , L aser R adar, C am era C ross Ford M

• v

eToP

• s

ition S c anP ath M

• veToW

ater S hiftTo4W hee lL

• M
• veToO

ppositeB ank S hiftTo4W hee lH i D ry B ra kes A s se ssFo rdTerrain S c anA rea ForE nem y A S ection Squ ad H M M W V #7 S quad H M M W V #3 S quad H M M W V #2 D eploy ToS tartP

• int

C

• nduc

tLe ftFla nkR ec

• n

Lo cate& S ecureO bs ta cleB ypas s M

• v

eToC

• ntrolPoint

Loc ateW ate rB ypas s P erform Ford R e con FordW a terO bstac le S ec ureA re a O verw atchS ec tion B S ectio n H M M W V #8 H M M W V #6 H M M W V #5 D eployToS tartPo int C

• nductR

ightFla nkR ec

• n

C

• nductD
• m

inan tTerrainR e con C

• nductA

m bus hS iteR ec

• n

M

• v

eToC

• ntrolP
• int

S e ndR e port NormalRouteReconSituation ConductRightFlankRecon ConductWaterObstacleRecon WaterObstacleDetected AssessMinefieldRecon MinefieldDetected DefileDetected ConductDefileRecon LateralRouteDetected ConductLateralRouteRecon

Selection Conditions Selected Plan

PLAN SELECTION TABLE PLAN STATE-TABLE

Input Conditions Output Commands

S1 sq5_NoBypassOnRouteSide S2 SetupReport&NewControlMeasures sq5_ReconToFarFlank sq6_ConductTravelingOverwatch sq8_VisuallyClearObstacle-FarSide NewPlan S1 SetupControlMeasures sq5_ReconToRouteFlank sq6_ConductTravelingOverwatch sq8_VisuallyClearObstacle-FarSide

ConductWaterObstacleRecon

S2 sq5_NoBypassTowardBoundary S0 SetupReport&NewControlMeasures sq5_MoveTacticallyToControlPoint sq6_ConductTravelingOverwatch sq8_MoveTacticallyToControlPoint S1 sq5_PossibleFordDetected S4 SetupReport&NewControlMeasures sq5_ConductFordRecon sq6_ConductNearSideOverwatch sq8_VisuallyClearObstacle-FarSide S1 sq5_LateralBypassFound S7 SetupReport&ControlMeasures sq5_ConductObstacleFarSideRecon sq6_ConductTravelingOverwatch sq8_VisuallyClearToRoute S4 sq5_FordNotPassable S1 SetupControlMeasures sq5_ReconToRouteFlank sq6_ConductTravelingOverwatch sq8_VisuallyClearObstacle-FarSide S4 sq5_FordLooksPassable S5 SetupReport&NewControlMeasures sq5_MoveToCover/ConcealPosition sq6_ConductNearSideOverwatch sq8_VisuallyClearObstacle-FarSide

ConductRightFlankRecon

BEHAVIOR GENERATION

STATUS STATUS NEXT SUBGOAL

.

COMMANDED TASK (GOAL) DriveOnTwoLaneRd PassVehInFront PassVehInFront DriveOnTwoLaneRd NegotiateLaneConstrict PassVehInFront . STATE-TABLES

WaterDepthToSixFeet WaterCoveredLand MajorGroundDeformation TractionSlip

LegalToPass

MarshDetected

SomeWaterVisible

Mosses,Evergreens,andShrubs StagnantWater IndeterminantGroundLevel SignificantTractionSlip OrganicMaterialOnWaterSurface

BogDetected

ErodedEarthEmbankments FlowingWater NonVegetatedWaterInMiddle NarrowWidth,IndeterminantLength

StreamDetected

ErodedEarthEmbankments FlowingWater NonVegetatedWaterInMiddle SignificantWidth,IndeterminantLength

RiverDetected

WaterSheenOnGroundSurface LittleToNoVegetation SignificantTractionSlip RuttedWithStandingWater

MudDetected

SignificantGroundDeformation

ExtensiveGrassyVegetation LongLeafGrasses - very flat, long green leaves-purple/rose/yellow flowers. Reeds - tall, woody, thin, round, hollow jointed (tan-to-green) stem plants, long narrow green blade leaves, large feathery panicles (elongated clusters of tan/white/purple flowers along main stem). Sedges - triangular tan/green stem plants, papyrus, narrow green to tan grass- like leaves, spikelets of inconspicuous tan-to-yellow-to-white flowers. Bulrushes - tall tan-to-green stems, with dark brown cylindrical seed heads that explode into white down, long flat green sword shaped leaves, cattails. Water Lilies - very large floating green leaves with white flowers. Saturated Ground Six Feet

• f Water

Plant Height 6-18" 6-48" 1-6' 3-9' Float

LargeSurfaceAreaOfStillWater LargeAreaWithoutGrasses,Trees,Shrubs OrganicMaterialMayBeOnWaterSurface

Pond/LakeDetected

BoundedBySwamp,Marsh MostlyTrees,SomeBushes SlowMovingWaterCoveredLand SignificantTractionSlip ExtensiveWaterSurfaceVisible

SwampDetected

MajorGroundDeformation

WaterObstacleDetected

15 cm 2.4 cm .9 to 2.7m 3 cm

DeployToStartPoint ConductMainRouteRecon Evaluate&ClassifyObstruction MoveToControlPoint

SLIDE 5

Interchange Formats and Upper Ontologies

• OWL

– Neutral (W3C) interchange format – XML base enables use XSLT transforms – Provides access to emerging semantic web technologies

• OWL-S

– Rich semantics for describing complex processes (without being too complicated) – Well suited to agent architectures

• Pieces of SUMO (Suggested Upper Merged Ontology)

– Class structure and properties provide a good starting point for developing domain specific ontology – Native KIF format too complex for target community and not necessary for requirements capture

• Namespaces

– Used quite a bit to make ontology more manageable

SLIDE 6

IGV Conceptual Model

Service Troop Commander Agent Service Platoon Leader Agent Service Section Lead Agent Service Vehicle Commander Agent Service Mobility System Agent Service Propulsion Subsystem Agent Service Platoon Leader Agent Service Platoon Leader Agent Service Section Lead Agent Service Section Lead Agent Service Vehicle Commander Agent Service Vehicle Commander Agent Service Survivability System Agent Service Surveillance System Agent Service Localization Subsystem Agent Service Auxiliary Subsystem Agent Engine Component Transmission Component External Service Request by a process Service Lethality System Agent Service Automotive Subsystem Agent Service Support System Agent Service Navigation Subsystem Agent Track-Drive Component Brake Component

SLIDE 7

Representing an IGV (cont.)

SLIDE 8

Tactical Behaviors Plan State-Table Selection

Scout Platoon 3rd B Section HMMWV #5 HMMWV #8 HMMWV #6 A Section Squad/Veh HMMWV #7 Squad/Veh HMMWV #3 Squad/Veh HMMWV #2 C Section HMMWV #10 HMMWV #9 HMMWV #4 PSG ConductRightFlankRecon ConductRouteReconToBridge ConductLeftFlankRecon Recon to RightFlank Conduct Traveling Overwatch Visually ClearObs Far-Side Visually Clear Route Conduct Traveling Overwatch Conduct Recon ToCP Conduct Traveling Overwatch Conduct Route Recon To CP Support Route Recon NoBypass OnLeft

ConductRouteRecon

Bridge Detected ConductWaterBridgeRecon ConductWaterObsRecon WaterObs Detected Bridge Detected

SENSORY INPUT

GENERIC 4D/RCS AGENT CONTROL MODULE

SENSORY PROCESSING KNOWLEDGE DATABASE BEHAVIOR GENERATION WORLD MODEL VALUE JUDGMENT STATUS STATUS COMMANDED TASK (GOAL) COMMANDED SUBGOALS BEHAVIOR GENERATION STATUS STATUS NEXT SUBGOAL

.

COMMANDED TASK (GOAL)

DriveOnTwoLaneRd PassVehInFront PassVehInFront DriveOnTwoLaneRd NegotiateLaneConstrict PassVehInFront . STATE-TABLES

Gear Change Required Commanded Velocity Positive But Gear Is In Reverse Commanded Velocity Negative But Gear Is In Forward (or) Commanded Velocity Positive (AND) Gears In Reverse Commanded Velocity Negative (AND) Gears In Forward

New StartupAndOperateCommand S1 proc_StartEngine S2 S1 EngineStarted S2 GearChangeRequired S3 proc_ChangeGear S3 GearChanged S2 S2 NewCommandedVelocity S4 proc_AdjustEngineThrottle S2 S4 EngineThrottleAdjusted S2 ShutDownRequested S5 proc_SetGearToPark S5 GearInPark S6 ShutDownEngine

StartUpAndOperate

Input Conditions Output Commands

S6 EngineShutDown S0 Done

New StartupAndOperateCommand S1 proc_StartEngine S2 S1 EngineStarted S2 GearChangeRequired S3 proc_ChangeGear S3 GearChanged S2 S2 NewCommandedVelocity S4 proc_AdjustEngineThrottle S2 S4 EngineThrottleAdjusted S2 ShutDownRequested S5 proc_SetGearToPark S5 GearInPark S6 ShutDownEngine

StartUpAndOperate

Input Conditions Output Commands

S6 EngineShutDown S0 Done

SLIDE 9

Representing a Propulsion Service

SLIDE 10

Propulsion Service Graph

SLIDE 11

More Visualization Features

SLIDE 12

Conditions

Scout Platoon 3rd B Section HMMWV #5 HMMWV #8 HMMWV #6 A Section Squad/Veh HMMWV #7 Squad/Veh HMMWV #3 Squad/Veh HMMWV #2 C Section HMMWV #10 HMMWV #9 HMMWV #4 PSG ConductRightFlankRecon ConductRouteReconToBridge ConductLeftFlankRecon Recon to RightFlank Conduct Traveling Overwatch Visually ClearObs Far-Side Visually Clear Route Conduct Traveling Overwatch Conduct Recon ToCP Conduct Traveling Overwatch Conduct Route Recon To CP Support Route Recon NoBypass OnLeft

ConductRouteRecon

Bridge Detected ConductWaterBridgeRecon ConductWaterObsRecon WaterObs Detected Bridge Detected SENSORY INPUT

GENERIC 4D/RCS AGENT CONTROL MODULE

SENSORY PROCESSING KNOWLEDGE DATABASE BEHAVIOR GENERATION WORLD MODEL VALUE JUDGMENT STATUS STATUS COMMANDED TASK (GOAL) COMMANDED SUBGOALS BEHAVIOR GENERATION STATUS STATUS NEXT SUBGOAL

.

COMMANDED TASK (GOAL) DriveOnTwoLaneRd PassVehInFront PassVehInFront DriveOnTwoLaneRd NegotiateLaneConstrict PassVehInFront . STATE-TABLES

New StartupAndOperateCommand S1 proc_StartEngine S2 S1 EngineStarted S2 GearChangeRequired S3 proc_ChangeGear S3 GearChanged S2 S2 NewCommandedVelocity S4 proc_AdjustEngineThrottle S2 S4 EngineThrottleAdjusted S2 ShutDownRequested S5 proc_SetGearToPark S5 GearInPark S6 ShutDownEngine

StartUpAndOperate

Input Conditions Output Commands

S6 EngineShutDown S0 Done

Gear Change Required Commanded Velocity Positive But Gear Is In Reverse Commanded Velocity Negative But Gear Is In Forward (or) Commanded Velocity Positive (AND) Gears In Reverse Commanded Velocity Negative (AND) Gears In Forward

Gear Change Required Commanded Velocity Positive But Gear Is In Reverse Commanded Velocity Negative But Gear Is In Forward (or) Commanded Velocity Positive (AND) Gears In Reverse Commanded Velocity Negative (AND) Gears In Forward

SLIDE 13

IGV Condition Example

SLIDE 14

Model Development Status

• OWL entities defined

– Classes 175 – Properties 130 – Instances 700

SLIDE 15

Issues and Lessons Learned

• Developing an ontology is a slow iterative process

– It difficult to evaluate a model construct without inputting detail. – It is very difficult to change the model once you have entered any level of detail.

• Difficult to develop consistent rules for when to use a

Classes vs. an Instance in a large domain

– Is knowledge in class restrictions or instances?

• Difficult to present large models to domain experts
• Experiences with OWL-S shows that it has applications
• utside of the semantic web.

– Would like to get involved in its development