am S Proposing a Meta-Language for Specifying Presentation - PowerPoint PPT Presentation

am S Proposing a Meta-Language for Specifying Presentation Complexity in order to Support System Situation Awareness Christoph Endres Michael Feld Christian Müller W3C Web & Automotive Workshop 14th-15th November 2012 – Rome, Italy

Situation Awareness  “knowing what is going on around you”  Automotive Domain: Helps us reduce accidents  Subgoal: Reducing distraction  System Situation Awareness  Endsley Model am S M. Feld

System Capability Endsley Model (original) Interface Design Stress & Workload Complexity Automation Task/System Factors Feedback SITUATION AWARNESS Performance State of the Decision Environment Level 1: Level 2: Level3: of Action Perception Comprehension Projection Individual Factors Information Processing Goals, Objectives Mechanisms Preconceptions (Expectations) Long-term Automaticity Memory Stores Abilities Experience Training am S M. Feld

Endsley Model (adjusted) System Capability Interface Design Complexity Automation Task/System Factors influences SYSTEM SITUATION AWARNESS Intelligent Presentation of State of the Level 1: Level 2: Level3: User / Driver Assessment Updating Mediation Information Impact of User and Information Estimation context Sources Individual Factors Information Processing Goals, Objectives Mechanisms Preconceptions (Expectations) Long-term Automaticity Memory Stores Abilities Experience Training am S M. Feld

Two-fold Research Question Presentation Aspects Driver Aspects Defining System Situation Awareness Preparation Driver related Presentation concepts Task Annotation Level 1 Assessing Estimating Cognitive Load Complexity Level 2 Updating User Presentation Profile with CL Meta Language Level 3 Achieving System Situation Awareness am S M. Feld

Estimating Presentation Complexity  Three Options:  Complexity specified by designer  “Ideal” case  nothing to do  Unstructured representation  Heuristic approaches  low confidence  Structured representation (e.g. HTML5)  ACE (Annotated Complexity Estimation)  Third case:  How to annotate complexity automatically?  ACE based on visual tree and complexity table am S M. Feld

Example Screen Layout (sim TD ) am S M. Feld

GUI Model and Visual Tree Empty Empty Empty Empty Empty Empty static Icon Icon Icon Icon Icon Icon LOGO NAMED PANEL NAMED PANEL Label with icon Label with icon Label with icon Label with icon Unnamed Panel Label with icon Label with icon Label root iconpanel named panel named panel unnamed panel button empty … empty label label label label label label icon icon with with with with with with icon icon icon icon icon icon am S M. Feld

Complexity Computation 9.3 root 2.4 1.7 1.0 3.4 iconpanel named panel named panel unnamed panel button 0.8 empty … empty label label label label label label icon icon with with with with with with icon icon icon icon icon icon 0.1 0.1 1.0 1.0 1.0 1.0 1.0 1.0 am S M. Feld

Presentation Meta Language  Developer can provides multiple presentation alternatives  System can choose based on complexity and driver workload  Goal: No new presentation language   Wrapper or Meta Language Example am S M. Feld

Implementation into a Dialogue Platform Situation-Adaptive Multimodal Dialogue Platform “In which city…?” “Hotel “Stuttgart” Search” “Stuttgart” “On what day…?” S “Where…?” “tomorrow” “Would you like to specify further search criteria?” “Please confirm…” “yes” “no” “Okay” … “Change” “I have found 7 hotels.” “Cancel” “This hotel…” “Details” “Booking” “The 2 star hotel Am Heusteig Pension in “Next” Stuttgart for 79 Euros. ” SCXML-based am S M. Feld

Dialogue Offline Evaluation  Modeling dialog cost (metrics)  Cognitive load  Time  Usability  Money  Total cost Time = 2:48 Time = 1:13 Load = 0,5 Load = 0,3  Anticipating the cost of a dialog already at design time (without expensive user study)  Expected cost on given path  Most costly transitions  Shortest / longest path  Average path  Best modality / modality comparison am S M. Feld

Estimating (Input) Interaction Workload Estimating interaction cost Analyzing the dialog model and task complexity Breaking up complex tasks into atomic tasks Rearrangement Text entry Number entry Pan / Zoom List selection Scrolling text text text text 1 text  Widgets Touchscreen Speech Eyegaze Micro-gesture cost determined in separate studies am S M. Feld

Two-fold Research Question Presentation Aspects Driver Aspects Defining System Situation Awareness Preparation Driver related Presentation concepts Task Annotation Level 1 Assessing Estimating Cognitive Load Complexity Level 2 Updating User Presentation Profile with CL Meta Language Level 3 Achieving System Situation Awareness am S M. Feld

Driver-related Cognitive Load Aspects Main Questions:  How to model cognitive load?  How to quantify cognitive load? am S M. Feld

Cognitive Load Model Tasks / Stimuli System Interaction e.g. Situation-adjustment (e.g. time) Dialog “Dry” Demand System situation-independent e.g. from Cognitive lookup table Demand Driver Cognitive Capacity distraction, stress,… Total Cognitive Load … Modality Stage Auditory Visual Metrics Processing Processing Modality Dimension Resources Resources Time, accuracy, driving Modality Code performance, pupils, Cognitive Cost biosensors,… Cognitive Resources Cognitive Stress / Tasks / Stimuli Cognitive Load Demand Distraction am S M. Feld

Cognitive User Model  User  ProcessingResource (1..n)  Dimension  CognitiveCapacity  CognitiveCost (1..n) • Amount  Dimensions: Wickens (2002)  Processing Stage: Perception / Cognition  Modality: Visual / Auditive / …  Visual Channel: Focal / Ambient  Processing Code: Spatial / Symbolic  Context  Stimuli (1..n) (permanent)  GetCurrentCognitiveDemand() : CognitiveDemand  Interaction (only temporarily present) am S M. Feld

RELATED EFFORTS am S M. Feld

Automotive Ontology On the one hand … Knowledge in the Modern Car  Sensors & Controls Inside – Outside –  Geographical Knowledge  Traffic Management  OEM Uplink  Car2car  Roadside Units (car2x)  Internet Services  Passenger Profiles “ Information Hub ”  Driving Habits Roads, times, driving styles… –  Personal Devices  … am S M. Feld

… and on the other hand Feature-rich In-car Applications  Driver Assistance  Navigation  Parking Assistance  Comfort Controls  eMail, SMS  Twitter, Instant Messaging  PIM  News  Information Search  Entertainment, Music  Navitainment  Local Information  … am S M. Feld

High-Level Structure View from the users‘s perspective AutomotiveWorld User Context Basic Dimensions Vehicle Interactions Devices Preferences External Physical Presentation Trip am S M. Feld

User Model User BasicDimensions GUMO – General User Modeling Ontology MentalState Personality extraversion timePressure agreeableness cognitiveLoad conscientiousness irritation neuroticism trauma openness Abilities canSee Characteristics talkative canHear assertive canSwim dominant sight quiet hearing thorough helpful PhysiologicalState EmotionalState heartbeat happiness bloodPressure anxiety arousal anger fatigue disgust alcoholLevel sadness am S M. Feld

Meta Information am S M. Feld

Driving Simulation We created a new 3D Driving Simulator in order  to measure the driver’s distraction in a controlled lab environment The simulator is connected via sockets with the  HMI that displays important information about the upcoming road segment The screens show examples from sim TD  dynamic objects supported! am S M. Feld

Driving Performance Measures The Simulator can record the driven path  as a list of way points In the Drive Analyzer this path can be  compared to a predefined “ideal line” by computing the average deviation. The smaller the area between both lines,  the higher the driving quality (c.f. evaluation of Lane Change Test) The new 3D Driving Simulator with the  shown features is now able to simulate the Lane Change Test from the beginning Drive Analyzer in top view and chase camera view. The Arbitrary map models can be loaded (as  pink line denotes the ideal path and the yellow line the long as they can be processed with driven path. Blender) The physics simulation is based on a  realistic car Triggers to hide/show lane signs can be  placed Evaluation after drive with common  “deviation computation” approach This approach can be modified and  extended to our future needs  distance more than 40 meters: hidden signs  distance less than 40 meters: visible signs am S M. Feld