Trans- -Modal Modal Interface Interface Trans Migration Migration Renata Bandelloni, Silvia Berti, Fabio Paternò HIIS Laboratory ISTI-CNR Pisa, Italy Mobile HCI 2004 13 - 16 September 2004 University of Strathclyde, Glasgow, Scotland
Migratory Interfaces Interfaces Migratory � Interface can transfer among different platform, allowing the users to continue their interaction. � Interaction Continuity � Platform Adaptation � Interface Usability 17 September 2004 Mobile HCI 2004 2
Our Approach Approach Our � Logical description of the application interface obtained from TERESA. � Platform adaptation � Runtime state of the interface. � Interaction continuity 17 September 2004 Mobile HCI 2004 3
Multilevel generation of Web generation of Web Multilevel interfaces interfaces � Task Model � Abstract User Interface � Platform specific implementation 17 September 2004 Mobile HCI 2004 4
Migration Service Service Architecture Architecture Migration Migration perform ance Migration perform ance ( new Url new Url) ) ( 3 3 Server Server 1 1 Migration request Migration request 2 2 ( url ( url, ,runtim e runtim e data) data) Mapping & & Mapping State extraction extraction and and Adaptation Adaptation State ( static static- - runtim e runtim e data) data) ( 17 September 2004 Mobile HCI 2004 5
Identification of the of the Identification corresponding presentation presentation corresponding � From the current page -> abstract presentation -> Corresponding tasks � Calculation of the most similar abstract presentation in the target platform � The page supporting the most similar set of tasks � In case of multiple similar pages -> the one with the last interaction is selected 17 September 2004 Mobile HCI 2004 6
Presentation Mapping Mapping Presentation DeviceA DeviceB Presentation1 Presentation1 One to to One One One task1 task1 task2 task2 task3 task3 task4 task4 DeviceB Presentation1 task1 DeviceA One to to Many Many One task2 Presentation1 task1 DeviceB task2 Presentation2 task3 task3 task4 task4 DeviceA Presentation1 task1 Many to to One One Many DeviceB task2 Presentation1 task1 DeviceA task2 Presentation2 task3 task3 task4 task4 17 September 2004 Mobile HCI 2004 7
Trans- -Modal Modal Migration Migration Trans � Many devices supporting Internet access, offering various interaction capabilities � Change modality of input depending on different contexts and environments � Natural interaction � Multichannel access 17 September 2004 Mobile HCI 2004 8
Scenario - - Graphic Graphic to to Vocal Vocal Scenario Louis is at Home and wants to While Louis is selecting the organize a dinner with his friends Mermaid restaurant main page, for this evening. he realizes that it is getting late He turns on his personal computer . Synthesiser: “If you want phone number say phone , if you want to know the location say address , if you want to know the menu of the day say menu . If you want to book a table say book .” Louis: “ book “. Synthesiser: “Say your name” Louis: “ Louis ” Synthesiser: “Say your surname” Louis: “ Lestat ” Synthesiser: “Say date of reservation” Louis: “ 2 June 2004” Synthesiser: “Say the time of reservation” Louis: “9 pm” Synthesiser: “How many people?” Louis: “ three ” Synthesizer: “Louis Lestat, you booked a table for three people on 2 June 2004 at 9 pm. Say confirm to confirm your booking and cancel to discard it”. Louis: ” confirm ”. 17 September 2004 Mobile HCI 2004 9
Graphic- -to to- -vocal Migration vocal Migration Graphic Migration Server Server Migration Graphic Graphic Vocal Vocal Desktop Desktop Task Mapping Mapping Task TASKS TASKS engine engine AUI AUI State State extraction extraction XHTML VoiceXML and and Adaptation Adaptation 17 September 2004 Mobile HCI 2004 10
Trans- -Modal Modal Runtime Runtime State State Trans Mapping Mapping Sender Sender Sender Sender Subject Subject Subject Subject Date Date Date Date Size Size Size Size &Tim e &Tim e &Tim e &Tim e � Vocabulary Luois Luois Last minute offer Last minute offer May 25 May 25 1k 1k lestat lestat 9: 23 9: 23 � Information organization George George Re: Cam eleon Re: Cam eleon June 3 June 3 3k 3k Clow Clow meeting meeting 11: 02 11: 02 Luois Luois Learn about Java Learn about Java June 5 June 5 2k 2k � Different concrete Lestat Lestat 12: 20 12: 20 Mary Rex Mary Rex Re: Cam eleon Re: Cam eleon June 7 June 7 1k 1k objects meeting meeting 10: 00 10: 00 � Not a simple one to one mapping � Task not Supported � Task implemented with different concrete objects � Adaptation of runtime data 17 September 2004 Mobile HCI 2004 11
Task implemented by means of different Task implemented by means of different concrete objects and task not supported concrete objects and task not supported (grouping sound) Please say your name. Please say your surname. Please say the date of reservation. What type of menu do you prefer: fish; meat; vegetable (grouping sound) Your name is … . Your surname is … . You have booked a table on … and You prefer… (grouping sound) If you want to cancel the reservation , say delete; If you want to confirm the reservation, say ok (grouping sound) . Object that that Support Support the task the task Object Grouping Operator Operator Grouping “ Provide Provide date” date” “ Task not not supported supported Task 17 September 2004 Mobile HCI 2004 12 in Voice interface in Voice interface
Adaptation of of runtime runtime data data Adaptation Your name is Louis Louis . You have 3 July July . booked a table on 3 (grouping sound) Please say your name. Please say your surname. Please say the date of reservation. Nam e: Louis Date: 3 July What type of menu do you prefer: fish; meat; vegetable (grouping sound) Your name is … . Your surname is … . You have booked a table on … and You prefer… (grouping sound) If you want to cancel the reservation , say delete; If you want to confirm the reservation, say ok (grouping sound) . 17 September 2004 Mobile HCI 2004 13
Conclusions and Future Work and Future Work Conclusions � Migratory interfaces for more natural interaction in multi-device environments � Extension for multi-modal interfaces under development � Distributed migratory interfaces 17 September 2004 Mobile HCI 2004 14
Recommend
More recommend
