Tools and and Methods Methods for for the Design the Design - - PowerPoint PPT Presentation

▶
tools and and methods methods for for the design the
SMART_READER_LITE
LIVE PREVIEW

Tools and and Methods Methods for for the Design the Design - - PowerPoint PPT Presentation

Dec 02, 2022 350 likes •945 views

Tools and and Methods Methods for for the Design the Design Tools of Multi Multi- -Device Device User User Interfaces Interfaces of Fabio Patern fabio.paterno@isti.cnr.it http://giove.isti.cnr.it/~ fabio/ ISTI-C.N.R. Pisa, Italy

slide-1
SLIDE 1

Tools Tools and and Methods Methods for for the Design the Design

  • f
  • f Multi

Multi-

  • Device

Device User User Interfaces Interfaces

Fabio Paternò

fabio.paterno@isti.cnr.it http://giove.isti.cnr.it/~ fabio/ ISTI-C.N.R. Pisa, Italy

slide-2
SLIDE 2

Tools and Methods for the Design of Multi-Device User Interfaces 2

Pervasive Pervasive Usability Usability

Ever-increasing introduction of new types of

interactive devices

How to support designers and developers? How to obtain interfaces able to adapt to

multiple devices (any device) while preserving usability?

slide-3
SLIDE 3

3

Structure Structure of the

  • f the Tutorial

Tutorial

Time Subject Duration

9.00-9.15 Introduction, Basic Concept, Issues 15’ 9.15-9.30 Model-based design of multi-device interfaces 15’ 9.30-9.45 Task/Platform Taxonomy 15’ 9.45-10.00 exercise – Multi-device interface 15’ 10.00-10.15 Task Analysis and Modelling 15’ 10.15-10.45 TERESA 30’ 10.45-11.00 Semantic redesign for different interaction platforms 15' 11.00-11.30 Coffee Break 30’ 11.30-11.45 Model-based design of multi-modal interfaces 15’ 11.45-12.15 Migratory Interfaces 30’ 12.15-12.30 exercise – Migratory interfaces 15’ 12.30-12.45 Architectures for Migratory Interfaces 30’ 12.45-13.00 Research agenda & Conclusions 15’

slide-4
SLIDE 4

Tools and Methods for the Design of Multi-Device User Interfaces 4

Possible Possible Views Views of

  • f

an an I nteractive I nteractive Systems Systems

Task and object – I want to select a

work of art

Abstract Interface – Single selection

  • bject with high cardinality

Concrete Interface – List Interaction

  • bject with X elements

Code – List object in Java or XHTML or

....

slide-5
SLIDE 5

5

Significant Significant Models Models in HCI in HCI

Models Task & Object Models Interactor User Platform Environment

Task and object model Abstract User Interface Concrete User Interface User Interface

Design time Multiple-Version

Run-time events Reconfiguration

Run time Adaptivity

slide-6
SLIDE 6

6

Tools Tools in in Model Model-

  • based

based environments environments

Models Task & Object Models Abstract UI Concrete UI Context Designer Modelling Tools Analysis Tools Development Tools User Interface Context of use Reverse Engineering Tools Design criteria Transformations

slide-7
SLIDE 7

Tools and Methods for the Design of Multi-Device User Interfaces 7

Adaptation Adaptation

Adaptability - Capacity of a UI to adapt

its behaviour from an explicit human intervention

Adaptivity - Capacity of a UI to adapt

without any explicit human intervention

What can be adapted? Presentation,

Navigation, Content

slide-8
SLIDE 8

Tools and Methods for the Design of Multi-Device User Interfaces 8

Context Context-

  • dependent

dependent Adaptation Adaptation

About the environment:

slide-9
SLIDE 9

Tools and Methods for the Design of Multi-Device User Interfaces 9

Design of Design of Multi Multi-

  • Device

Device I nterfaces I nterfaces: : Current Current Practice Practice

Manual solutions,

expensive

Transcoders,

low cost/low usability

Style sheets,

partial solution

slide-10
SLIDE 10

Tools and Methods for the Design of Multi-Device User Interfaces 10

State of Art State of Art

Aura project at CMU (adaptation at application

level)

Pebbles project at CMU (limited to appliances

control)

XIML has not public tool support

http://www.ximl.org/ (developed by a forum driven by RedWhale software)

UIML does not support high-level task descriptions

http://www.uiml.org/ (developed by Harmonia and cooperation with Virginia Tech)

slide-11
SLIDE 11

Tools and Methods for the Design of Multi-Device User Interfaces 11

XForms XForms

Apply concepts from model-based design Separate presentation from content (form

controls markup is separated from data-types and returned values)

XForms 'native' form controls are device-

independent

Reduce need for scripting through client-side

checking

XML instance is returned allowing strong

typing

slide-12
SLIDE 12

Tools and Methods for the Design of Multi-Device User Interfaces 12

Motivations Motivations for for task task analyis analyis and and modelling modelling

Main usability principle:

Focus on the users and their tasks

Tasks represent the logical activities

performed for reaching user goals

Need for modelling is most acutely felt when

the design aims to support system implementation as well

Especially for large projects and some application

domains

slide-13
SLIDE 13

Tools and Methods for the Design of Multi-Device User Interfaces 13

Task – activity that has to be performed to

reach a goal

Goal

desired modification of state Attempt to receive state information

Each task is associated with one goal Each goal is associated with one or multiple

tasks

Multiple abstraction levels - Basic task Task Analysis Task Models

Defini Definitions tions

slide-14
SLIDE 14

Tools and Methods for the Design of Multi-Device User Interfaces 14

Task Task – –related issues in multi related issues in multi-

  • platform environments

platform environments

Platform definition Same task on multiple platforms in the

same manner

Same task on multiple platforms but

performed in different manner

Dependencies among tasks performed on

different platforms

Tasks meaningful only on a single platform

type

slide-15
SLIDE 15

Tools and Methods for the Design of Multi-Device User Interfaces 15

Examples Examples of

  • f Platform

Platform-

  • dependent

dependent tasks tasks

Comparing prices of flights and making reservations. Checking status of a particular flight. Gathering background on a company, including maps. Getting driving directions to a company—while on the road. Browsing medical information. Monitoring a medical condition. Reading a movie review and/or watching a trailer. Purchasing a cinema ticket to avoid the line.

Desktop system Mobile System

slide-16
SLIDE 16

Tools and Methods for the Design of Multi-Device User Interfaces 16

Same Same task on multiple task on multiple platforms platforms with with different different user user interface interface

  • bjects
  • bjects
slide-17
SLIDE 17

Tools and Methods for the Design of Multi-Device User Interfaces 17

Same Same task on multiple task on multiple platforms platforms with with different different task task decomposition decomposition

slide-18
SLIDE 18

Tools and Methods for the Design of Multi-Device User Interfaces 18

Dependencies Dependencies among among tasks tasks performed performed on

  • n different

different platforms platforms

slide-19
SLIDE 19

Tools and Methods for the Design of Multi-Device User Interfaces 19

Exercise Exercise Multi Multi-

  • device

device I nterfaces I nterfaces

slide-20
SLIDE 20

Tools and Methods for the Design of Multi-Device User Interfaces 20

TERESA TERESA

Mixed initiative Model-based XML-based Flexible development Web-oriented but can be extended to other

environments

Available at

http://giove.isti.cnr.it/teresa.html

slide-21
SLIDE 21

Tools and Methods for the Design of Multi-Device User Interfaces 21

One Model – Many Interfaces

slide-22
SLIDE 22

Tools and Methods for the Design of Multi-Device User Interfaces 22

TERESA TERESA Environment Environment for for Flexible Flexible Development Development

… System task model Desktop (XML) System task model Cellphone (XML) System task model Voice (XML) Task model for envisioned nomadic applications XHTML MP, WML … XHTML, Java … VoiceXML, Salt … Abstract UI Cellphone ( XML ) Abstract UI Desktop ( XML) Abstract UI (XML) Voice Concrete UI Cellphone (XML) Concrete UI Desktop ( XML) Concre te UI (XML) Voice

slide-23
SLIDE 23

23

TERESA XML TERESA XML

Two platform-independent languages : task (CTT)

and abstract interface

One level (concrete interface) represented

through a number of platform dependent languages

Designers aware of the potential platforms (not

devices) early on in the design process

Method allows developers to avoid dealing with a

plethora of low-level details (transformation from concrete description to implementation is automatic)

Easy to add support for new implementation

languages

slide-24
SLIDE 24

Tools and Methods for the Design of Multi-Device User Interfaces 24

Design Design Practice Practice

Grouping Grouping Ordering Relation Important !

slide-25
SLIDE 25

Tools and Methods for the Design of Multi-Device User Interfaces 25

Another Another example example

slide-26
SLIDE 26

Tools and Methods for the Design of Multi-Device User Interfaces 26

Communication Communication-

  • oriented
  • riented

Composition Composition operators

  • perators

Grouping: a set of elements logically

related to each other

Ordering: existing of an order among

interactors (i.e. temporal)

Relation: 0ne interactor related to a

group of other interactors (i.e. disabling them)

Hierarchy: a logical hierarchy among a

set of interactors

slide-27
SLIDE 27

Tools and Methods for the Design of Multi-Device User Interfaces 27

Structuring Structuring the the User User I nterface I nterface

Grouping – Example: Task decomposition ->

grouping of correspoding interaction techniques

Ordering – Sequential communicating tasks -

> adjacent interaction techniques

Relation – Control tasks (one to many

relations)

Hierarchy – Frequent tasks –> More space

  • r larger attributes
slide-28
SLIDE 28

Tools and Methods for the Design of Multi-Device User Interfaces 28

The The Structure Structure of the

  • f the

Abstract Abstract User User I nterface I nterface

Language platform-

independent

Interactors (selection,

navigator, activator, …)

Communication-

  • riented composition
  • perators

Connections among

presentations

slide-29
SLIDE 29

Tools and Methods for the Design of Multi-Device User Interfaces 29

Presentation2

Hierarchy

Description Edit Multiple-selection

…….

The Structure of the Abstract User Interface

User Interface

Presentation1

Grouping

Selection Edit Navigator

Connection

slide-30
SLIDE 30

Tools and Methods for the Design of Multi-Device User Interfaces 30

A nomadic museum A nomadic museum application application

slide-31
SLIDE 31

Tools and Methods for the Design of Multi-Device User Interfaces 31

TERESA TERESA support support in in Development Development

Choice of device platform/type Settings for general attributes, How to implement composition operators How to implement interactors Summary of design choices and preview Recording and reuse of concrete aspects

defined

slide-32
SLIDE 32

Tools and Methods for the Design of Multi-Device User Interfaces 32

Concrete User Interface

Defines some concrete aspects of the user interface Provides indications for the implementation of abstract

interactors

  • < interactor id= "Go_to_section3"> -

< interaction> - < control> -

  • < navigator>
  • < button label= "Go to selected section"/>
  • < /navigator>
  • < /control>
  • < /interaction>
  • < /interactor>
  • < interactor id= "Go_to_section3"> -

< interaction category= "interaction"> -

  • < control type= "control">
  • < navigator object= "navigator"/>
  • < /control>
  • < /interaction>

Abstract levelI Concrete level

EXAMPLE

slide-33
SLIDE 33

Tools and Methods for the Design of Multi-Device User Interfaces 33

Example of platform-dependent concrete interactor choice

EXAMPLE: Single choice abstract interactor

Mobile Phones

Radio Button Drop Down List Drop Down List

Desktop Computers

Radio Button List Box List with scrollbars

Cardinality

Low cardinality Medium cardinality High cardinality

slide-34
SLIDE 34

Tools and Methods for the Design of Multi-Device User Interfaces 34

Example Example of platform-dependent f platform-dependent composition composition operator implementation perator implementation

EXAMPLE: Grouping Operator

Desktop Computers

Fieldset Bullet Background Color

Mobile Phones

Unordered List On Column Fieldset (only for medium-large phones)

Column-oriented organization Row-oriented organization

slide-35
SLIDE 35

Tools and Methods for the Design of Multi-Device User Interfaces 35

Example Example of

  • f TERESA

TERESA-

  • generated

generated User User I nterface I nterface-

  • DEMO

DEMO

slide-36
SLIDE 36

Tools and Methods for the Design of Multi-Device User Interfaces 36

Use Use of

  • f Reverse

Reverse Engineering Engineering

Tasks and Objects Abstract UI Concrete UI Final UI Tasks and Objects Abstract UI Concrete UI Final UI

Platform X Platform y

Transcoding Redesign

slide-37
SLIDE 37

Tools and Methods for the Design of Multi-Device User Interfaces 37

Semantic Semantic Redesign Redesign

Use of abstractions for changing the

design for a new target platform

Use of reverse engineering to obtain

the abstractions

Different possible solutions

slide-38
SLIDE 38

Tools and Methods for the Design of Multi-Device User Interfaces 38

Semantic Redesign with Semantic Redesign with Forward Engineering Forward Engineering

Redesign

4 4

Abstract User Interface Concrete User Interface

5 5 5

Mobile User Interface

6 1

Abstract User Interface Concrete User Interface

2

Desktop User Interface

3

slide-39
SLIDE 39

Tools and Methods for the Design of Multi-Device User Interfaces 39

Reverse and Forward Reverse and Forward Engineering Engineering

Abstract User Interface Concrete User Interface

2

Desktop User Interface

1

Redesign

3 3

Abstract User Interface Concrete User Interface

4 4 4

Mobile User Interface

5

slide-40
SLIDE 40

Tools and Methods for the Design of Multi-Device User Interfaces 40

Support Support for for Redesign Redesign

Page splitting based on the composition

  • perators and the number of interactors

Connections: original ones + those

derived from page splitting

Images: resize depending on target

device keeping the same aspect ratio

Tables for converting terms and labels

slide-41
SLIDE 41

Tools and Methods for the Design of Multi-Device User Interfaces 41

From From Desktop Desktop to to Cell Cell-

  • phone

phone

Grouping 0 Grouping 1 Grouping 2

slide-42
SLIDE 42

Tools and Methods for the Design of Multi-Device User Interfaces 42

From From Desktop Desktop to to Cell Cell-

  • phone

phone

slide-43
SLIDE 43

Tools and Methods for the Design of Multi-Device User Interfaces 43

Another Another example example of

  • f

semantic semantic redesign redesign

slide-44
SLIDE 44

Tools and Methods for the Design of Multi-Device User Interfaces 44

Task Task-

  • based Semantic

based Semantic Redesign Redesign

5 5

Abstract User Interface Concrete User Interface

5

Mobile User Interface

6 1

Abstract User Interface Concrete User Interface

2

Desktop User Interface

3 4 4

Redesign Nomadic Task Model

4

slide-45
SLIDE 45

Tools and Methods for the Design of Multi-Device User Interfaces 45

From From Desktop Desktop to to Cell Cell-

  • phone

phone

slide-46
SLIDE 46

Tools and Methods for the Design of Multi-Device User Interfaces 46

Semantic Semantic redesign redesign -

  • demo

demo

slide-47
SLIDE 47

Tools and Methods for the Design of Multi-Device User Interfaces 47

Vocal Vocal I nteraction I nteraction

Characteristics: linear, not persistent, faster

and more natural for some operations

Provide feedback to check the status of

application

Brief prompts and short lists of options to

reduce memory capability

Management of events (no-input , no-match,

help)

slide-48
SLIDE 48

Tools and Methods for the Design of Multi-Device User Interfaces 48

Speech Speech implementation implementation of

  • f

composition composition operators

  • perators
  • Grouping:
  • Insert a sound
  • Insert a pause
  • Use some keywords
  • Use a specific volume of

synthesizer voice

  • Ordering
  • Alphabetical order
  • Use some keywords
  • Relation
  • Change context

(change type of menu)

  • Hierarchy
  • Increase or decrease the volume
  • f synthesizer voice
slide-49
SLIDE 49

Tools and Methods for the Design of Multi-Device User Interfaces 49

Specifying Specifying general general parameters parameters for for all all presentations presentations

slide-50
SLIDE 50

Tools and Methods for the Design of Multi-Device User Interfaces 50

Vocal Vocal I nteraction I nteraction Generation Generation -

  • DEMO

DEMO

slide-51
SLIDE 51

51

VUI VUI vs vs GUI GUI

  • Welcome

Welcome message message

System: Welcome to the Marble Museum Voice Response System. This service recognises your speech to provide you with the information you request. System: (grouping sound ) If you would like some general information, say information; if you would like information about specific artworks, say artworks; if you would like to book a ticket, say ticket. (grouping sound ) Caller: Artworks First presentation System: Ok, loading information about artworks. (The system goes to presentation about Artworks Section)

System: (grouping sound ) The artworks contained in the section are the following: Boat, Totem,

  • Hole. If you would like information on one of

these please say its name (grouping sound ) Remember that if you would like to return to the main menu, say home. System: (Time out) Please say your choice. Caller Boat Fourth presentation System: (feedback) ok, you have chosen Boat. (The system goes to presentation about Boat) System: The Boat has been achieved through the subtle divisions of the planes enveloping its central part, which is only rough-hewn; the material is white marble. (Five second pause) Remember that if you would like to return to the main menu, say home or if you would like to go back to the previous menu, say back. Fifth presentation Caller: Home

  • Management of no input

Management of no input event event

  • Provide

Provide feedback feedback

  • Description

Description Object Object

  • Composition

Composition operators

  • perators
slide-52
SLIDE 52

Tools and Methods for the Design of Multi-Device User Interfaces 52

Generation of Generation of Multi Multi-

  • Modal

Modal I nterfaces I nterfaces

X+ V Supported by OPERA Browser, also for

PDAs

EMMA not supported by any public tool SMIL not interaction oriented Identification of design criteria for

multimodal platforms

slide-53
SLIDE 53

Tools and Methods for the Design of Multi-Device User Interfaces 53

I nteractor I nteractor I nteraction I nteraction

Prompt: represents the interface output

indicating that it is ready to receive an input.

Input: represents how the user can

actually provide the input.

Feedback: represents the feedback of

the system after the user input.

slide-54
SLIDE 54

Tools and Methods for the Design of Multi-Device User Interfaces 54

Multimodal properties Multimodal properties

Complementary Redundancy Assignment Equivalence

slide-55
SLIDE 55

55

Design of Design of Multimodality Multimodality Support Support

Identification of new platforms (multimodal

desktop, multimodal PDA, …)

Design how to support composition operators

and interactors

multimodal desktop:

compositon operators -> graphically supported interactors -> graphical prompt, input either graphical

  • r vocal, feedback in both modalities

multimodal pda:

compositon operators -> supported both graphically

and vocally

interactors -> vocal prompt, input either graphical or

vocal, feedback in both modalities

slide-56
SLIDE 56

Tools and Methods for the Design of Multi-Device User Interfaces 56

Multimodal desktop Multimodal desktop

Composition operators Graphical

Assignment

Interactors

OnlyOutput Graphical Assignment Interaction

Prompt: Graphical Assignment Input: Equivalence Feedback: Graphical Assignment

slide-57
SLIDE 57

Tools and Methods for the Design of Multi-Device User Interfaces 57

Multimodal PDA Multimodal PDA

Composition operators Redundancy Interactors

OnlyOutput Complementary/

Redundancy

Interaction

Prompt: Redudancy Input: Equivalence Feedback: Redudancy

slide-58
SLIDE 58

Tools and Methods for the Design of Multi-Device User Interfaces 58

Example Example -

  • Demo

Demo

(grouping sound) Welcome in the movie description page. In the Robots film a world is populated entirely by

  • robots. This is the story of a young genius, Rodney,

who wants to make robots capable…. (grouping sound) Would you like to book a ticket or come back to home? Welcome in the description page

  • f Robots film