An Integrated Approach for Platform-Independent Web Applications - - PowerPoint PPT Presentation

▶
an integrated approach for platform independent web
SMART_READER_LITE
LIVE PREVIEW

An Integrated Approach for Platform-Independent Web Applications - - PowerPoint PPT Presentation

Apr 26, 2023 241 likes •523 views

An Integrated Approach for Platform-Independent Web Applications W3C Multimodal Interaction Workshop Sophia-Antipolis, 19.-20.7.2004 Gerald Hbsch and Thomas Springer Dresden University of Technology {huebsch,springet}@rn.inf.tu-dresden.de

slide-1
SLIDE 1
  • G. Hübsch, T. Springer – Dresden University of Technology

An Integrated Approach for Platform-Independent Web Applications

W3C Multimodal Interaction Workshop Sophia-Antipolis, 19.-20.7.2004 Gerald Hübsch and Thomas Springer Dresden University of Technology {huebsch,springet}@rn.inf.tu-dresden.de

slide-2
SLIDE 2
  • G. Hübsch, T. Springer – Dresden University of Technology

2

Motivation

Hotel Device Profile

Trip Office WAN

GSM/UMTS/ Connexion LAN Device Profile Device Profile User Profile Context

  • f use

Context

  • f use

Context

  • f use

User Profile User Profile

Application Server Customer visit

Device Profile User Profile Context

  • f use

WLAN + WAN

  • Existing and

upcoming wireless network infrastructures enable ubiquitous web-based applications

  • State-of-the-Art web

application design: desktop-style target platforms for fully- fledged browsers

  • Today requirements

imposed by the heterogeneity and limitations of mobile devices are neglected

slide-3
SLIDE 3
  • G. Hübsch, T. Springer – Dresden University of Technology

3

Adaptation as a promising solution

Adaptation is a promising solution to overcome the increasing differences in requirements imposed by mobile device technology Today’s adaptation approaches focus on User-Interface-level and Widget-level adaptation and support single modalities

Context Information

User-Interface and Widget-Level Structure-Level Application-Level

Application-specific Application-independent

Functional adaptation Complexity adjustment Adaptation of navigation structures & dialog flows Layout, semantic and syntactic UI transformation

slide-4
SLIDE 4
  • G. Hübsch, T. Springer – Dresden University of Technology

4

Dialog Description Language

slide-5
SLIDE 5
  • G. Hübsch, T. Springer – Dresden University of Technology

5

Single Source Authoring

  • Single Source Authoring uses generic dialog descriptions and

applies dedicated adaptation algorithms parameterised by context information to generate device-specific content representations

Application

Generic Description + Semantic meta- information

Adaptation algorithms

</table> <textcell> <f> Das ist aber egal </f> <text> Auf ihre Augen kommt’s nicht an </text> </textcell> <textblock> <textline> Dieser Text sagt </textline> <textline> auch nichts aus. </textline> </textblock> <textblock> <textline> Aber das sollte auch OK
  • sein. Es folgt ein Bild,
das sagt mehr als tausend Worte: <img src=“bild.gif”/> </p> </body>

Description B Description C Description A Context Information DDL

slide-6
SLIDE 6
  • G. Hübsch, T. Springer – Dresden University of Technology

6 XSL Stylesheet for WML

Adaptation Concepts

Semantic and Syntactic Dialog Adaptation (UI & Widget Level Adaptation)

Dialog Source: Generic Device Independent Markup with Semantic Annontations Semantic Adaptation (Selective Content, Fragmentation) Syntactic Adaptation (XSLT) XSL Stylesheet for HTML Context Informationen

  • Device Class
  • Network type
  • User

preferences selects Adapted Dialog: Device Specific Layout, Content and Markup

slide-7
SLIDE 7
  • G. Hübsch, T. Springer – Dresden University of Technology

7

<ddl> … <dialog> … <part> <property name=“type“>label</property> … </part> … </dialog> … </ddl>

Dialog Description Language

  • XML-based device independent markup language

for Single Source Authoring of web content and web application UIs

  • Separation of structure, content and

presentation

  • Inheritance concept for content reuse
  • Inline meta-information governs adaptation
  • Language concepts support semantic and

syntactic adaptation

  • Support for dynamic content and interaction with

application logic (MVC)

  • Modular adaptation pipeline implements adaptation

algorithms

slide-8
SLIDE 8
  • G. Hübsch, T. Springer – Dresden University of Technology

8

Dialog Description Language

Abstract DDL Syntax

ddl include src test head dialog class name extends test content name extends test meta name content property name part test part class name extends test constant name test reference constant property part test #PCDATA

slide-9
SLIDE 9
  • G. Hübsch, T. Springer – Dresden University of Technology

9

Dialog Description Language

DDL Syntax - Example

ddl include src test head dialog class name extends test content name extends test meta name content property name part test part class name extends test constant name test reference constant property part test #PCDATA

<ddl> <property name="title">MyDDL</property> <dialog> <part> <property name="type">label</property> <property name="content"> Single Source Authoring </property> <property name="bold">true</property> </part> </dialog> </ddl>

slide-10
SLIDE 10
  • G. Hübsch, T. Springer – Dresden University of Technology

10

Selective exclusion / Alternative content based on context information

Semantic Adaptation

<part test="class='pda' or class='desktop'"> <property name="type">image</property> <property name="src">logo.jpg</property> </part> <part test="class='wap'"> <property name="type">label</property> <property name="content"> Rent-A-Bike Login </property> </part> Context information query

slide-11
SLIDE 11
  • G. Hübsch, T. Springer – Dresden University of Technology

11

part part part part part part part part part part part part part part part

Semantic Adaptation

Fragmentation: Approach to meet size and usability constraints imposed by the target platform and the network protocol through semi-automatic dialog splitting

atom atom atom Atoms form sematically indivisable units Dialog-split calculated by fragmentation algorithm

Dialog

Fragment 1 Fragment 2

slide-12
SLIDE 12
  • G. Hübsch, T. Springer – Dresden University of Technology

12

DDL Atom Example: Dialog fragmentation

Semantic Adaptation

slide-13
SLIDE 13
  • G. Hübsch, T. Springer – Dresden University of Technology

13

Adaptation Engine

slide-14
SLIDE 14
  • G. Hübsch, T. Springer – Dresden University of Technology

14

ClientRecognizer URLGetter XMLParser

Image DDL

DDLPreprocessor ImageFilter

HTTP Request

DocumentValidation Content Storage

WML smallHTML, HTML

Fragment Cache

DOM DOM

Fragmentation

Fragment URL Document URL DOM HTTP Response

XSLT- Stylesheets Device Profiles Filter Configuration Transcoding Servlet FragmentGetter WMLCompiler XSLTProcessor

Dialog Adaptation Architecture

DDL Adaptation Engine Architecture (Simplified)

slide-15
SLIDE 15
  • G. Hübsch, T. Springer – Dresden University of Technology

15

Application Logic and Backend Integration

slide-16
SLIDE 16
  • G. Hübsch, T. Springer – Dresden University of Technology

16

Application Logic and Backend access

Application logic interaction and Backend Integration

  • Event-driven MVC pattern
  • Dedicated language extensions for resource interface

declaration, dynamic content sections and event listeners

  • Integration with adaptation concepts and Adaptation Engine

<resource name="LoginHandler" class="tud.eventhandler.RabHandler"> <method name="validatelogin"> <param name="lastname"/> <param name="account"/> </method> </resource>

DDL Event Handler Resource

<resource name="RAB_Data" class="sql.RABConnection"> <method name="getreservations"> <field name="__label" /> <field name="__rid" /> </method> </resource>

DDL Backend Access Resource

slide-17
SLIDE 17
  • G. Hübsch, T. Springer – Dresden University of Technology

17

Structure-Level Adaptation

slide-18
SLIDE 18
  • G. Hübsch, T. Springer – Dresden University of Technology

18

Structure-Level Adaptation

  • Semi-automated adaptation of web applications with complex

user interfaces is important for high user acceptance and low development costs

  • Different application dialog flows for one task
  • Serialisation of complex dialogs and user interactions
  • Adaptation through dialog modularisation

– Application user interfaces are described as a set Transactional Dialog Units (TDU), e.g. authored in DDL

TDU set

Composition rules Navigation structure UI & Widget-level Adaptation

slide-19
SLIDE 19
  • G. Hübsch, T. Springer – Dresden University of Technology

19

Structure-Level Adaptation: Example

Example: Phonebook Application

TDU 1: Search for… TDU 2: Search

  • ptions

TDU 3: Search results Navigation structure Composition rules

Structure-level Adaptation

slide-20
SLIDE 20
  • G. Hübsch, T. Springer – Dresden University of Technology

20

Future Research

Adaptation & Multimodal Interaction

slide-21
SLIDE 21
  • G. Hübsch, T. Springer – Dresden University of Technology

21

Adaptation and Multimodality Multimodal user interaction + Adaptation

Current I/O modalities Device Capabilities Device Collaboration Context

  • f Use,

Environment User Preferences

slide-22
SLIDE 22
  • G. Hübsch, T. Springer – Dresden University of Technology

22

Adaptation and Multimodality Use Cases: Adaptive Multimodal Interfaces for industry and business applications

  • Multimodal Maintenance Documentation (EU FP6

SNOW): Display + Voice + AR + Gesture + Robotic vision

– Hands-free access to maintenance documentation and procedures in complex work situations – On-site capture of repair and maintenance data – Dynamic selection of interaction channels

  • Multimodal Business Applications

– Adaptive multimodal access to corporate applications: device and modality independent description of UI – Adaptive interaction channel selection – Dedicated platform services for multimodality support

slide-23
SLIDE 23
  • G. Hübsch, T. Springer – Dresden University of Technology

23

Adaptation and Multimodality Multimodal user interaction

  • Adaptive multimodal applications can provide dynamic

support for different modalities

  • Novel adaptation concepts are required for modality-

independent application description, interaction channel selection and channel utilization

slide-24
SLIDE 24
  • G. Hübsch, T. Springer – Dresden University of Technology

24

Adaptation and Multimodality

Adaptation architecture for multimodal applications

  • Additional components for output splitting and input fusion
  • Client supports voice recognition, visual rendering and speech

synthesis (e.g. XHTML+Voice Profile capable browser)

  • Client supports visual rendering only, voice recognition and

speech synthesis on VXML gateway

Application Server Adaptation Engine Output Splitting Input Fusion VXML Gateway VXML HTML XHTML + Voice

slide-25
SLIDE 25
  • G. Hübsch, T. Springer – Dresden University of Technology

25

Adaptation in all phases of the software engineering process

Adaptation and Multimodality

Analysis Design Implementation Test & Deployment

  • Mobile scenarios

in Requirements Engineering

  • Mobile

computing in Business Processes

  • Context-aware

applications

  • Modality

annotations

  • Ambient-aware

software architectures

  • Adaptation

infrastructures

  • Design of

adaptive UIs

  • Weaving context

into application logic and user interface

  • Interaction with

adaptation engines

  • Implementation of

multimodal and adaptive UIs

  • Deployment on

heterogeneous platforms

  • Integration with

adaptation services and multimodal infrastructures

  • Complex test

cases

slide-26
SLIDE 26
  • G. Hübsch, T. Springer – Dresden University of Technology

26

Q & A