Supporting the Construction of Context-Aware Applications Anind - - PowerPoint PPT Presentation

▶
supporting the construction of context aware applications
SMART_READER_LITE
LIVE PREVIEW

Supporting the Construction of Context-Aware Applications Anind - - PowerPoint PPT Presentation

Jan 12, 2024 120 likes •1.11k views

Supporting the Construction of Context-Aware Applications Anind K. Dey Intel Berkeley Lab Intel Research How UbiComp is Different from Traditional GUI Not only direct interaction for input and output Need to use RANGE of explicit

slide-1
SLIDE 1

Supporting the Construction of Context-Aware Applications

Anind K. Dey

Intel Berkeley Lab Intel Research

slide-2
SLIDE 2

9/17/2001 3

How UbiComp is Different from Traditional GUI

  • Not only direct interaction for input

and output

  • Need to use RANGE of explicit AND

implicit interaction

  • Different computing paradigm
slide-3
SLIDE 3

9/17/2001 4

Context and Context-Awareness

  • Focused on input
  • Context: any information relevant to an

interaction that can be used to characterize the situation of an entity

  • Context-awareness
  • General model of interactive computing
  • Addresses subset of ubicomp problems: input
slide-4
SLIDE 4

9/17/2001 5

Value of Context

  • Potential for improved usability
  • Very important for mobile users with

poor input devices

  • “Smarter” applications
  • Increased communications bandwidth
slide-5
SLIDE 5

9/17/2001 6

Design Space for Context- Aware Applications

  • Toolkit allows exploration of design space
  • Basic types of context:
  • Location, identity, time, activity
  • Simple/singular complex/multiple
  • Combinations
  • Uses of context:
  • Present to user
  • Automatically perform set of services
  • Tag captured information to ease retrieval
slide-6
SLIDE 6

9/17/2001 7

Example

  • Tour guides, travel assistants,

personalization software

  • Reminder to buy milk
  • When to deliver: not time/location

specific

  • How to deliver: appropriate modality
slide-7
SLIDE 7

9/17/2001 8

Outline

  • Motivation
  • Problems dealing with context
  • Contribution: Context Toolkit
  • Validation:
  • Design space and applications
  • Building more realistic applications
  • Conclusions and future work
slide-8
SLIDE 8

9/17/2001 9

Building Applications

  • M. Weiser: The whole point of

ubiquitous computing, of course, is the applications.

slide-9
SLIDE 9

9/17/2001 9

Building Applications

  • M. Weiser: The whole point of

ubiquitous computing, of course, is the applications.

  • But … what if the applications are

hard to build? And, what if this inhibits our ability to build compelling applications?

slide-10
SLIDE 10

9/17/2001 10

Issues in Context-Awareness

  • What is context?
  • Representation of context
  • Application domains
  • Which behaviors to support
  • When to execute behaviors
  • Privacy, Quality of Service, …
  • Evaluation of applications
  • make it easier to build explore
slide-11
SLIDE 11

9/17/2001 11

Why Context is Hard to Use

  • Acquired from sensors
  • Not just keyboards and mice – lots of

heterogeneous devices

  • Need to abstract data
  • Distributed
  • Dynamic
slide-12
SLIDE 12

9/17/2001 12

Results of Difficulties

  • Ad hoc application building
  • Difficult to build, reuse and evolve
  • Small variety of sensors
  • Small variety of context: mostly location
  • Few applications, mostly simple: mostly

presenting context

  • Practical: difficult to prototype, test and

evaluate

slide-13
SLIDE 13

9/17/2001 13

Why Applications are Hard to Build: A Case Study

  • Cyberguide case study: no separation
  • f concerns
slide-14
SLIDE 14

9/17/2001 14

Outline

  • Motivation
  • Problems dealing with context
  • Contribution: Context Toolkit
  • Validation:
  • Design space and applications
  • Building more realistic applications
  • Conclusions and future work
slide-15
SLIDE 15

9/17/2001 15

Need Programming Support

  • Goal: make application development easier
  • Identified number of requirements for

architectural support and design process

  • Examined existing support and determined

how developers might think about building context-aware applications

  • Developed Context Toolkit: architecture

with supporting library of components

slide-16
SLIDE 16

9/17/2001 16

Related Work

  • Existing Context-Aware Systems
  • Schilit (Columbia, 1995)
  • Stick-e notes (Pascoe, Kent, 1996)
  • CyberDesk (Dey, Georgia Tech, 1997)
  • CALAIS (Ward, Cambridge, 1998)
  • MUSE (Castro, UCLA, 2000)
  • Context-Awareness SDK (Tangis Corp., 2000)
  • Proposed/Related Systems
  • Situated Computing Service (HP, 1997)
  • Contextual Information Service (Pascoe,Kent, 1998)
  • HIVE (Minar, MIT, 1998)
  • OAA (Cohen, OGI, 1996)
slide-17
SLIDE 17

9/17/2001 17

Research Contributions

  • Conceptual framework requirements
  • Provide framework for designing apps more

easily

  • Lower threshold to enable more designers
  • Context Toolkit
  • Implementation and exploration of design space
  • Support investigation of complex problems

and more realistic apps

  • Raise ceiling
  • Privacy, uncertainty, security, end-user

programming

slide-18
SLIDE 18

9/17/2001 18

Toolkit Requirements

  • Context specification
  • Discovery
  • Storage
  • Separation of concerns
  • Interpretation
  • Transparent communications
  • Constant availability
slide-19
SLIDE 19

9/17/2001 19

Design Process

  • 1. Specification
  • 2. Acquisition
  • 3. Delivery
  • 4. Reception
  • 5. Action
slide-20
SLIDE 20

9/17/2001 19

Design Process

  • 1. Specification
  • 2. Acquisition
  • 3. Delivery
  • 4. Reception
  • 5. Action

1.

Specification

2.

Acquisition

3.

Action

slide-21
SLIDE 21

9/17/2001 20

Time for a Big Insight!

  • Have a design process – complex and

simplified

  • Have a set of architecture

requirements

  • Need to figure out how to support

these

slide-22
SLIDE 22

9/17/2001 21

Look to input handling

  • Graphical User Interface (GUI)

widgets

  • separation of concerns
  • callbacks and attributes
  • query/subscribe
  • common interface
  • e.g. button
slide-23
SLIDE 23

9/17/2001 22

Context Widgets

  • Responsible for acquiring and

abstracting data from particular sensor, separation of concerns, storage

Widget Sensor Widget Application Application Sensor Context Architecture

slide-24
SLIDE 24

9/17/2001 22

Context Widgets

  • Responsible for acquiring and

abstracting data from particular sensor, separation of concerns, storage

Widget Sensor Widget Application Application Sensor Context Architecture Face Recognition Location Widget Location Widget In/Out Board Smart Card Reader

slide-25
SLIDE 25

9/17/2001 23

Context Interpreters

  • Convert or interpret context to

higher level information

  • Context not available at appropriate

level

Face Recognition Location Widget Location Widget In/Out Board Smart Card Reader

slide-26
SLIDE 26

9/17/2001 23

Context Interpreters

  • Convert or interpret context to

higher level information

  • Context not available at appropriate

level

Face Recognition Location Widget Location Widget In/Out Board Smart Card Reader ID to Name Interpreter

slide-27
SLIDE 27

9/17/2001 23

Context Interpreters

  • Convert or interpret context to

higher level information

  • Context not available at appropriate

level

Face Recognition Location Widget Location Widget In/Out Board Smart Card Reader ID to Name Interpreter

slide-28
SLIDE 28

9/17/2001 24

Context Aggregators

  • Collect context relevant to particular

entities (recall definition)

  • Further separation, simplifies design

Face Recognition Location Widget Location Widget In/Out Board Smart Card Reader ID to Name Interpreter

slide-29
SLIDE 29

9/17/2001 24

Context Aggregators

  • Collect context relevant to particular

entities (recall definition)

  • Further separation, simplifies design

Face Recognition Location Widget Location Widget In/Out Board Smart Card Reader ID to Name Interpreter Building Aggregator

slide-30
SLIDE 30

9/17/2001 25

Context Toolkit Framework

  • Supports real-world model/methodology and provides library

(distributed: XML/HTTP, input-focused)

  • Component model: facilitates building of applications

Widget Sensor Widget Application Application Interpreter Interpreter Aggregator Sensor Context Architecture

slide-31
SLIDE 31

9/17/2001 25

Context Toolkit Framework

  • Supports real-world model/methodology and provides library

(distributed: XML/HTTP, input-focused)

  • Component model: facilitates building of applications

Widget Sensor Widget Application Application Interpreter Interpreter Aggregator Sensor Context Architecture Service Actuator

slide-32
SLIDE 32

9/17/2001 25

Context Toolkit Framework

  • Supports real-world model/methodology and provides library

(distributed: XML/HTTP, input-focused)

  • Component model: facilitates building of applications

Widget Sensor Widget Application Application Interpreter Interpreter Aggregator Sensor Context Architecture Service Actuator Discoverer

slide-33
SLIDE 33

9/17/2001 26

Experiences: Benefits

  • Provides separation of concerns
  • Lightweight integration and re-use of

components

  • Easy to create and evolve apps, allowing

exploration of the design space

  • Add context to context-less apps
  • Add more context to context-aware apps
slide-34
SLIDE 34

9/17/2001 27

Outline

  • Motivation
  • Problems dealing with context
  • Contribution: Context Toolkit
  • Validation:
  • Design space and applications
  • Building more realistic applications
  • Conclusions and future work
slide-35
SLIDE 35

9/17/2001 28

Validation

  • Used to build existing applications
  • Used to explore the design space
  • Used to build more complex and

realistic applications

slide-36
SLIDE 36

9/17/2001 29

Additional Validation

  • Facilitating larger community outside of Georgia Tech, including:
  • British Telecom Labs
  • MIT
  • Trinity College
  • PLAY Research Group
  • Stuttgart
  • SICS, Sweden
  • ETH
  • Philips
  • Telenor
  • Nokia
  • Arch:
  • CMU (mobile agents)
  • Motorola (arch/mobile user apps)
  • Autonama de Madrid (arch/smart spaces

apps)

  • Apps
  • Keele (desktop apps)
  • Novator (apps for mobile workers)
  • Technical University Munich/CMU

(informal meeting support)

slide-37
SLIDE 37

9/17/2001 30

Aware Home (MANSE ’99)

  • Great testbed for context-aware

computing

  • 3 goals: elderly, infants, everyone
  • Context Toolkit is the s/w

infrastructure in the Aware Home

slide-38
SLIDE 38

9/17/2001 30

Aware Home (MANSE ’99)

  • Great testbed for context-aware

computing

  • 3 goals: elderly, infants, everyone
  • Context Toolkit is the s/w

infrastructure in the Aware Home

slide-39
SLIDE 39

9/17/2001 31

Design Space for Context- Aware Applications

  • Toolkit allows exploration of design space
  • Types of context:
  • Location, identity, time, activity
  • Simple/singular complex/multiple
  • Combinations
  • Uses of context:
  • Present to user
  • Automatically perform set of services
  • Tag captured information to ease retrieval
slide-40
SLIDE 40

9/17/2001 32

Applications Built

  • Simple use of location:
  • Turn lights on and off (perform service)
  • Location and id (perform service)
  • Information Guide: present info about

user’s group (CHI ’99)

  • Context-Aware Mailing List
slide-41
SLIDE 41

9/17/2001 33

In/Out Board – 3 versions

(CHI ’99)

  • Context used: location, identity, time
  • How used: present context
slide-42
SLIDE 42

9/17/2001 34

In/Out Board Architecture

Location Widget In/Out Board iButton Dock ID to Name Interpreter Web-based In/Out Board Context-Aware Mailing List

  • Simple apps demonstrates support for reusability

(don’t have to re-build infrastructure on per- application basis) and evolving applications

slide-43
SLIDE 43

9/17/2001 34

In/Out Board Architecture

Location Widget In/Out Board iButton Dock ID to Name Interpreter Web-based In/Out Board Context-Aware Mailing List

  • Simple apps demonstrates support for reusability

(don’t have to re-build infrastructure on per- application basis) and evolving applications

PinPoint 3D-iD

slide-44
SLIDE 44

9/17/2001 35

  • Context used: location, id, time, activity
  • How used: present, perform service, tag
  • Work done by others in research group

Serendipitous Meetings

slide-45
SLIDE 45

9/17/2001 35

  • Context used: location, id, time, activity
  • How used: present, perform service, tag
  • Work done by others in research group

Serendipitous Meetings

Playback controls Filters Ink written before current time is in

  • riginal color

Ink written after current time is in

  • riginal color

Current time within session Selected session Selected day Day containing whiteboard activity

slide-46
SLIDE 46

9/17/2001 36

Meeting Architecture

Context Architecture For each possible location

  • f the

mobile board Location Widget DUMMBO Location Widget ID to Name Interpreter iButton Dock iButton Dock

  • Demonstrates support for evolution
  • Use by others
slide-47
SLIDE 47

9/17/2001 37

Conference Assistant (ISWC ’99)

  • Context used: location, multiple levels of identity,

activity, time

  • How used: present, service, tag
slide-48
SLIDE 48

9/17/2001 37

Conference Assistant (ISWC ’99)

  • Context used: location, multiple levels of identity,

activity, time

  • How used: present, service, tag

Slide User Notes Interest Control Audio/Video Indicator

slide-49
SLIDE 49

9/17/2001 37

Conference Assistant (ISWC ’99)

  • Context used: location, multiple levels of identity,

activity, time

  • How used: present, service, tag

Slide User Notes Interest Control Audio/Video Indicator

slide-50
SLIDE 50

9/17/2001 37

Conference Assistant (ISWC ’99)

  • Context used: location, multiple levels of identity,

activity, time

  • How used: present, service, tag

Slide User Notes Interest Control Audio/Video Indicator Slide text User notes Retrieved slide Query Interface Schedule context widgets

Identity, Location, Activity

  • f People, Places, Things

Joe Smith context

slide-51
SLIDE 51

9/17/2001 38

Conference Assistant Arch.

  • Complex application: reuse, evolution
slide-52
SLIDE 52

9/17/2001 39

Outline

  • Motivation
  • Problems dealing with context
  • Contribution: Context Toolkit
  • Validation:
  • Design space and applications
  • Building more realistic applications
  • Conclusions and future work
slide-53
SLIDE 53

9/17/2001 40

Complex and Realistic Applications

  • Privacy: Dynamic Door Displays
  • Ambiguity: In/Out Board extension
  • Security: Service/Context Access

(SACMAT 2001)

  • End-user programming: CybreMinder

(HUC 2000)

slide-54
SLIDE 54

9/17/2001 41

Component Abstraction

X X Situation realization P X Context specification P X Evolution P X Sensor failure P X Sensor addition V X Multiple Apps V X Storage V X Query/subscribe V X Distributed communications P X Context acquisition Component None Abstractions Features Level of Support X – none P – partial V – complete

slide-55
SLIDE 55

9/17/2001 41

Component Abstraction

X X Situation realization P X Context specification P X Evolution P X Sensor failure P X Sensor addition V X Multiple Apps V X Storage V X Query/subscribe V X Distributed communications P X Context acquisition Component None Abstractions Features Level of Support X – none P – partial V – complete X P P P P V V V V P Component

slide-56
SLIDE 56

9/17/2001 42

Situation Abstraction: Declarative Style

  • Revisit context definition
  • Allow programmer to define a situation (real-

world callbacks)

  • Declare what context you want, not how to
  • btain it
  • Architecture’s responsibility to deliver it
  • Makes specification in design process

simpler, more robust, easier to evolve

slide-57
SLIDE 57

9/17/2001 43

Revised Framework

Widget Sensor Widget Application Application Interpreter Interpreter Aggregator Sensor Context Architecture Discoverer

  • Supports blackboard/box model of the world
  • Different than component model

Actuator Service

slide-58
SLIDE 58

9/17/2001 44

Situation Abstraction

X X Situation realization P X Context specification P X Evolution P X Sensor failure P X Sensor addition V X Multiple Apps V X Storage V X Query/subscribe V X Distributed communications P X Context acquisition Component None Abstractions Features

slide-59
SLIDE 59

9/17/2001 44

Situation Abstraction

X X Situation realization P X Context specification P X Evolution P X Sensor failure P X Sensor addition V X Multiple Apps V X Storage V X Query/subscribe V X Distributed communications P X Context acquisition Component None Abstractions Features V V V V V V V V V P Situation

slide-60
SLIDE 60

9/17/2001 45

CybreMinder (HUC ’00)

slide-61
SLIDE 61

9/17/2001 45

CybreMinder (HUC ’00)

slide-62
SLIDE 62

9/17/2001 45

CybreMinder (HUC ’00)

slide-63
SLIDE 63

9/17/2001 46

CybreMinder (HUC ’00)

slide-64
SLIDE 64

9/17/2001 46

CybreMinder (HUC ’00)

slide-65
SLIDE 65

9/17/2001 46

CybreMinder (HUC ’00)

slide-66
SLIDE 66

9/17/2001 46

CybreMinder (HUC ’00)

slide-67
SLIDE 67

9/17/2001 47

CybreMinder (HUC ’00)

slide-68
SLIDE 68

9/17/2001 47

CybreMinder (HUC ’00)

slide-69
SLIDE 69

9/17/2001 47

CybreMinder (HUC ’00)

slide-70
SLIDE 70

9/17/2001 47

CybreMinder (HUC ’00)

slide-71
SLIDE 71

9/17/2001 48

Outline

  • Motivation
  • Problems dealing with context
  • Contribution: Context Toolkit
  • Validation:
  • Design space and applications
  • Building more realistic applications
  • Conclusions and future work
slide-72
SLIDE 72

9/17/2001 49

Research Contributions

  • Conceptual framework requirements
  • Provide framework for designing apps more

easily

  • Lower threshold to enable more designers
  • Context Toolkit
  • Implementation and exploration of design space
  • Support investigation of complex problems

and more realistic apps

  • Raise ceiling
  • Privacy, uncertainty, security, end-user

programming

slide-73
SLIDE 73

9/17/2001 50

What’s Next?

  • Complex interpretation, sensor fusion, dealing with

ambiguity how to infer what a user really wants

  • Ontology, QoS, privacy, security
  • Data models
  • Development environment
  • Evaluation of context-awareness
  • What to do, and when and why
  • Overload/how interruptible is the user
  • End-user control of what happens
  • Broaden scope of framework to be a general model
  • f interactive and ubiquitous computing: look at

implicit output

slide-74
SLIDE 74

9/17/2001 51

Acknowledgements

  • Gregory D. Abowd & FCE
  • Motorola & NSF
  • Contact info:
  • anind@cc.gatech.edu
  • http://www.cc.gatech.edu/~anind
  • http://www.cc.gatech.edu/fce/ctk
  • Questions?
slide-75
SLIDE 75

9/17/2001 52

Intercom System

  • Facilitate communications between family

members:

  • In the same house
  • Between houses
  • While mobile
  • Uses 4 types of context in combination

(primarily present, service, but potential for tagging when learning)

  • Leverage social mediation skills
slide-76
SLIDE 76

9/17/2001 53

Design Process

  • 1. Specification – context and behaviors
  • 2. Acquisition – install, API, query/notify,

store, interpret

  • 3. Delivery – deliver context to multiple, remote

applications

  • 4. Reception – locate relevant sensors, request

context, interpret

  • 5. Action – analysis and action
slide-77
SLIDE 77

9/17/2001 54

How to simplify?

  • Brooks 87: “No Silver Bullet: Essence and

Accidents of Software Engineering”

  • essential problems
  • inherent problems
  • specific to the task at hand
  • accidental problems
  • problems induced by design tools
  • not specific to the task
slide-78
SLIDE 78

9/17/2001 55

Callback Model

  • Transparent communications, always available
  • Similar to GUI architectures

Application Base Object handle () Location Widget Base Object Sensor

slide-79
SLIDE 79

9/17/2001 55

Callback Model

  • Transparent communications, always available
  • Similar to GUI architectures

Application Base Object handle () Location Widget Base Object Sensor (a) subscribe: David in Room 343

slide-80
SLIDE 80

9/17/2001 55

Callback Model

  • Transparent communications, always available
  • Similar to GUI architectures

Application Base Object handle () Location Widget Base Object Sensor (a) subscribe: David in Room 343

slide-81
SLIDE 81

9/17/2001 55

Callback Model

  • Transparent communications, always available
  • Similar to GUI architectures

Application Base Object handle () Location Widget Base Object Sensor (a) subscribe: David in Room 343

slide-82
SLIDE 82

9/17/2001 55

Callback Model

  • Transparent communications, always available
  • Similar to GUI architectures

Application Base Object handle () Location Widget Base Object Sensor (a) subscribe: David in Room 343 (b) sensor data arrives Sensor

slide-83
SLIDE 83

9/17/2001 55

Callback Model

  • Transparent communications, always available
  • Similar to GUI architectures

Application Base Object handle () Location Widget Base Object Sensor (a) subscribe: David in Room 343 (b) sensor data arrives Sensor

slide-84
SLIDE 84

9/17/2001 55

Callback Model

  • Transparent communications, always available
  • Similar to GUI architectures

Application Base Object handle () Location Widget Base Object Sensor (a) subscribe: David in Room 343 (b) sensor data arrives Sensor (c) callback if data matches: David in 343

slide-85
SLIDE 85

9/17/2001 55

Callback Model

  • Transparent communications, always available
  • Similar to GUI architectures

Application Base Object handle () Location Widget Base Object Sensor (a) subscribe: David in Room 343 (b) sensor data arrives Sensor (c) callback if data matches: David in 343 (d) callback delivered to a handler

slide-86
SLIDE 86

9/17/2001 56

Access Control

  • Dynamic Door Displays
slide-87
SLIDE 87

9/17/2001 56

Access Control

  • Dynamic Door Displays
slide-88
SLIDE 88

9/17/2001 57

Ambiguous Context

speakers microphone motion detector keyboard display dock

slide-89
SLIDE 89

9/17/2001 58

Experiences: Limitations

  • Continuously changing context
  • Dealing with unreliable context and
  • ther quality of service issues
  • Component failure
  • Privacy
slide-90
SLIDE 90

9/17/2001 59

Important Distinction

slide-91
SLIDE 91

9/17/2001 59

Important Distinction

  • Behavior that “looks easy” but is not.
  • Star Trek’s doors
  • Real-time classroom control
  • Incremental speech recognition

improvement

slide-92
SLIDE 92

9/17/2001 59

Important Distinction

  • Behavior that “looks easy” but is not.
  • Star Trek’s doors
  • Real-time classroom control
  • Incremental speech recognition

improvement

  • Behavior that “looks hard” but is not.
  • Mobile tour guide (GPS, IR beacons)
  • Temporal synching
slide-93
SLIDE 93

9/17/2001 60

Distribution of Sensing

  • Heterogeneity of platforms and

languages

  • No guarantees on what sensors require
  • No guarantees on what what’s available
  • No guarantees on what developers

prefer

slide-94
SLIDE 94

9/17/2001 61

Abstraction: Interpretation

  • Provide meaning to sensed data
  • Simple converters
  • Complex inferences
slide-95
SLIDE 95

9/17/2001 62

Abstraction: Aggregation

  • Eases interpretation
  • Maps to notion of an entity
  • Efficiency mechanism
slide-96
SLIDE 96

9/17/2001 63

Component Persistence

  • Not like GUI widgets
  • Execute autonomously
  • Always running
slide-97
SLIDE 97

9/17/2001 64

Context History

  • Not like GUI widgets
  • Don’t want to leave to apps
  • Components always running
  • Store data for future apps
slide-98
SLIDE 98

9/17/2001 65

Situations: Declarative Style

  • Say what you want, not how you want it

done – framework figures it out

  • Allow programmer to define a situation

(complex real-world callbacks)

  • Specification in design process simpler
  • More robust w.r.t. component failures and

easier to evolve