My Research - The Problem: Computer Science Some people with Severe - - PDF document

▶
my research the problem
SMART_READER_LITE
LIVE PREVIEW

My Research - The Problem: Computer Science Some people with Severe - - PDF document

May 25, 2023 17 likes •300 views

9/11/2012 Adaptable Interfaces for People with Disabilities John Magee Boston University / Clark University Computer Science Computer Science My Research - The Problem: Computer Science Some people with Severe cerebral palsy Traumatic

slide-1
SLIDE 1

9/11/2012 1

Computer Science Computer Science

Adaptable Interfaces for People with Disabilities

John Magee

Boston University / Clark University

Computer Science

My Research - The Problem:

Some people with

  • Severe cerebral palsy
  • Traumatic brain injury
  • Stroke
  • Multiple sclerosis (MS)
  • ALS (Lou Gehrig’s)

cannot communicate with traditional means:

  • Nonverbal
  • Limited voluntary motion

Lack of communication ability ≠ lack of active minds! Communication technology: not available, or too expensive,

inefficient, difficult, complex, etc.

slide-2
SLIDE 2

9/11/2012 2

Computer Science

Talk Outline

  • Observations and user studies with people with

disabilities.

  • HAIL – Hierarchical Adaptive Interface Layout
  • Adaptive Mouse Control Functions
  • Study of automatic generation of online social

networking messages.

Computer Science

Team - Collaborators

Margrit Betke at Boston University Jim Gips at Boston College Many others, undergraduates and grad students:

  • O. Al-Hinai, W. Akram, M. Chau, R. Cloud, C. Connor, S.

Crampton, S. Epstein, C. Fagiani, I. Fedyuk, P. Fleming, M. Gorman, O. Gusyatin, K. Grauman, W.-B. Kim, C. Kwan, J. Lombardi, E. Missimer, W. Mullally, L. Tiberii, M. Urinson, M. Scott, M. Shugrina, B. Waber, E. Yu, Z. Wu Users

  • The Boston Home, Dorchester, MA
  • Children’s Hospital, Communication Enhancement Center, Boston, MA
  • Campus School, Boston College
  • LGF Public Middle School, Boston, MA
  • McLaughlin Center, Connecticut
  • Hollybank Home, Yorkshire, UK
  • Several schools in Eastern Massachusetts & Northern Ireland
slide-3
SLIDE 3

9/11/2012 3

Computer Science

Traditional Approaches

  • Touch Switches
  • Hit plate
  • Wobble stick
  • Grip handle
  • Pinch
  • Pull string
  • Photocell Switches
  • Sip or Puff Switches
  • Voice activated

Switches Binary Switch (blue button)

Computer Science

The Camera Mouse

  • Camera-based tracking of

body to enable control of a mouse pointer

  • Mouse replacement

system: hands free, uses dwell time for clicking

  • Typical audience:

Nonverbal users, people with quadriplegia, people with paralysis, etc

User with ALS

Free Download – http://www.cameramouse.org Over 500,000 downloads

slide-4
SLIDE 4

9/11/2012 4

Computer Science

The Camera Mouse

Computer Science

Camera Mouse

slide-5
SLIDE 5

9/11/2012 5

Computer Science

Observations

We’ve used educational programs, text entry, games, image editing, web browser…

  • Every user is different, different abilities

 strong need to customize software (more than people without disabilities).

  • Some programs only work for people who can

control pointer precisely.

Our study included 12 people with moderate to severe motor disabilities.

Computer Science

Applications

slide-6
SLIDE 6

9/11/2012 6

Computer Science

Two-level Text Entry

Computer Science

Block Escape Game

slide-7
SLIDE 7

9/11/2012 7

Computer Science

Some Lessons Learned

Designing Human-Computer Interface:

  • Applications need large, strategically placed areas for buttons to

reduce the problem of false selection.

  • “Rest” areas needed (“Midas Touch Problem”)

Using Human-Computer Interface:

  • Choice of tracking method should depend on application used.
  • Nose is most reliable feature for non-disabled users; allows fast and

smooth motion.

  • Need more customizability.

Assistive technology can be social! Challenging to evaluate systems with users with severe disabilities.

  • Limited session lengths. Inability to repeat many times. High variability .

Computer Science

Different Users

Adults with stable conditions Learning curve, rehabilitation. Children Learning movement abilities. Adults with degenerative conditions: Movement ability may decrease.

Should we design an individual interface for each user’s ability at a specific time?

slide-8
SLIDE 8

9/11/2012 8

Computer Science

Talk Outline

 Observations and user studies with people with disabilities.

  • HAIL – Hierarchical Adaptive Interface Layout
  • Adaptive Mouse Control Functions
  • Study of automatic generation of online social

networking messages.

Computer Science

HAIL

Need to have hierarchical solution – allow interaction in a more principled way: The application should change and adapt to the user, rather than require the user to adapt to the program.

slide-9
SLIDE 9

9/11/2012 9

Computer Science

Web browsing

Important application for accessing information and communication: But…

  • Complex layout
  • Interactive elements mixed with content
  • Too many buttons, accidental activations
  • Small links are hard to click

Computer Science

Interface Idea

The configuration of the interface remains a manual setting.

slide-10
SLIDE 10

9/11/2012 10

Computer Science

HAILtwitter

Computer Science

HAILbrowser

General purpose web browser application.

  • Basic features:
  • Select Next/Previous Link.
  • Follow Selected Link.
  • Back/Forward.
  • Page Up/Down.
  • Advanced features:
  • Bookmarks.
  • Text Entry / Keyboard.
  • Jump 10 Links.
  • Smart navigation.
  • Default to mobile web (?)

HAILbrowser prototype

slide-11
SLIDE 11

9/11/2012 11

Computer Science

Web 2.0

Web pages provide both information and presentation:

html Browser View

Specialized browsers can reinterpret, but the presentation is decided by the provider. Web 2.0: Database driven web

  • applications. User generated

content.

data Application User’s View

Consumer decides how to present the information.

Access data through APIs.

Computer Science

HAILtwitter

  • Specific-purpose application client for Twitter.
  • The web browser’s buttons navigate the interactive

elements on a web page..

  • A specific application client can instead directly control

interaction elements with the user interface.

  • Basic features:
  • Get more / Refresh.
  • Page Up/Down.
  • Select Next/Previous Msg.
  • Retweet (forward msg).
  • Advanced features:
  • See other timelines
  • Text Entry / Keyboard.
  • Trending topics.
  • Follow/Unfollow friends.
slide-12
SLIDE 12

9/11/2012 12

Computer Science

HAIL – Lessons Learned

  • Size of buttons was not all that important. Edges create infinite width.
  • Some difficulty using dwell time to select buttons.

Task: Select left-top, right- top, and bottom-center buttons.

Computer Science

Current and Future Work

  • Add features to current prototype applications.
  • Continue to test applications with users with disabilities.
  • Incorporate further feedback.
  • Make applications publicly available.
  • Extend to other applications:
  • Facebook, Google+
  • RSS Reader.
  • Email Client.
  • SMS Messaging.
  • Media/Video/Photo Center.
  • Model abilities for automatic adaptation.
slide-13
SLIDE 13

9/11/2012 13

Computer Science

HAIL Conclusions

All users are different, and their abilities may change over time:

  • Interfaces should adapt to the user’s abilities,

rather than forcing them to conform to a program.

HAIL-based applications can address some of the problems faced with traditional programs.

  • Small buttons, “midas touch”, different users,

changing abilities.

Designed for Camera Mouse, but could be used with other mouse replacements.

Computer Science

Talk Outline

 Observations and user studies with people with disabilities.  HAIL – Hierarchical Adaptive Interface Layout

  • Adaptive Mouse Control Functions
  • Study of automatic generation of online social

networking messages.

slide-14
SLIDE 14

9/11/2012 14

Computer Science

User Capability Assumptions

  • Cannot assume people with disabilities will

be able to comfortably move their heads in a certain way.

Observed in user studies.

Computer Science

User Capability Assumptions

Also observed in interface tests:

  • Mouse target acquisition tests
  • Move the mouse to a highlighted target
  • Once reached, a new target is highlighted
  • Horizontal movements were difficult
  • Diagonal movements were easier
  • Small adjustments easier in diagonal motion

Plots by: Emily Yu

slide-15
SLIDE 15

9/11/2012 15

Computer Science

User Study Observation

Computer Science

Adaptive Interfaces: The application should change and adapt to the user, rather than require the user to adapt to the program.

slide-16
SLIDE 16

9/11/2012 16

Computer Science

Non-Adaptive System

Computer Science

Adaptive System

slide-17
SLIDE 17

9/11/2012 17

Computer Science

Mouse Motion Functions

Computer Science

Experimental Setup

slide-18
SLIDE 18

9/11/2012 18

Computer Science

Adaptive Function Result

Computer Science

Click Tester

Users moves mouse pointer to highlighted circle. Same sequence used for all subjects. 4 people with CP participated.

slide-19
SLIDE 19

9/11/2012 19

Computer Science

Interface Tests

Camera Mouse

Computer Science

Case Studies

User with CP who likes to lay on his couch: “It feels more natural”

slide-20
SLIDE 20

9/11/2012 20

Computer Science

Case Studies

3 Middle School Students All four were able to complete both EagleAliens and Clicktester tasks with and without adaptive mouse function.

Computer Science

Other modalities: Stereo Thermal Video

slide-21
SLIDE 21

9/11/2012 21

Computer Science

Towards a Multi-Camera Camera Mouse

How can we improve the Camera Mouse?

  • We would like to address:
  • Lost Tracking
  • Improved mouse pointer response

A Multi-Camera System could help:

  • Redundant input avoids single point of tracking failure.
  • Stereoscopic information may improve reliability.
  • 3D Feature tracking may give better pointer response.

Computer Science

Feature Motion

slide-22
SLIDE 22

9/11/2012 22

Computer Science

Trajectory Analysis

Computer Science

Feature Trajectory

slide-23
SLIDE 23

9/11/2012 23

Computer Science

3D Feature Movement

Computer Science

Feature Movement

  • Total movement: 18.5cm
  • Moves 4.6cm away from left camera, and

then 3.0cm back toward the camera.

  • This component of motion cannot be

accounted for with single-camera system.

Towards the multi-camera system:

  • Redundant input avoids single point of tracking failure.
  • Stereoscopic information may improve reliability.
  • 3D Feature tracking may give better pointer response.

Future: Microsoft Kinect sensor.

slide-24
SLIDE 24

9/11/2012 24

Computer Science

Talk Outline

 Observations and user studies with people with disabilities.  HAIL – Hierarchical Adaptive Interface Layout  Adaptive Mouse Control Functions

  • Enhancing Social Connections Through

Automatically-Generated Online Social Network Messages

Computer Science

Camera Mouse in the classroom: Can we foster the same social engagement

  • nline?

“I saw [T] using it and thought ‘this looks like fun’ so I wanted to try it”.

slide-25
SLIDE 25

9/11/2012 25

Computer Science

System Flowchart

Computer Science

slide-26
SLIDE 26

9/11/2012 26

Computer Science Computer Science

We asked 4 users about this type of message.

slide-27
SLIDE 27

9/11/2012 27

Computer Science

Case Studies

“I think that would be cool, because then they would know how much work I did.” “...she would know what I got, and I would know what she got. . . we might have something in common.” Positive Responses: Users would want to share with family, friends, and teachers:

3 Users - How likely are you to use this feature: 4.6 on 5 point Likert scale.

Computer Science

Case Studies

“I would like it”. But… Expressed concerns about privacy and what kind of messages he would like to share.

slide-28
SLIDE 28

9/11/2012 28

Computer Science

Conclusions

Studying interfaces for people with disabilities is both challenging and rewarding.

  • The challenge of creating individualized

interfaces becomes an opportunity to create adaptive interfaces.

Much more work still to be done!

  • Big challenge to go from the lab to the home or

school.

  • Small-scale projects, such as creating a game,

can have a big impact.

Computer Science

Acknowledgements

Funding by the National Science Foundation (IIS- 0713229, IIS-0093367, IIS-0308213, IIS-039009, P200A01031, and EIA-0202067), and the Office of Naval Research (N000140110444). Thank you to all the people who participate in our research studies!

Thank you! Questions?