Employing Multimodal Interactions to Ensure Transparent - - PowerPoint PPT Presentation

IRIT - University of Toulouse

World Class Standards

Employing Multimodal Interactions to Ensure Transparent Authentication in Pervasive Environments

Dana Al Kukhun Florence Sèdes

Multimodal Interaction with Mobile Devices

IRIT - University of Toulouse 2

World Class Standards

Plan

Introduction Pervasive Environments Security Challenges in Pervasive Environments Current Authentication Mechanisms The need for transparent interactions for mobility Some directions towards transparency Conclusion

IRIT - University of Toulouse 3

World Class Standards

Introduction

Ubiquitous / Pervasive Computing:

Weiser’s vision for the computing future in the 21st century:

• Computing elements will disappear from user’s consciousness while

functioning homogeneously with his environment.

Pervasive Computing:

Mobile User Interacting with a system Using a mobile device

A Quality of Service challenge:

How can we enable the user to interact seamlessly with a system at :

• Anytime
• Anyhow different devices, connectivity channels, etc.
• Anywhere

IRIT - University of Toulouse 4

World Class Standards

Pervasive computing

For hardware and computing elements to disappear

Software needs to disappear. The spatial and temporal relationships between people and objects (HCI) has to be well defined in the early design phase

Pervasive information systems

Are interactive user-Centered systems That aim to facilitate user interaction within unfamiliar environments.

Objective:

Transparent services Proactivity (trying to anticipate the user needs) Efficiency, usability & user satisfaction

IRIT - University of Toulouse 5

World Class Standards

Challenges in Pervasive Environments

Data Hardware Software

Usability Vs Automation Accessibility Vs Security Heterogeneity Vs Connectivity

Adaptation

IRIT - University of Toulouse 6

World Class Standards

Importance of Multimodal Interaction

Would allow a greater portion of users to access a system:

Old aged people Disabled people

• Handicapped, blind, …
• Sick patients (robotics, medicine taking, calling for an emergency, etc)

Would allow easier interaction

Mobile devices have small screens, small keybords, etc

Would allow users to be more functional & productive

Multi functioning Faster interaction (voice commands, etc.)

IRIT - University of Toulouse 7

World Class Standards Mobile Devices: some Limitations & some future directions for solutions

Low battery Capacities

Multimodal interaction can consume the battery power .

• Waiting for increased battery power for higher efficiency.
• Research works are looking for means to employ the surrounding

environment to generate power (solar power, physical power, etc.)

Not all machines are fully equipped to apply successful multimodal interactions

cameras for Iris recognition need high resolution. Challenges in voice recognition for commands in different languages & in noisy environments (mobile users).

The cost of equipped devices is high

High technology tends to be for the previleged society

• Looking for cheap & good quality machines for all society portions

IRIT - University of Toulouse 8

World Class Standards

Accessibility Vs Security

How to provide users with high accessibility while guaranteeing the system’s security? Security requires frequent re-authentication, but this would limit usability. Security

Authentication Access Control Cryptography

IRIT - University of Toulouse 9

World Class Standards

Current authentication Mechanisms

Password (textual input)

Bank kiosks, credit card shopping (french system), email

Digital Signature

Credit card shopping (American system) text recognition

Voice

Vocal input voice recognition

Digital finger print, Iris

Biometric & Image recognition

Machine Tokens

RFID chips, smart cards, …

IRIT - University of Toulouse 10

World Class Standards

Some Works on Multimodal Authentication

• 1. MUST: Multimodal User Authentication for Secured Transaction

Objective: to investigate effective methods for the recognition of people over the internet based on voice and facial characteristics in

• rder to facilitate, protect and promote various financial and other

services over this growing telecommunication medium.

• Speaker verification
• Face localization and verification
• taking into consideration lighting effects and several conditions.
• Fusion of modalities for multimodal authentication.

European Project SER C01.0120 (2002 – 2005)

IRIT - University of Toulouse 11

World Class Standards

2. A Multimodal Authentication System for Authorizing the Access to NGN Services

• Context : New generation networks
• Many new services , Offered by different players (service providers).
• Each service might have it’s own authentication method and use different

credentials.

Need for more reliable, flexible and easy-to-use methods.

• Proposition: Separating application-specific credentials (e.g. smart cards)

hosted in the network, from personal identification (e.g. Smart cards).

• After the biometric identification is carried out, the contactless smart card

allows to sense the user presence, in order to unlock his/her workstation and to recover from a server, using a secure protocol

• The mechanism strength is guaranteed by limiting in time and space the

validity of the logical association between biometric identification and card

• wnership

Melen et al., “A Multimodal Authentication System for Authorizing the Access to NGN Services”, International conference on Networking and Services, ICNS’06. Page: 47

IRIT - University of Toulouse 12

World Class Standards

Authentication @ the Pervasive age

The ultimate goal for pervasive computing is achieving invisibility. For ensuring security and privacy in those environments,

Both human beings and surrounding devices should be authenticated under the interaction methods that are used for ubiquitous services. But there exist many challenges as:

• The invisibility of devices,
• the adaptiveness of interactions,
• and the varying performance of devices

That will make it difficult to achieve invisibility.

IRIT - University of Toulouse 13

World Class Standards

Camera GPS Microphone Loudspeaker Stylus-Activated Sketch Pad Wearable Compute Server Docking Station for Car PC Biosensor for Authentication & Emotional Feedback GSM for Telephone, Fax, Internet Connectivity

Directions to Efficient Solutions: 1) The SmartCom Project - The Handheld Assistant 2001

IRIT - University of Toulouse 14

World Class Standards

Smartcard/ Credit Card for authentication and billing Docking station for PDA/Notebook/ Camcorder high speed and broad bandwidth Internet connectivity High-resolution scanner Loudspeaker Room microphone Face-tracking camera Virtual touchscreen protected against vandalism Multipoint video conferencing

The SmartCom Project - The Public Kiosk

IRIT - University of Toulouse 15

World Class Standards

Some Applications

Augmented Reality: Combining Speech, Gestures and Graphics for Mobile Web Access

1. Mobile Dialog with a Virtual Tourist Guide 2. Location-adaptive Query Interpretation 3. Multimodal Route Description 4. Mobile Speech Translation & Multilingual Information Access 5. Speech-based Access to 3D Virtual Views 6. Multimodal Output from a Digital Library and Speech-based Access to Internet Content

IRIT - University of Toulouse 16

World Class Standards

2) The Secure Phone Project

A technical solution that improves security is needed Solution joining

Biometric authentication

• Makes use of VOICE, FACE and SIGNATURE

Multi-modal authentication Mobile-device based authentication Electronic signature

to secure information exchange Privacy??

Protected by storing info on a secure SIM module

The use of multiple modaliaties would enhance security

IRIT - University of Toulouse 17

World Class Standards

Secure Phone

Strong authentication anywhere, to access databases, deal transactions, business operations, etc… Necessary security:

Gartner: more than 250.000 PDA are stolen and lost yearly in USA airports

http://www.secure-phone.info

Voice F a c e W r i t t e n S i g n a t u r e

IRIT - University of Toulouse 18

World Class Standards

Fortifying Security by joining modalities

Pre-processing Modelling Modelling

Modelling

Pre-processing Pre-processing Access Denied Access Granted

FUSION

Face Voice Written Signature

Modelling

IRIT - University of Toulouse 19

World Class Standards

Some Applications

Corporate:

Secure transactions between sales force and headquarters Secure transactions with suppliers

Commercial:

Mobile electronic banking Mobile private banking Mobile payments, on-line reservations

Governments and Public Administration:

Administrative transactions through mobile phone Police force communications Mobile medical services

IRIT - University of Toulouse 20

World Class Standards New Technologies for more transparency in authentication and interaction

Phidget Sensors:

Touch Sensor RFID Tags Finger print reader And other multimodal authentication techniques.

Are suitable for mobile users and to enforce more security and Are

important aspects for transparency & interactivity with pervasive environments