SLIDE 1
What is a shamanic interface?
Application with gestural interfaces normally use a mimicking approach. (e.g. dancing games in Kinect)
2
An Approach of a Research Tool based in a Shamanic Interface What - - PowerPoint PPT Presentation
An Approach of a Research Tool based in a Shamanic Interface What - - PowerPoint PPT Presentation
An Approach of a Research Tool based in a Shamanic Interface What is a shamanic interface? Application with gestural interfaces normally use a mimicking approach. (e.g. dancing games in Kinect) 2 What is a shamanic interface? Shamanic
SLIDE 2
SLIDE 3
What is a shamanic interface?
Shamanic interface concept was proposed in 2014 based in the science fiction novel FreedomTM.
"It’s called the shamanic interface because it was designed to be comprehensible to all people on earth, regardless of technological level or cultural background."
3
SLIDE 4
What is a shamanic interface?
The idea is creating a gestural interface that can be used by users with different cultures, through meaningful gestures, like cultural gestures (emblems).
4
SLIDE 5
What is a shamanic interface?
5
SLIDE 6
Motivation
Natural User Interfaces suggest a natural interaction. With globalization, the same interface is used by users with different cultures. Different cultures implies different gestures.
6
SLIDE 7
Problem
There is not any research tool that allows researchers to perform empirical evaluations and validations on the cultural awareness in gestural commands
7
SLIDE 8
Goals
Create an application based on a cultural gestural interface, as a tool for empirical research on the shamanic interface concept.
8
SLIDE 9
Background Leap Motion SDK & Accord.NET Framework
9
SLIDE 10
Background - Leap Motion SDK
10
SLIDE 11
Background - Leap Motion SDK
Frame Hands Hand ... Hand ... Position Direction Fingers ...
11
SLIDE 12
Background - Accord.NET Framework
It provides methods and algorithms through different libraries on diverse scientific computing techniques of computer vision and artificial intelligence such as machine learning, mathematics, statistics, computer vision and computer audition.
12
SLIDE 13
Approach
13
SLIDE 14
Application based on a SI
A game that use commands that explore the different gestures between cultures. Maze game with gestures as: selecting, indicate a number, moving, rotating, drink, grab, mute/unmute, pause/resume, quit.
14
SLIDE 15
Approach - Gesture Detection
The capture of the hand’s user is made through a Leap Motion Controller.
15
SLIDE 16
Approach - Gesture Detection
6 features (18 values) - direction of all 5 fingertips + normal vector to the palm
16
SLIDE 17
Approach - Creation of models
Each gesture is represented by an HMM. Using the Accord.NET Framework it is possible to initialize, teach, save and load HMMs.
17
SLIDE 18
Approach - Record lists of sequences
In order to teach the HMMs, samples of each gesture must be stored with a recorder of gestures.
18
SLIDE 19
Approach - Gesture classification
Through the Accord.NET Framework it is possible to create a HMM classifier to evaluate between several HMMs the most promising model to create a given sequence (gesture).
19
SLIDE 20
Approach - Cultural layer
The cultural layer is the main core on a SI. It is responsible for create the relation between the gesture and the command, based on the user’s culture.
20
SLIDE 21
Approach - Cultural layer
Previous approach: (Gesture, Culture) ↓ Command New approach: (Command, Culture) ↓ Gesture
21
SLIDE 22
Approach - Cultural layer
N = 6 P(x) = 1/6 N = 20 P(x) = 1/20
22
SLIDE 23
Approach - Implementation in the game
The SI is implemented in the game through:
- A Cultural Layer responsible for store all the
association between commands, gesture names and respective cultures.
- A map between gesture name and HMM
- A variable to store the user’s culture
23
SLIDE 24
Approach - Implementation in the game
24
SLIDE 25
Implementation
The work is divided in three different targets Recorder SI Library Game
25
SLIDE 26
Implementation - Prototype 0
Game’s structure and the game’s mechanics.
26
SLIDE 27
Implementation - Prototype 1
- SI library;
- Recorder;
- Record and
implemented some gestures (moving and rotating)
27
SLIDE 28
Implementation - Prototype 2
Finished to record and implement all gestures. New feature: normal vector to the palm.
28
SLIDE 29
Implementation - Prototype 2.5
New feature: Variation in the hand’s position. Result: static gestures when performed were misrecognized by dynamic gestures.
29
SLIDE 30
Implementation - Prototype 3
Added generic set of gestures, based on default gestures available in Leap Motion.
30
SLIDE 31
Playability tests with users
Divided in three different groups, 4 users each, to test the game’s playability with all set of gestures:
- A group with 4 Portugueses
- A group with 4 Dutch
- A group with a Dutch, a Belgium, an Indian
and a Brazilian
31
SLIDE 32
Playability tests with users
32
SLIDE 33
Conclusions
✔ An application as a tool for empirical research on the shamanic interface concept. ! The application can still be improved.
33
SLIDE 34
Future work
- Perform some empirical research in the SI
concept;
- Test other methods in the approach used;
- Try different paths in the SI concept, such as
identify the culture of a user based on the gestures.
34
SLIDE 35