Haptic Drum Performance Mya Sithu, Pingguo Huang, and Yutaka - - PowerPoint PPT Presentation
Influence of Network Delay in Networked Haptic Drum Performance Mya Sithu, Pingguo Huang, and Yutaka Ishibashi Department of Scientific and Engineering Simulation Nagoya Institute of Technology, Japan Jan. 24, 2013
Music performance Arts Telerobotics Remote education Medical Surgery
Networked haptic virtual environments provide a user an illusion of sharing a 3D virtual space with other users who are in geographically separated areas. A teacher can instruct a student at a remote place how to play musical instruments by controlling the student’s haptic interface devices by his/her own devices. Two users at different places can also play musical instruments together by using their independent devices.
force, and the other song is done without it.
a 3D virtual space where there exists one drum component.
*1 K. Huang et.al., Proc. IEEE ISWC, Sept. 2008. *2 I. Arimoto et.al., Proc. IEEE HAVE, Oct. 2005.
The system is stand-alone and does not handle networked environments. There is no haptic interaction between the users.
tambourine according to the keyboard harmonica played by the
*3 Q. Zeng et.al., Record of 2012 Tokai-Section Joint Conference on Electrical and Related Engineering,
In virtual environments as well as in the real environment, such a kind of examination is needed.
synchronization quality of sound, we carried out subjective QoE assessments for the two types of usage.
The network delay influences the operability of haptic interface devices.
The network delay makes large differences in output timing of sounds between their terminals.
Network Terminal Terminal Display
PC PC PC PC Headset Headset PHANToM Omni PHANToM Omni Switching hub Switching hub User 1 (Teacher) User 2 (Student) User 1’s drumstick High-hat cymbals Snare drum User 2’s drumstick Floor tom Bass drum
The teacher controls the student’s drumsticks. Passive mode Active mode The student is passively taught by the teacher. The student plays the drum set, and teacher evaluates his/her ability. In both modes, as the network delay increases, the reaction force becomes larger. Two users play the drum set together with the same rhythm at the same tempo. The synchronization quality of sound is seriously deteriorated when the network delay is large.
A user can feel reaction force through his/her haptic interface
A sound depending on the component is also generated.
Network emulator (NIST Net) Terminal Terminal Display
Switching hub PHANToM Omni PHANToM Omni Headset Headset PC PC PC PC
Haptic media are transmitted as media units (MUs), each of which includes position information of PHANToM cursor, user ID, and sequence number.
Switching hub
1 2 3
Rhythm 1 Rhythm 2
High-hat cymbals Snare drum Floor Tom
Each subject hits the same drum component repeatedly. The subject does not need to move his/her drumsticks to the other drum components. He/she needs to move the right drum stick between the snare drum and the floor tom. The distances of the drumstick movements are different between the two rhythms. Subjects played the rhythms at 60 bpm and 100 bpm. These tempos are used in fast ballads and slow rock music, respectively. We used a metronome to play the drum set at two tempos precisely. The two rhythms are used in 8 beats or 4 beats rhythms and popular in today jazz and rock music.
Rhythm 1 Rhythm 2 Slow Tempo Fast Tempo Slow Tempo Fast Tempo 0 to 200 ms Constant Delay (random order) 0 to 100 ms At intervals of 20 ms Subjects Number of subjects : 20 Age : between 20 and 28 Gender : male and female
Each subject behaves as the student and one of the authors took the
role as the teacher.
The passive mode was carried out before the active mode. Each subject carried out several tests and it took one minute for each
test.
He/she needs to hit the drum set repeatedly within each test. He/she was asked to judge about the easiness of hitting.
Score Description 5 Imperceptible 4 Perceptible , but not annoying 3 Slightly annoying 2 Annoying 1 Very annoying We obtain Mean Opinion Score (MOS). Five-grade Impairment Scale
A pair of subjects (users 1 and 2) played the drum set with their
drumsticks independently.
The pair of subjects tried to get the exact synchronization of sound. Each subject was asked to judge about the synchronization quality of
sound. Score Description 5 Excellent 4 Good 3 Fair 2 Poor 1 Bad We obtain Mean Opinion Score (MOS). Five-grade Quality Scale
We introduced a networked haptic drum system. We presented two types of usage (remote teaching and
We carried out subjective QoE assessment by using two
Human perception characteristics are dependent on the
Study QoS control to improve QoE especially by taking
Carry out subjective QoE assessment to investigate the