MidiLens An Augmented Reality Piano Learning System using HoloLens - - PowerPoint PPT Presentation

▶

Feb 14, 2023 15 likes •185 views

MidiLens An Augmented Reality Piano Learning System using HoloLens Course: Interaction Engineering (WS_17/18) Ruben Schlagowski 954174 Course Instructor: Prof. Dr. Michael Kipp Martin Stadlmaier 954084 Submission: 02. February 2018 2

SLIDE 1

MidiLens

An Augmented Reality Piano Learning System using HoloLens

Ruben Schlagowski 954174 Martin Stadlmaier 954084 Course: Interaction Engineering (WS_17/18) Course Instructor:

Prof. Dr. Michael Kipp

Submission:

02. February 2018

SLIDE 2

SLIDE 3

[Motivation]

Simplify musical training
Make it more effective and fun
Motivate more people to start playing music

SLIDE 4

[Idea] Using Augmented Reality on a Head Mounted Display (HMD) for music learning on a Piano!

SLIDE 5

[Related Work]

SLIDE 6

SLIDE 7

[Interaction Techniques]

Song Selection: Mouse, Keyboard (PC)
Musical Input: Classical Keyboard

(Master-Keyboard)

HMD Application Control:

Speech, Gaze & Gesture (HoloLens)

Visual Output: Augmented Reality on OST-

Displays (HoloLens)

Auditory Output: Speakers (PC)

SLIDE 8

[Technical Realisation]

(Unity/C#)

(Windows Forms, C#)

SLIDE 9

Learning Mode A Learning Mode B

SLIDE 10

[User Tests]

SLIDE 11

Song Playthroughs

Introduction Song: 1 Mode: A Song: 1 Mode: B Song: 2 Mode: A Song: 2 Mode: B Interview

[Test Sequence]

SLIDE 12

[Results]

Prefered Mode?

Potential for excessive use?

B A

Yes No Maybe

55,6% 44,4% 22,2% 33,3%

SLIDE 13

[Live Demo]

SLIDE 14

[Conclusion]

Generally positive feedback (fun factor!)
Prefered Learning Mode: A
Positive effect on skill development

questionable

tracking precision must improve

SLIDE 15

[Future Work]

Gamification via Score and rising difficulty
Improve Reliability with Midi over TCP/IP
More user studies regarding timing/precision
Add finger positioning

SLIDE 16