MERGING AR AND PIANO EDUCATION: AN APPLICATION DESIGNED AROUND PIANO - PowerPoint PPT Presentation

Presentation 1 MERGING AR AND PIANO EDUCATION: AN APPLICATION DESIGNED AROUND PIANO INSTRUCTORS AND ASSISTED TEACHING Amare re Birha hanu nu Stefan Rank Paul Diefenbach & Youngmoo Kim

Research Question

Research Question 1. I will describe the process of creating an AR application for the HoloLens with the purpose of it being integrated into piano lessons as a tool instructors can use to teach their students

Research Question 1. I will describe the process of creating an AR application for the HoloLens with the purpose of it being integrated into piano lessons as a tool instructors can use to teach their students 2. because I want to uncover design choices, features, and visuals that support student learning based on the opinion and expertise of experienced piano instructors,

Research Question 1. I will describe the process of creating an AR application for the HoloLens with the purpose of it being integrated into piano lessons as a tool instructors can use to teach their students 2. because I want to uncover design choices, features, and visuals that support student learning based on the opinion and expertise of experienced piano instructors, 3. in order to create a basic standard for AR piano training application intended to be integrated into formal piano lessons.

Contribution – What I Hope to Uncover

Contribution – What I Hope to Uncover ■ General set of guidelines for AR piano applications based on expert opinions.

Contribution – What I Hope to Uncover ■ General set of guidelines for AR piano applications based on expert opinions. ■ Teacher preferences, dislikes, and concerns.

Contribution – What I Hope to Uncover ■ General set of guidelines for AR piano applications based on expert opinions. ■ Teacher preferences, dislikes, and concerns. ■ Minimum requirements for creating an application intended to be used by piano instructors who intend on incorporating AR technology in their lessons.

Contribution – What I Hope to Uncover ■ General set of guidelines for AR piano applications based on expert opinions. ■ Teacher preferences, dislikes, and concerns. ■ Minimum requirements for creating an application intended to be used by piano instructors who intend on incorporating AR technology in their lessons. ■ Unexplored and experimental ways of providing augmented visual support to the student.

Common Attributes & Features – as seen in previous examples Hand Reference Floating Sheet Music Hand Reference Light up Keys Falling Notes Falling Notes Mainly Improvisation Song Selection Song Selection

LATEST RESEARCH Week 01 – Week 03

The Cognitive Theory of Multimedia Learning ■ Theory centers around the idea that learners build meaningful connections between words and pictures. – Meaningful learning happens when the learner engages in cognitive process such as using relevant words or images to organize a mental model and integrate those representations with prior or new knowledge. ■ To be effective it must be designed to allow for an adequate period of usage. – Actively constructing new knowledge. ■ Design must be learner-centered and not technology centered. – Understand learner limitations. Sorden, Stephen D. "The Cognitive Theory of Multimedia Learning." (n.d.): n. pag. Web. Apr. 2017.

Transfer Learning ■ Storing knowledge gained while solving one problem then applying that new knowledge to a different yet relatable problem. – Source Task Target Task ■ Has happened if a task has led to improvements in another task. – Target task is new to the user but requires similar skills as the source task. – Improve learning in target task by leveraging knowledge in the source task. ■ Goal is to produce positive transfer between two appropriately related tasks while avoiding negative transfer between tasks that are less related. – Dependent on relationship between tasks. – Strong relationship in commonalities will result in a stronger transfer of knowledge. Torrey, Lisa, and Jude Shavlik. "Transfer Learning." Handbook of Research on Machine Learning Applications (2009): n. pag. IGI Global . Web.

AR in Education: A Meta-Review & Cross-Media Analysis ■ Compares 26 publications that have compared AR versus non-AR learning methods. – Provides a list of positive and negative impacts of AR experiences on learning. ■ Tend to show significantly faster speed in locating important information. – Show significantly less head movements. – Faster task completions with Less Errors. ■ Learning detriments that could potentially come from AR: – Tunneling: High attentional demand on non-essential parts of the experience. – Usability: May be effective for learning but are commonly difficult to use. Radu, Iuli. "Augmented reality in education: a meta-review and cross-media analysis" Springer-Verlag London, 2014. Web. Apr. 2017.

AR in Education: A Meta-Review & Cross-Media Analysis ■ Systems can monitor user activity and provide relevant feedback. – Bringing into context progression that has been made. – Reducing the need for the learner to switch their attention between sources. ■ Users that are presented content through AR have stronger memories of their experiences. – Physical immersion and learning from tactile information. ■ Digital content can be used to guide user’s attention. – Good for helping to complete a tasks. – Utilizes visual perception of the spatial relationships of objects in the user’s environment. Radu, Iuli. "Augmented reality in education: a meta-review and cross-media analysis" Springer-Verlag London, 2014. Web. Apr. 2017.

AR in Education: A Meta-Review & Cross-Media Analysis ■ A heuristic questionnaire constructed from the above study. 1. The application transforms the problem representation such that difficult concepts are easier to understand. 2. The application presents relevant educational information at the appropriate time and place, providing easy access to information and/or reducing extraneous learner tasks. 3. The application directs learner attention to important aspects of the educational experience. 4. The application enables learners to physically enact, or to feel physically immersed in, the educational concepts. 5. The application permits students to interact with spatially challenging phenomena. Radu, Iuli. "Augmented reality in education: a meta-review and cross-media analysis" Springer-Verlag London, 2014. Web. Apr. 2017.

Affordances and Limitations of Immersive Participatory for AR simulations for Teaching and Learning ■ Documents how teachers and students participating in AR simulations can aid or hinder the teaching or learning process. ■ Students develop patterns of participation that shapes their identities as learners. – The way in which the engage in learning. – What the student believes their ability to learn is. ■ AR has a unique capacity to activate, recruit, and cultivate a sense of projective identity that serves as a mediation between the students’ real world identity and their virtual or game identity. – Virtual identities it can then leverage to influence and shape an ongoing transformation of real-world identities (Gee 2003). Dunleavy, Matt, Chris Dede, and Rebecca Mitchell. "Affordances and Limitations of Immersive Participatory Augmented Reality Simulations for Teaching and Learning." Journal of Science Education and Technology 18.1 (2008): 7-22. Web.

The Role of Perceived Enjoyment in the Students Acceptance of an AR Teaching Platform: A Structural Equation Modelling Approach ■ Investigate the user’s acceptance to understand the factors that influence a user’s intention to use AR technology during learning experiences. – The TAM model (perceived ease of use, perceived usefulness, and perceived enjoyment) determine one’s behavioral intention to use a technology. ■ AR devices tend to be harder to use than more traditional learning tools but, – Provide a greater intrinsic motivation because of novelty and versatility associated with new technology. ■ Perceived enjoyment (intrinsic motivational factor) and perceived usefulness (extrinsic motivational factor ) are important to understand user intentions. Balog, Alexandru , and Costin Pribeanu . "The Role of Perceived Enjoyment in the Students’ Acceptance of an Augmented Reality Teaching Platform: a Structural Equation Modelling Approach." Studies in Informatics and Control 19.3 (2010): 319-30. Web. Apr. 2017.

KeyboARd "KeyboARd." Devpost. N.p., n.d. Web. 24 Apr. 2017. <https://devpost.com/software/keyboard>. https://github.com/camelKaiser/KeyboARd

Options for MIDI/Real-Time Pitch Detection Tutorial/Source Code Unity Asset ■ Using Microphone Input in Unity3D ■ Koreographer - $36 / Pro $116 – Kappine.fi – MIDI Converter ■ Fundamental Frequencies and ■ Pitch Detector - $15 Detecting Notes – Real-Time Pitch Detection – Kappine.fi ■ Pitch Detection for Music – StackOverflow.com

PROJECT UPDATE Spring Break – Week 03