A NOVEL MUSIC-BASED GAME WITH MOTION CAPTURE TO SUPPORT COGNITIVE AND MOTOR FUNCTION IN THE ELDERLY PROF. KAT AGRES, NATIONAL UNIVERSITY OF SINGAPORE PROF. SIMON LUI, & TENCENT MUSIC PROF. DORIEN HERREMANS, SINGAPORE UNIVERSITY OF TECHNOLOGY AND DESIGN IEEE CONFERENCE ON GAMES, LONDON, 2019
BACKGROUND TECHNOLOGY FOR THE ELDERLY
Our aging population • North America and Europe: > 1 in 5 people is > 60 years in 2017 (UN, 2017) • Globally, the number of people >80 yrs is expected to increase more than 3 times by 2050, rising from 137 million in 2017 to 425 million • High rates of population aged 65 and above include Japan (26.3%), Italy (22.4%), Germany (21.2%), France (19.1%), and the United Kingdom (17.8%) • Decreasing old-age support ratio => increasingly large burden on already-stressed health care systems across the globe.
What can we do about this? • What strategic measures can be taken in preparation? How can technology play a central role in preventive medicine? And how can we support the elderly in maintaining their motor and cognitive functions? • Preventive medicine, e.g. music-based eHealth technologies => Prototype music system to improve cognitive and physical health and wellness for the elderly .
Existing work • Since 2008: steady rise of games for health care, but few for older adults. • Games for improving cognition in the elderly : • Meta review 20 studies (1986-2013) (Toril, 2014) : positive effect on multiple cognitive functions such as memory, reaction time (RT), attention, and global cognition. • Mostly existing games , e.g Luminosity (Brain games), real-time strategy games etc. • Games for preventing physical decline : • Physical exercises for motor rehabilitation in stroke patients (Chang, 2013; Agres, 2017) • Balance rehabilitation (Lange, 2011) • Improving balance in the elderly (Lai, 2013)
Integrating music • Music therapy : effective in supporting both cognition and mental health: • Improve a variety of cognitive functions , including executive control (Thaut, 2009) , attention (Kim, 2008) , verbal recall (Gfeller, 1983; Wolfe, 1993) , memory (Prickett, 1991; Simmons, 2010; Irish, 2006). • Practicing music improve nonverbal memory and executive processing (Hanna 2011) • Musical interventions significantly decrease anxiety & depression in the elderly (Thaut 2009) • In-person music therapy is expensive (vs. eHealth tech.) • Music can motivate people and increase compliance => We integrate music in our game to help prevent motor & cognitive decline in the elderly.
PROTOTYPE GAME
Game play • The user hears a melody comprised of a series of four excerpts • Each excerpt played by a different instrument • When a particular instrument plays, the cartoon character holding that instrument on the screen moves, to ensure that the user always knows which instrument is playing. • Objective : remember the correct order of instruments, and indicate that order by using the appropriate series of gestures. • Recall memory tasks has been shown to contribute to the maintenance and improvement of cognitive functioning in at-risk elderly people (Ngandu, 2015)
Smart gesture detection • Kinect sensor with the Microsoft SDK & Unity • Shown to parallel the functionality of a full multi-camera 3D motion system in clinical settings (Clark, 2012) • Rule based model to detect if correct instrument is being mimicked. • Why these 4 instruments? They require: 1. Processing and remembering how each instrument is played 2. Bilateral movements that require coordination (e.g. bringing hands together for playing trumpet) 3. Each instrument: distinct movements from the user.
Smart gesture detection
System architecture
Demo
Customized music • Music was composed to fit our requirements • Four two-bar long phrases, each played by a different instrument • The order of instruments is randomly varied across musical sequences, and every instrument occurs exactly once in the sequence.
Conclusions We have created a prototype serious game for the elderly that aims to: 1. leverage the health-related aspects of music 2 . increase motivation to remain physically active in old age 3 . encourage the elderly to make coordinated, bilateral upper limb exercises 4 . support cognitive function (interactive memory task) The prototype is part of a larger suite of games for eHealth that includes games for strengthening and rehabilitation (Agres & Herremans, 2017; Beveridge, Cano, & Agres, 2018). Future work: personalised gesture detection, clinical trial, art design
SUTD Game Lab Focus on serious games in the field of AI, education, healthcare, and AR/ VR technologies. The lab delivers commercially ready systems through collaboration with industry and faculty. • Job opening: senior Unity developer • Always looking for research collaborations in AI, healthcare, VR games • gamelab.sutd.edu.sg
A NOVEL MUSIC-BASED GAME WITH MOTION CAPTURE TO SUPPORT COGNITIVE AND MOTOR FUNCTION IN THE ELDERLY PROF. KAT AGRES, PROF. SIMON LUI, & PROF. DORIEN HERREMANS IEEE CONFERENCE ON GAMES, LONDON, 2019 DORIENHERREMANS.COM
Recommend
More recommend
