Enhancing Human Synchronization through Embodied Digital - PowerPoint PPT Presentation

EuroMov Enhancing Human Synchronization through Embodied Digital Architectures: From basic science to innovation Benoît G. Bardy EuroMov Montpellier University, France www.euromov.eu

Between agents Human – agent interactions and mental health (Bardy et al., 2014; Slovinski et al., 2016) Between music and behavior Music-induced synchronisation of Sync biological rhythms (Bardy et al., 2015; Dotov et al., 2019) Between the senses Energy-1 Specification across energy arrays (Mantel et al., 2015) Energy-2 3 - g y e r n E

B E T W E E N T H E S E N S E S Stoffregen & Bardy, 2001; Mantel et al., 2015 Sync Energy-1 The global array Energy-2 Energy-3 SPATIAL ORIENTATION DURING FLIGHT

RESEARCH ARTICLE Exploratory Movement Generates Higher- C H A L L E N G E S Order Information That Is Sufficient for Accurate Perception of Scaled Egocentric Distance q Modelling and testing the global array in Bruno Mantel 1,2,3 , Thomas A. Stoffregen 4 , Alain Campbell 2,5 , Benoît G. Bardy 1,6 * natural/specific situations q Transport: q Road: Cycling, biking, driving q Air: Flying (Liners, Helicopters) q Rail: Tram, train q Motion sickness and see leg acquisition q Walking dynamics q Phobia, vertigo, disorientation q Etc.

B E T W E E N M U S I C & B E H A V I O R Sync Bardy et. al., 2015; Dotov et al., 2019; Cochen et al. 2018 %FR&mode MECHANICS NEUROPHYS Couplage Music stabilization of Running and of Locomotion-Respiratory Coupling (LRC)

B E T W E E N M U S I C & M O V E M E N T Sync Bardy et. al., 2015; Dotov et al., 2016; Hoffmann et al., 2015, Dotov et al., 2019 q Manipulation of variability of RAS [isochronous, random, 1/f] q Manipulation of adaptability [strong, weak] q Multicentric clinical trial in progress (300 patients) – SATT AxLR

F R O M B A S I C S C I E N C E q State-of-the-art on music-movement synchronization q Technological development (app + sensors+Sync algorithms) q Experiments during Running/Walking in various populations q Stabilizes walking, increases cadence/efficiency, strengthens LRC T O I N N O V A T I O N ( B E A T R U N ) Beat Run: Outdoor/Treadmill Cruise Control: Run TrailMix Pro q Asics (RunKeeper) q Nike+ q UnderArmour (Endomondo) TempoRun PaceDJ Upbeat Workouts for Runners q Adidas (Runtastic) q Spotify (running) q The SYNC project ! Existing apps Market/Players Potential partners With: Simone Dalla Bella & Loic Damm ( EuroMov ), Valérie Cochen ( Montp Hospital ), Marc Leman & Bart Moens ( Univ. Ghent ), Rudi Willing & Tomas Ward ( Univ. Maynooth ), Erik Sanchez ( Tecnalia )

B E T W E E N H U M A N S A N D A G E N T S Sync Bardy et al., 2014; Slowinski et al., 2016; Gueugnon et al., 2016; Zhong et al., 2015 Create new human - artificial agent interactions through the concept of similarity in order to enhance social competence in patients suffering from social deficits

Similarity, Entrainment, Synchronization, and Affiliation q Similarity increases synchronization q Synchronization increases cooperation

Similarity and Synchronization - In physics -> two systems possessing the same frequency tend to synchronize better (Huygens, 1666) - In physiology -> two biological systems possessing the same intrinsic properties tend to synchronize (von Holst, 1939, Bramble & Carrier, 2003) - In motor control -> two individuals coordinate better when they exchange more similar information (Schmidt & Richardson, 2008; Varlet et al., 2012)

- In social psychology -> people tend to like more someone that reproduces the same behavioral components (such as mimicry) as their owns (Chameleon effect; Chartrand & Bargh, 1999) - In social neuroscience -> Motor Resonance (Gallese, 2003), Mirror Neurons (Rizzolatti et al., 1996) – Motor theories of perception – Theory of mind, etc. - In social robotics -> Uncanny valley (e.g., Ishiguro, 2005) and Learning by demonstration (e.g., Billard et al., 2008)

Synchronization and affiliation q Social entrainment: Reciprocal relation between movement entrainment and social feelings (e.g., Tomasello, 2009; Kleinspehn, 2008) q Joint action develops social connection (e.g., Marsh et al., 2009; Sebanz et al., 2006). q Acting in synchrony with others increases affiliation and cooperation (e.g., Wiltermuth & Heath, 2009)

From Similarity… Morphological Temporal Behavioural To Difference ! q Similarity triggers social interaction q A bridge toward interacting with others

Evaluating similarity in a social perceptuo-coordination context The mirror game task ( e.g. Noy et al., 2011 ) Between humans, humans and artificial agents Manipulation of several parameters, mental, social, physical, etc.

Manipulating Morphology (Physical resemblance)

Manipulating movement similarity (kinematics) 1.8 (a) position [m] 0.9 0 10 20 30 40 50 2.52 (b) velocity [m/s] 0 − 2.52 10 20 30 40 50 time [s] 2.52 (c) (d) velocity [m/s] velocity [m/s] 0 − 2.52 4 8 12 16 frequency time [s]

PDF : Probability Density Functions of players’ velocity EMD : Earth Mover’s Distance Similar (0) Different (1)

Individual Motor Signature (IMS) and the similarity space Solo Solo against VP x: (first dimension of MDS): Average velocity y: (second dimension of MDS): Kurtosis

Leader 1 Leader 2 Joint Improv § IMS Dyad 1 § Morpho. Similarity § Movement Similarity § Social Role (L / F / JI) § Behavioural plasticity Dyad 2 § Personality traits § Social Penetrability § Attraction § Pathologies Dyad 3 § Rehabilitation § Etc. Slowinski et al., 2016

The AlterEgo cognitive architecture

Similarity space for patients vs. controls (a) 0.59 Position [a.u.] 0.03 Controls Patients -1.15 (b) (c) 4 2 Frequency [Hz] 0 y 1 0.5 0.25 0.125 0.0625 0 1 | φ r (f)| -0.03 (d) (e) -0.08 0 0.08 0.1 x φ r (t) 0 -0.1 0 10 20 30 40 50 Probability Time [s] ( ) : distribution of absolute phase lag over frequencies φ r f 22 schizophrenic patients vs. 22 healthy controls § § ICub leader / Subject follower Slowinski et al., submitted

F R O M B A S I C S C I E N C E q Push SoA: Synchronization, Similarity, Pathology, IMS q Technological development (cognitive architecture, rendering) q Experiments in controls and patients (Sz, ASD, Social. Phob.) q Enhance Synchronization and affiliation q Pre-clinical trial (monocentric) in progress T O I N N O V A T I O N q Therapy as usual q Schizophrenia q Individual Therapy q ASD q Group Therapy q Social phobia q Etc. ---------------------------- q Cerebral Palsy q Stroke q Parkinson q etc Multiple platforms Multiple populations Multiple Scenarios With: Ludovic Marin ( EuroMov ), Delphine Capdevielle & Stéphane Raffard ( Montp Hospital ), Aude Billard ( EPFL ), Mario Di Bernardo ( Univ. Bristol ), Krasi Tsaneva ( Univ. Exeter )

Movement Sciences Virtual Reality Simone Dalla Bella Thank you ! José Henriques Loic Damm Norbert Schmitz Dobri Dotov Didier Stricker Jérémie Landrieu Music Cognition Mathieu Gueugnon Marc Leman Petra Ihalainen Robotics Bart Moens Ludovic Marin Aude Billard Robin Salesse Madhi Khoramshahi Zhong Zhao Psychology Laura Cohen Bruno Mantel Richard Schmidt Kinesiology Mathematical modeling Psychiatry Tom Stoffregen Francesco Alderisio Mario Di Bernardo Catherine Bortolon Exploitation Piotr Slowinski Delphine Capdevielle Krasimira Tsaneva Erik Hernandez Stephane Raffard Chao Zhai Engineering Aeronautics Rudi Villing Jean-Christophe Sarrazin Tomas Ward Valérie Juppet / Charlotte Heulin