Enhancing Human Synchronization through Embodied Digital - - PowerPoint PPT Presentation

Enhancing Human Synchronization through Embodied Digital Architectures:

From basic science to innovation

Benoît G. Bardy

EuroMov Montpellier University, France www.euromov.eu

EuroMov

Between agents

Human – agent interactions and mental health (Bardy et al., 2014; Slovinski et al., 2016)

Between music and behavior

Music-induced synchronisation of biological rhythms (Bardy et al., 2015; Dotov et al., 2019)

Between the senses

Specification across energy arrays (Mantel et al., 2015)

Sync

Energy-1 E n e r g y

• 3

Energy-2

Energy-1 Energy-3 Energy-2

The global array

Sync

B E T W E E N T H E S E N S E S

Stoffregen & Bardy, 2001; Mantel et al., 2015

SPATIAL ORIENTATION DURING FLIGHT

C H A L L E N G E S

q Modelling and testing the global array in natural/specific situations q Transport: q Road: Cycling, biking, driving q Air: Flying (Liners, Helicopters) q Rail: Tram, train q Motion sickness and see leg acquisition q Walking dynamics q Phobia, vertigo, disorientation q Etc.

RESEARCH ARTICLE

Exploratory Movement Generates Higher- Order Information That Is Sufficient for Accurate Perception of Scaled Egocentric Distance

Bruno Mantel1,2,3, Thomas A. Stoffregen4, Alain Campbell2,5, Benoît G. Bardy1,6*

B E T W E E N M U S I C & B E H A V I O R

Bardy et. al., 2015; Dotov et al., 2019; Cochen et al. 2018

Music stabilization of Running and of Locomotion-Respiratory Coupling (LRC)

Sync

Couplage %FR&mode

MECHANICS NEUROPHYS

B E T W E E N M U S I C & M O V E M E N T

Bardy et. al., 2015; Dotov et al., 2016; Hoffmann et al., 2015, Dotov et al., 2019

Sync

q Manipulation of variability of RAS [isochronous, random, 1/f] q Manipulation of adaptability [strong, weak] q Multicentric clinical trial in progress (300 patients) – SATT AxLR

F R O M B A S I C S C I E N C E

q State-of-the-art on music-movement synchronization q Technological development (app + sensors+Sync algorithms) q Experiments during Running/Walking in various populations q Stabilizes walking, increases cadence/efficiency, strengthens LRC

With: Simone Dalla Bella & Loic Damm (EuroMov), Valérie Cochen (Montp Hospital), Marc Leman & Bart Moens (Univ. Ghent), Rudi Willing & Tomas Ward (Univ. Maynooth), Erik Sanchez (Tecnalia)

Existing apps

T O I N N O V A T I O N ( B E A TR U N)

Beat Run: Outdoor/Treadmill Cruise Control: Run TrailMix Pro TempoRun PaceDJ Upbeat Workouts for Runners !

Market/Players Potential partners q Asics (RunKeeper) q Nike+ q UnderArmour (Endomondo) q Adidas (Runtastic) q Spotify (running) q The SYNC project

B E T W E E N H U M A N S A N D A G E N T S

Bardy et al., 2014; Slowinski et al., 2016; Gueugnon et al., 2016; Zhong et al., 2015

Create new human - artificial agent interactions through the concept of similarity in order to enhance social competence in patients suffering from social deficits

Sync

Similarity, Entrainment, Synchronization, and Affiliation

q Similarity increases synchronization q Synchronization increases cooperation

Similarity and Synchronization

• In physics
• > two systems possessing the same frequency tend

to synchronize better (Huygens, 1666)

• In physiology
• > two biological systems possessing the same

intrinsic properties tend to synchronize (von Holst,

1939, Bramble & Carrier, 2003)

• In motor control
• > two individuals coordinate better when they

exchange more similar information (Schmidt & Richardson,

2008; Varlet et al., 2012)

• In social psychology
• > people tend to like more someone that

reproduces the same behavioral components (such as mimicry) as their owns (Chameleon effect; Chartrand &

Bargh, 1999)

• In social neuroscience
• > Motor Resonance (Gallese, 2003), Mirror Neurons

(Rizzolatti et al., 1996) – Motor theories of perception –

Theory of mind, etc.

• In social robotics
• > Uncanny valley (e.g., Ishiguro, 2005) and Learning by

demonstration (e.g., Billard et al., 2008)

Synchronization and affiliation

q Social entrainment: Reciprocal relation between movement entrainment and social feelings

(e.g., Tomasello, 2009; Kleinspehn, 2008)

q Joint action develops social connection

(e.g., Marsh et al., 2009; Sebanz et al., 2006).

q Acting in synchrony with others increases affiliation and cooperation

(e.g., Wiltermuth & Heath, 2009)

From Similarity…

Morphological Behavioural Temporal

To Difference !

q Similarity triggers social interaction q A bridge toward interacting with others

Evaluating similarity in a social perceptuo-coordination context

Between humans, humans and artificial agents Manipulation of several parameters, mental, social, physical, etc.

The mirror game task (e.g. Noy et al., 2011)

Manipulating Morphology (Physical resemblance)

Manipulating movement similarity (kinematics)

10 20 30 40 50 0.9 1.8 position [m] (a) 10 20 30 40 50 −2.52 2.52 time [s] velocity [m/s] (b) 4 8 12 16 −2.52 2.52 time [s] velocity [m/s] (c) frequency velocity [m/s] (d)

PDF : Probability Density Functions of players’ velocity EMD : Earth Mover’s Distance

Similar (0) Different (1)

Individual Motor Signature (IMS) and the similarity space

x: (first dimension of MDS): Average velocity y: (second dimension of MDS): Kurtosis

Solo Solo against VP

Slowinski et al., 2016

§ IMS § Morpho. Similarity § Movement Similarity § Social Role (L / F / JI) § Behavioural plasticity § Personality traits § Social Penetrability § Attraction § Pathologies § Rehabilitation § Etc.

Leader 1 Leader 2 Joint Improv Dyad 1 Dyad 2 Dyad 3

The AlterEgo cognitive architecture

Similarity space for patients vs. controls

Slowinski et al., submitted

§ 22 schizophrenic patients vs. 22 healthy controls § ICub leader / Subject follower

• 1.15

0.59

Position [a.u.]

(a) (b) 4 2 1 0.5 0.25 0.125 0.0625

Frequency [Hz]

(c) 1

|φr(f)|

10 20 30 40 50

Time [s]

• 0.1

0.1

φr(t)

(d)

Probability

(e)

: distribution of absolute phase lag over frequencies

φr f

( )

• 0.08

0.08

x

• 0.03

0.03

y

Controls Patients

F R O M B A S I C S C I E N C E

With: Ludovic Marin (EuroMov), Delphine Capdevielle & Stéphane Raffard (Montp Hospital), Aude Billard (EPFL), Mario Di Bernardo (Univ. Bristol), Krasi Tsaneva (Univ. Exeter)

Multiple platforms

T O I N N O V A T I O N

Multiple populations Multiple Scenarios q Therapy as usual q Individual Therapy q Group Therapy q Etc. q Push SoA: Synchronization, Similarity, Pathology, IMS q Technological development (cognitive architecture, rendering) q Experiments in controls and patients (Sz, ASD, Social. Phob.) q Enhance Synchronization and affiliation q Pre-clinical trial (monocentric) in progress q Schizophrenia q ASD q Social phobia

• q Cerebral Palsy

q Stroke q Parkinson q etc

Thank you !

Virtual Reality

José Henriques Norbert Schmitz Didier Stricker

Movement Sciences

Simone Dalla Bella Loic Damm Dobri Dotov Jérémie Landrieu Mathieu Gueugnon Petra Ihalainen Ludovic Marin Robin Salesse Zhong Zhao Bruno Mantel

Mathematical modeling

Francesco Alderisio Mario Di Bernardo Piotr Slowinski Krasimira Tsaneva Chao Zhai

Robotics

Aude Billard Madhi Khoramshahi Laura Cohen

Psychiatry

Catherine Bortolon Delphine Capdevielle Stephane Raffard

Psychology

Richard Schmidt

Kinesiology

Tom Stoffregen

Music Cognition

Marc Leman Bart Moens

Exploitation

Erik Hernandez

Engineering

Rudi Villing Tomas Ward

Aeronautics

Jean-Christophe Sarrazin Valérie Juppet / Charlotte Heulin