ROBOT CONTROL USING LIVING ROBOT CONTROL USING LIVING NEURONAL CELLS: - - PowerPoint PPT Presentation

ROBOT CONTROL USING LIVING ROBOT CONTROL USING LIVING NEURONAL CELLS: PROGRESS AND CHALLENGES

Vi M B Victor M. Becerra School of Systems Engineering

Whitehead Lecture Series

G ld i h M h Goldsmiths, 09 March 2011

Introduction Introduction

Background Background

Th bi l i l b i b l

The biological brain can be seen as a complex

computational platform b d d h b d

Progress is being made towards hybrid

systems that integrate biological neurons and l t i t electronic components.

Detached brains Detached brains

A group from NW University

g p y interfaced the detached brain

• f a lampray with a mobile

robot (Reger et al 2000) robot (Reger et al, 2000)

Robot light sensors provide

stimulation to the brain. stimulation to the brain.

Wheel motors activated by

signals derived from neural activity.

Showed stable behaviours,

and adaptation to changes in and adaptation to changes in sensory input

Living creatures Living creatures

A group from NY State University remotely

ll d h f l l controlled the motion of a rat (Talwar et al, 2002)

Direct stimulation of

the rat’s brain using l t d electrodes

Rat trained to interpret

stimuli as cues stimuli as cues.

Rewards given as

separate electrical separate electrical stimuli.

Source: http://www.nature.com/nature/journal/v417/n6884/full/417037a.html

Issues Issues

Detached whole brains are difficult to keep

alive alive.

Interfaces to the brain in living beings are

problematic due to barriers (e.g. skin, skull) problematic due to barriers (e.g. skin, skull)

They may be invasive and destructive,

leading to potential ethical problems. g p p

Data interpretation is difficult given the large

number of neurons.

Data collection usually restricted to few

regions of the brain.

Cultured neurons Cultured neurons

Neurons cultured in laboratory conditions and

interfaced using a planar multi electrode array interfaced using a planar multi‐electrode array (MEA)

Non‐invasive approach

• as e app oac

Allows probing the operation of biological

neuronal networks

Allows measuring activity in the whole structure

The cultures The cultures

Created by dissociating

t k f ti l neurons taken from cortical tissue from foetal rodents using enzymes. g y

Cultured in a special

chamber and provided with bl l suitable environmental conditions and nutrients.

Spontaneous reconnection Spontaneous reconnection

to nearby neurons in a short period.

Importance of this type of Importance of this type of study

Better understanding of interaction between

the brain and external devices the brain and external devices.

Understanding the relations between

neuronal activity and behaviour is critical for neuronal activity and behaviour is critical for dealing with neurological disorders.

Re embodiments (real or virtual) may help Re‐embodiments (real or virtual) may help

the study of biological learning mechanisms.

Related work – simulated Related work simulated robot

D M

t l ( ) d Shk l ik ( )

DeMarse et al (2001) and Shkolnik (2003)

interfaced a neuronal culture with a simulated mobile robot simulated mobile robot

A MEA was employed in both cases Patterns of the electrical activity of the

Patterns of the electrical activity of the network were interpreted as robot commands.

DeMarse and co‐workers provided electrical

stimulation based on information from the i l d i simulated environment

Related work – aircraft Related work aircraft simulator

DeMarse and Dockendorf DeMarse and Dockendorf

(2005) interfaced a neuronal culture with a simulated aircraft simulated aircraft

A MEA was employed The weights of the network

were manipulated through were manipulated through stimulation.

The living network was

d t t l it h d used to control pitch and roll of the simulated aircraft.

Source: http://neural.bme.ufl.edu/page12/page1/page1.html

The work at Reading The work at Reading

EPSRC f di

EPSRC funding 2007‐2010 Collaboration between Systems Engineering

d h and Pharmacy

Four academics One RA and three PhD students

The work at Reading The work at Reading

H h i Di b di d bi l i l k

Hypothesis: Disembodied biological networks

must develop within a closed loop sensory interaction with the environment The loop interaction with the environment. The loop may be closed with a robot. S t d b t di th t h th t

Supported by studies that show that

development in a sensory deprived environment results in dysfunctional neural environment results in dysfunctional neural circuitry

Experimental setup ‐ MEA Experimental setup MEA

The MEA enables voltage to be recorded at

59 out of 64 electrodes.

Detection area 100 μm around each

electrode, which have a radius of 30 μm

Sampling frequency of 25 kHz

p g q y 5

Software allows to separate the firing of

small groups of neurons from an electrode

The same electrodes can do stimulation

e sa e e ect odes ca do st u at o

A picture of the global activity of the

network can be formed

Culture population order around 104‐105

Culture population order around 10 10 neurons

The measured neural activity The measured neural activity

Linux based open source MEAbench software

Stimulation Stimulation

Stimulation software developed at Reading Stimulation software developed at Reading

Experimental setup ‐ robot Experimental setup robot

T d i

Two motor‐driven

wheels S

Sonar sensors to

detect obstacles

Wireless

communications

Programmable on‐

board processor

Experimental setup: Experimental setup: processing

Signal processing can be broken into two

sections: sections:

Culture‐to‐robot: neuronal activity is procesed

and features of interest are mapped into robot pp commands

Robot‐ to‐culture: the robot sensor readings are

mapped into stimulus to the culture

Experimental setup: Experimental setup: Hardware

H d i h MEA

Head stage connecting to the MEA 60 channel amplifier PC data acquisition card Stimulus generator Workstation acquires and processes neural

data

PC runs robot control software Miabot robot platform

Miabot robot platform

Experimental setup Experimental setup

Experimental setup‐ detail Experimental setup detail

Experiments: Experiments: Input‐output pair selection

S i bl l h h

Suitable neuronal pathways were sought.

Biphasic pulses were applied with a magnitude of 600 mV 100 μs each phase repeated 16 times 600 mV, 100 μs each phase, repeated 16 times.

Input‐output pairs were defined as follows:

Activity measured in electrode “B” in response to Activity measured in electrode B in response to

stimulus in electrode “A”, within 100 ms

Electrode “B” responds more than 60% of the

Electrode B responds more than 60% of the time to electrode “A”, and less than 20% to stimulation in other electrodes.

Experiments: standard control Experiments: standard control

Experiments: culture in the loop Experiments: culture in the loop

Experiments Experiments

Sonar measured distance to the wall

When the robot approaches a wall, the distance (green line)

decreases below a threshold (30 cm), triggering a stimulation pulse pulse.

Significant activity events at the output electrode are

translated into a 90 degree turn (rotation starts on the yellow line and stops on the red line) line, and stops on the red line).

Some random turns were registered due to the network

spontaneous activity

Experiments: statistics Experiments: statistics

6 % i l i

67% stimulation – response event Run time: 140 s Total closed loop time: 0.2‐0.5 s Meaningful turns: 46% (s.d. 15%) Spontaneous turns: 54% (s.d. 19%)

Studying the culture Studying the culture

I d b d d h i i f

In order to better understand the activity of

the neural culture and its evolution over time (which should help to improve the closed loop (which should help to improve the closed loop scheme), we have done the following work:

Modelling the network states transition patterns Modelling the network states transition patterns Investigating the functional connectivity of the

network and its evolution over time network and its evolution over time

Modelling the network state Modelling the network state transition patterns

W h d Hidd M k M d l hi h

We have used Hidden Markov Models, which

are probabilistic models of state transitions. d l d d h l

Models trained on MEA data with stimulus Network discrete ‘observables’ were defined

• n the basis of the presence of a spike on a

given channel during a short time interval (0.1 ) ms).

Modelling the network state Modelling the network state transition patterns

Modelling the network state Modelling the network state transition patterns

State flow diagram of state transitions

Modelling the network state Modelling the network state transition patterns

Including knowledge of culture’s dynamic

state and how that changes may allow for i h l f l l i i i l d tighter control of neuronal plasticity in closed loop environments.

HMM’s are capable of uncovering hidden HMM’s are capable of uncovering hidden

state sequences which can provide detailed knowledge of the culture’s activity and even knowledge of the culture s activity and even the prediction of its future behaviour.

The identified state of the network may be

y used as an means to determine robot actions.

Functional connectivity Functional connectivity

Th f

ti l t k f lt d

The functional networks of cultured neurons

exhibit similar properties to in‐vivo networks

Cultures have not pre built connectivity and Cultures have not pre‐built connectivity, and

they undergo reorganization over time

Little is know about how a complex

Little is know about how a complex functional network evolves from isolated neurons

This knowledge may inform the choice of

stimulation and output electrodes in a closed l h loop scheme.

Functional connectivity Functional connectivity

Evolution of functional connectivity estimated Evolution of functional connectivity estimated

from correlations of spontaneous activity

Young cultures (14 days) exhibited a random

Young cultures (14 days) exhibited a random topology

This evolved into a small world topology

g

Increased network efficiency and presence of

hubs with age

The small network topology balances integration

• f areas with segregation of specialised

processing processing

Functional connectivity Functional connectivity

The graph illustrates the spatial organization

g p p g

• f the network with nodes corresponding to

MEA electrodes

Functional connectivity Functional connectivity

The graph illustrates the location of hubs in a The graph illustrates the location of hubs in a

representative network

Functional connectivity Functional connectivity

Burst propagation time as a function of

p p g culture age, which is measured as the time taken to recruit all channels in a network wide burst

Challenges and open questions Challenges and open questions

R bili f i l l

Repeatability of experimental results Development of culture training protocols Quantification of learning Survivability of cultures over the required

length of experiments

Choice of effective mappings between

pp g

network activity and robot commands robot sensing and network stimulation

Conclusions Conclusions

D ib d h bl i i d

Described the problem, motivation and

background. b d b l

Described our progress on robot control

involving cultured neuronal networks

Discussed some of the challenges and open

questions

There is much work to be done Emphasis is on studying the culture in a

closed loop, rather than on any practical use.

Acknowledgements Acknowledgements

Sl i N

Slawomir Nasuto Kevin Warwick Ben Whalley Dimitris Xydas Julia Downes Mark Hammond

Mark Hammond