CS 378: Autonomous Intelligent Robotics Instructor: Jivko Sinapov - - PowerPoint PPT Presentation

▶
cs 378 autonomous intelligent robotics
SMART_READER_LITE
LIVE PREVIEW

CS 378: Autonomous Intelligent Robotics Instructor: Jivko Sinapov - - PowerPoint PPT Presentation

Sep 20, 2022 48 likes •849 views

CS 378: Autonomous Intelligent Robotics Instructor: Jivko Sinapov http://www.cs.utexas.edu/~jsinapov/teaching/cs378/ The Sense of Touch Announcements Remember this? Announcements Project Deliverables Final Report (6+ pages in PDF)

slide-1
SLIDE 1

CS 378: Autonomous Intelligent Robotics

Instructor: Jivko Sinapov

http://www.cs.utexas.edu/~jsinapov/teaching/cs378/

slide-2
SLIDE 2

The Sense of Touch

slide-3
SLIDE 3

Announcements

slide-4
SLIDE 4

Remember this?

slide-5
SLIDE 5

Announcements

slide-6
SLIDE 6

Project Deliverables

  • Final Report (6+ pages in PDF)
  • Code and Documentation (posted on

github)

  • Presentation including video and/or demo
slide-7
SLIDE 7

Readings for next week

As before, your pick.

slide-8
SLIDE 8

The Sense of Touch

slide-9
SLIDE 9

Overview of Haptic Sensing

“The haptic system uses sensory information derived from mechanoreceptors and thermoreceptors embedded in the skin (“cutaneous” inputs) together with mechanoreceptors embedded in muscles, tendons, and joints (“kinesthetic” inputs).”

slide-10
SLIDE 10

Meissner corpuscle Merkel cell complex Ruffini ending Pacinian corpuscle

slide-11
SLIDE 11

Properties of Mechanoreceptors

  • Relative size of receptive field

– Small vs. Large

  • Relative adaptation rate

– Response to onset/offset of skin deformation

  • vs. continued response during sustained skin

deformation

slide-12
SLIDE 12
slide-13
SLIDE 13

Meissner corpuscle Merkel cell complex Ruffini ending Pacinian corpuscle

slide-14
SLIDE 14
slide-15
SLIDE 15
slide-16
SLIDE 16
slide-17
SLIDE 17

Measuring Spatial Acuity

slide-18
SLIDE 18

Measuring Spatial Acuity

  • Two-point touch threshold:

– Represents the smallest spatial separation that can be detected some arbitrary percentage of the time

slide-19
SLIDE 19

Measuring Spatial Acuity

indistinguishable distinguishable

slide-20
SLIDE 20
slide-21
SLIDE 21
slide-22
SLIDE 22
slide-23
SLIDE 23
slide-24
SLIDE 24
slide-25
SLIDE 25

Temporal Resolving Capacity

  • People can resolve a temporal gap of

5 msec between successive taps on the skin

  • The temporal resolving capacity of skin is

better than that of vision but worse than that of audition

slide-26
SLIDE 26

How do people use haptic / tactile sensations to perceive objects?

slide-27
SLIDE 27

Exploratory Procedures

Lateral Motion Pressure Static Contact Unsupported Holding Enclosure Contour Following Insertion Part Motion Test [Lederman and Klatzky, 1987]

slide-28
SLIDE 28

Object Properties

  • Material properties:

– Surface texture, compliance, thermal quality

  • Geometric Properties:

– Shape and size

  • The weight of an object reflects both its

material density and its size

slide-29
SLIDE 29

[Power, 2000] [Lederman and Klatzky, 1987]

slide-30
SLIDE 30

The Sense of Touch: A Case Study with a Robot

slide-31
SLIDE 31

Sinapov, J., Sukhoy, V., Sahai, R., & Stoytchev, A. (2011). Vibrotactile recognition and categorization

  • f surfaces by a humanoid robot, IEEE

Transactions on Robotics, 27(3), 488-497.

http://home.engineering.iastate.edu/~alexs/lab/publications/papers/IEEEtran_Robotics_2011/IEEEtran_Robotics_2011.pdf

slide-32
SLIDE 32

The Vibrotactile Sensory Modality

slide-33
SLIDE 33

Merkel cell complex

slide-34
SLIDE 34
slide-35
SLIDE 35
slide-36
SLIDE 36

Can a robot use the vibrotactile sensory modality to recognize surface textures?

slide-37
SLIDE 37

Artificial Finger Tip

slide-38
SLIDE 38

Artificial Finger Tip

slide-39
SLIDE 39

Full Setup

slide-40
SLIDE 40

Exploratory Behaviors

slide-41
SLIDE 41

Exploratory Behaviors

slide-42
SLIDE 42

Surfaces

slide-43
SLIDE 43

Control Condition

  • The 21st “surface” consisted of scratching

in mid-air

slide-44
SLIDE 44

Data Collection

  • Each scratching behavior was performed
  • n each surface a total of 10 times
  • This produced a total of 5 x 21 x 10 =

1050 behavioral interactions

  • Each surface was changed after the robot

scratched it once with all five exploratory behaviors and not scratched again until the robot scratched all other surfaces

slide-45
SLIDE 45

Signal Processing Pipeline

slide-46
SLIDE 46

Signal Processing Pipeline

slide-47
SLIDE 47

Signal Processing Pipeline

Magnitude vector: Magnitude deviation vector:

slide-48
SLIDE 48

Signal Processing Pipeline

slide-49
SLIDE 49

Signal Processing Pipeline

Spectrogram of Magnitude Deviation Vector

slide-50
SLIDE 50

Signal Processing Pipeline

Spectrogram of Magnitude Deviation Vector

4 Hz 200 Hz

slide-51
SLIDE 51

Signal Processing Pipeline

slide-52
SLIDE 52

Surface Recognition Formulation

  • Given a sensory signal, estimate the

probability that a given surface was present, i.e.:

slide-53
SLIDE 53
  • k-NN: memory-based learning algorithm

? Test point With k = 3: 2 neighbors 1 neighbors

Therefore, Pr(red) = 0.66 Pr(blue) = 0.33

Machine Learning Models

slide-54
SLIDE 54
  • Support Vector Machine: a discriminative learning algorithm
  • 1. Finds maximum margin

hyperplane that separates two classes

  • 2. Uses Kernel function to

map data points into a feature space in which such a hyperplane exists

[http://www.imtech.res.in/raghava/rbpred/svm.jpg]

Machine Learning Models

slide-55
SLIDE 55

Machine Learning Models

slide-56
SLIDE 56

Surface Recognition Rate for a Single Behavior

slide-57
SLIDE 57

Surface Recognition Rate for a Single Behavior

slide-58
SLIDE 58

Surface Recognition Rate for a Single Behavior

Chance accuracy = 5 %

slide-59
SLIDE 59

Can we improve the recognition of surfaces after applying all 5 behaviors?

slide-60
SLIDE 60

Can we improve the recognition of surfaces after applying all 5 behaviors?

slide-61
SLIDE 61

Can we improve the recognition of surfaces after applying all 5 behaviors?

slide-62
SLIDE 62
slide-63
SLIDE 63
slide-64
SLIDE 64

Summary of Results

slide-65
SLIDE 65

Latest and Greatest in Tactile Sensing

Fishel, Jeremy A., and Gerald E. Loeb. "Bayesian exploration for intelligent identification

  • f textures." Frontiers in neurorobotics 6 (2012).
slide-66
SLIDE 66

The BioTac Artificial Finger

Fishel, Jeremy A., and Gerald E. Loeb. "Bayesian exploration for intelligent identification

  • f textures." Frontiers in neurorobotics 6 (2012).
slide-67
SLIDE 67

Surface Texture Exploration Setup

Fishel, Jeremy A., and Gerald E. Loeb. "Bayesian exploration for intelligent identification

  • f textures." Frontiers in neurorobotics 6 (2012).
slide-68
SLIDE 68
slide-69
SLIDE 69

Surface Recognition using Bayesian Inference

slide-70
SLIDE 70

Active Selection of Exploratory Movements

  • Using prior estimates of pair-wise surface

confusion, select the behavior that is most likely to be informative and/or resolve the current ambiguity

slide-71
SLIDE 71
slide-72
SLIDE 72

Surface Texture Recognition Results

slide-73
SLIDE 73

Surface Texture Recognition Results

slide-74
SLIDE 74

The Skilsense Project

slide-75
SLIDE 75

The Roboskin Project

slide-76
SLIDE 76

Sensory Substitution

slide-77
SLIDE 77
slide-78
SLIDE 78

Other ongoing projects:

  • Skilsens:

– http://www.youtube.com/watch?v=FQkC-gJGKmw

  • RoboSKIN:

– http://www.youtube.com/watch?v=yQGXYGS0Ojo

  • In the news:

– http://www.youtube.com/watch?v=49KmS0IkyW8 – http://www.youtube.com/watch?v=APTNpGZ7mWc

slide-79
SLIDE 79

THE END

slide-80
SLIDE 80