Visual Servoing Henrik I. Christensen Robotics and Intelligent - - PDF document

Visual Servoing

Henrik I. Christensen

Robotics and Intelligent Machines @ GT College of Computing Georgia Institute of Technology Atlanta, GA hic@cc.gatech.edu

February 14, 2008

• H. I. Christensen (RIM@GT)

Outline

1

Introduction

2

Problem Structure

3

Servoing

4

Image Based Servoing

5

Smart Strategies

6

Summary

• H. I. Christensen (RIM@GT)

Introduction

Recap of camera model Overview of strategies to visual servoing The basic theory behind models for visual servoing A few examples to illustrate use of the system.

• H. I. Christensen (RIM@GT)

Pin-Hole Model

• H. I. Christensen (RIM@GT)

Pin-Hole Model

The relations are then: x λ = X Z y λ = Y Z ⇒ x = λX Z y = λY Z

• H. I. Christensen (RIM@GT)

Pin-Hole Model

Remember Homogeneous Coordinates?

• P =

    X Y Z 1     Define the Perspective transform as T =     1 1 1

1 λ

   

• H. I. Christensen (RIM@GT)

The general problem of servoing

3D WORLD

GENERATION CONTROL TASK PLANNING

y u y*

ROBOT

STATE ESTIMATION

x

• H. I. Christensen (RIM@GT)

Outline

1

Introduction

2

Problem Structure

3

Servoing

4

Image Based Servoing

5

Smart Strategies

6

Summary

• H. I. Christensen (RIM@GT)

Coordinate Systems

PUMA 560

O O O O OC O

O

• H. I. Christensen (RIM@GT)

Basic Organization

• H. I. Christensen (RIM@GT)

Camera - Robot Configurations

1 2

EYE−IN−HAND STAND−ALONE EYE−IN−HAND STAND−ALONE

VISUAL MEASUREMENTS

VM1 VM2 VM3 VM4 number of cameras VM5

>2

• H. I. Christensen (RIM@GT)

Outline

1

Introduction

2

Problem Structure

3

Servoing

4

Image Based Servoing

5

Smart Strategies

6

Summary

• H. I. Christensen (RIM@GT)

Approaches to servoing

Position Based Servoing Image Based Servoing 21

2D Servoing

• H. I. Christensen (RIM@GT)

Position Based Servoing

BASED CONTROL POSITION (CARTESIAN) FEATURE EXTRACTION ESTIMATION POSE 3D CAMERA ROBOT + − T Tc d V, Ω

• H. I. Christensen (RIM@GT)

Position Servoing

Td Tc Oc Od

Y Z X X Y Z Y X

O

Z

• H. I. Christensen (RIM@GT)

Image Based Servoing

ROBOT + − c d V, Ω BASED CONTROL IMAGE EXTRACTION FEATURE CAMERA S S

• H. I. Christensen (RIM@GT)

Image Servoing

fc

fc

fc

fc

fd

fd

f

f4

Oc

• H. I. Christensen (RIM@GT)

2.5D Servoing

pe * pe uθ* uθ

Ω

• H. I. Christensen (RIM@GT)

Outline

1

Introduction

2

Problem Structure

3

Servoing

4

Image Based Servoing

5

Smart Strategies

6

Summary

• H. I. Christensen (RIM@GT)

Image Based Servoing

Specify the task as an image based task Associate an error function (e) with the task Goal is achieved when e = 0 Derivation of an Image Jacobian to relate image changes to control of the robot

• H. I. Christensen (RIM@GT)

Image Jacobian

J(P) =

• δf

δP

• =

   

δf1(P) δP1

. . .

δf1(P) δPm

. . . ... . . .

δfk(P) δP1

. . .

δfk(P) δPm

   

• H. I. Christensen (RIM@GT)

Point Motion

The point motion can be written as

C ˙

P =

CΩ × CP + CV

Or more detailed ˙ x = zωy − yωz + Tx ˙ y = xωz − zωx + Ty ˙ z = yωx − xωy + Tz

• H. I. Christensen (RIM@GT)

Point flow

Given the perspective projection u v

• = λ

z x y

• The image flow is then

˙ u = λz ˙ x − x ˙ z z2 ˙ v = λz ˙ y − y ˙ z z2

• H. I. Christensen (RIM@GT)

Image Jacobian

˙ u ˙ v

• =

ku kv

1 Z

− x

Z

−xy 1 + x2 −y

1 Z

− y

Z

−1 − y2 xy x

• 

       VX VY VZ ωX ωY ωZ        

• H. I. Christensen (RIM@GT)

For k image points?

       ˙ u1 ˙ v1 . . . ˙ uk ˙ vk        = A        

1 Z1

− x1

Z1

−x1y1 1 + x2

1

−y1

1 Z1

− y1

Z1

−1 − y2

1

x1y1 x1 . . . . . . . . . . . . . . . . . .

1 Zk

− xk

Zk

−xkyk 1 + x2

k

−yk

1 Zk

− yk

Zk

−1 − y2

k

xkyk xk                 VX VY VZ ωX ωY ωZ        

• H. I. Christensen (RIM@GT)

Considerations

Naive use of image based servoing might generate surprising motion

• f the camera

Comparison: Approach Pros Cons PBVS Intuitive Calibration? IBVS Accuracy Singularities?

• H. I. Christensen (RIM@GT)

Illustrative Examples

• H. I. Christensen (RIM@GT)

Outline

1

Introduction

2

Problem Structure

3

Servoing

4

Image Based Servoing

5

Smart Strategies

6

Summary

• H. I. Christensen (RIM@GT)

Smart Strategies

Doing large scale servoing is really hard Doing limited recognition is much easier How about recognition based servoing?

• H. I. Christensen (RIM@GT)

Model Based Servoing

• H. I. Christensen (RIM@GT)

Initial Pose Estimation

• H. I. Christensen (RIM@GT)

Initialization

• H. I. Christensen (RIM@GT)

Moving Edge Model

90 45 135

P P P t t t+1

• H. I. Christensen (RIM@GT)

Fitting the model

How does one fit the model to the object? A local pertubation model might be adequate Or optimization of differential motion (Drummond & Cipola)

• H. I. Christensen (RIM@GT)

An polygonal approximation

xt+1

i

yt+1

i

• =

a0 a1

• +

a2 a3 a4 a5 xt

i

yt

i

• +

a6 a7 a6 a7   xt2

i

xt

i yt i

yt2

i

  = W(X t

i )Θ

• H. I. Christensen (RIM@GT)

Model Based Servoing

• H. I. Christensen (RIM@GT)

Outline

1

Introduction

2

Problem Structure

3

Servoing

4

Image Based Servoing

5

Smart Strategies

6

Summary

• H. I. Christensen (RIM@GT)

Summary

Overview of methods for visual servoing Major camera - robot configurations The basic motion equations for design of control Strategies to arrive at a good solution

• H. I. Christensen (RIM@GT)