Robotic Deformable Object Cutting From Simulation to Experimentation - - PowerPoint PPT Presentation

Introduction Simulation Control Scheme Simulator Results Experiments Experimental Results Future Work

Robotic Deformable Object Cutting

From Simulation to Experimentation

Philip Long, Wisama Khalil and Philippe Martinet

IRCCyN ´ Ecole Centrale de Nantes

March 20, 2014

Introduction Simulation Control Scheme Simulator Results Experiments Experimental Results Future Work

Overview

Goal Simulation and experimental validation of a force/vision controller to separate soft materials

Introduction Simulation Control Scheme Simulator Results Experiments Experimental Results Future Work

Overview

1

Introduction Summary Motivation

2

Simulation Objectives Cell Modeling Cutting Strategy

3

Control Scheme Global Control Scheme Robot Control

4

Simulator Results Robotic Cell Behavior Graphical Results

5

Experiments Objectives Robot Control

6

Experimental Results Robot Behavior Graphical Results

7

Future Work Force Image Control Questions

Introduction Simulation Control Scheme Simulator Results Experiments Experimental Results Future Work

Motivation

Meat processing industry is the largest sector of food industry in France [1] Hazardous, strenuous and uncomfortable working conditions leading to a high rate of musculoskeletal injuries Robotization of meat separation process has been shown to increase both hygiene and accuracy in the manufacturing environment [2] Robotic cutting has so far been limited to highly repeatable separation scenarios

Introduction Simulation Control Scheme Simulator Results Experiments Experimental Results Future Work

ARMS Project

ARMS: A multi arms robotic system for muscle separation ANR ARPEGE funded project contributing to the robotization

• f the meat processing industry

A multi-arm system that can separate variable beef muscles arms.irccyn.ec-nantes.fr

Introduction Simulation Control Scheme Simulator Results Experiments Experimental Results Future Work

Contents

1

Introduction Summary Motivation

2

Simulation Objectives Cell Modeling Cutting Strategy

3

Control Scheme Global Control Scheme Robot Control

4

Simulator Results Robotic Cell Behavior Graphical Results

5

Experiments Objectives Robot Control

6

Experimental Results Robot Behavior Graphical Results

7

Future Work Force Image Control Questions

Introduction Simulation Control Scheme Simulator Results Experiments Experimental Results Future Work

Objectives

• 1. Modeling

Design of a robotic cell simulator that accurately represents the interactions and challenges of the meat cutting environment Simulator can be used to optimize robot and cell position, test redundancy resolution and simulation of control schemes

• 2. Control

Propose a control scheme that is able to complete separation task Control scheme uses force and vision sensor to cope with

• nline changes

Introduction Simulation Control Scheme Simulator Results Experiments Experimental Results Future Work

Summary

Modeling & Control of Robotic Meat Cutting Cell Global control of robotic cell in order to separate meat muscle Force vision strategies to cope with object deformation

Introduction Simulation Control Scheme Simulator Results Experiments Experimental Results Future Work

Simulator Global View

Mode Shapes

Trajectory Generator Visual Deviation Force Controller Position Controller

Force Feedback Visual Feedback Position Feedback Joint Velocities Dynamic Simulator FEM Simulink Controller

Introduction Simulation Control Scheme Simulator Results Experiments Experimental Results Future Work

Deformable Object Modeling I

Specifications: 3D scan of beef shoulder Cutting surface is extracted Muscles rebuilt using FEM

Introduction Simulation Control Scheme Simulator Results Experiments Experimental Results Future Work

Deformable Object Modeling II

Description: 3 deformable models are created[3] Muscles are meshed using FEM Aponeurosis (tendons that link the muscles) are modeled using spring damper systems

Beef Muscle 1 Beef Muscle 2 Aponeurosis FEM FEM Spring-Damper System Cutting Path

Introduction Simulation Control Scheme Simulator Results Experiments Experimental Results Future Work

Cutting Strategy

Cutting Relations pc must be below the virtual spring-damper, zc ≤ min (z1, z2) pc must lie within the bounding box (b1 . . . b4) The projection of pc must lie on the line segment |a1a2|

b.1 b.2 b.3 b.4

Introduction Simulation Control Scheme Simulator Results Experiments Experimental Results Future Work

Visual Information

Vision Extraction: Vision data is obtained from attachment points’ position The data is used to reconstruct an interpolated surface from which a trajectory can be obtained

Introduction Simulation Control Scheme Simulator Results Experiments Experimental Results Future Work

Contents

1

Introduction Summary Motivation

2

Simulation Objectives Cell Modeling Cutting Strategy

3

Control Scheme Global Control Scheme Robot Control

4

Simulator Results Robotic Cell Behavior Graphical Results

5

Experiments Objectives Robot Control

6

Experimental Results Robot Behavior Graphical Results

7

Future Work Force Image Control Questions

Introduction Simulation Control Scheme Simulator Results Experiments Experimental Results Future Work

Robotic Cell Controller

Cutting Robot Pulling Robot Vision System Position Controller Impedance Controller Feature Extraction

Guide Line

Trajectory Generator Trajectory Generator

t

Vision Robot Local Update Optimize FOV

Introduction Simulation Control Scheme Simulator Results Experiments Experimental Results Future Work

Cutting Robot

Description: For each passage a curve is fitted to the surface 5 degree polynomial trajectory gives xd,˙ xd and ¨ xd A local update is used to compensate for online deformation/poor approximation

X Y

Initial State On-line Deformation Local Update

τi = Aiw + H y∗d(t) = yd(t) + ∆y ∆y = yg − yc

Introduction Simulation Control Scheme Simulator Results Experiments Experimental Results Future Work

Pulling Robot

Description: Impedance controlled around a defined set point A desired force is given in the pulling direction As the Aponeurosis are severed the meat opens gradually

hp

¨ xd = ¨ x + λ (Kd (∆˙ x) + Kp (∆x) − Kf (∆h)) − ˙ Ji ˙ q

Introduction Simulation Control Scheme Simulator Results Experiments Experimental Results Future Work

Vision Robot

Description: Controlled to

• ptimize field
• f view

Robot moves to avoid

• cclusions

Field of View

˙ xd = −λL+

s (sd − sim)

Introduction Simulation Control Scheme Simulator Results Experiments Experimental Results Future Work

Contents

1

Introduction Summary Motivation

2

Simulation Objectives Cell Modeling Cutting Strategy

3

Control Scheme Global Control Scheme Robot Control

4

Simulator Results Robotic Cell Behavior Graphical Results

5

Experiments Objectives Robot Control

6

Experimental Results Robot Behavior Graphical Results

7

Future Work Force Image Control Questions

Introduction Simulation Control Scheme Simulator Results Experiments Experimental Results Future Work

Control of a Meat Cutting Robot Simulator

Introduction Simulation Control Scheme Simulator Results Experiments Experimental Results Future Work

Redundancy Resolution of Robotic Cell

Redundancy Resolution of multi-arm system

Introduction Simulation Control Scheme Simulator Results Experiments Experimental Results Future Work

Cutting Results- Cutting Trajectory

0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.03 0.035 0.04 0.045 0.05 0.055 0.06 0.065 0.07 0.075

Passage 4 x(m) y(m)

Surface(t0) Surface(t) Interpolation(t0) Robot(t)

Introduction Simulation Control Scheme Simulator Results Experiments Experimental Results Future Work

Cutting Results- Pulling Force

Time (s)

Introduction Simulation Control Scheme Simulator Results Experiments Experimental Results Future Work

Contents

1

Introduction Summary Motivation

2

Simulation Objectives Cell Modeling Cutting Strategy

3

Control Scheme Global Control Scheme Robot Control

4

Simulator Results Robotic Cell Behavior Graphical Results

5

Experiments Objectives Robot Control

6

Experimental Results Robot Behavior Graphical Results

7

Future Work Force Image Control Questions

Introduction Simulation Control Scheme Simulator Results Experiments Experimental Results Future Work

Review of Dynamic Simulator

Advantages Simulation of complex dynamic control schemes Contact simulation with realistic flexible objects Visualization of resulting behavior Disadvatages

1 Separation is carried out in intermediate region between

• bjects → no force feedback on knife

2 Simulation of idealistic vision system limited to spring

attachment points

Introduction Simulation Control Scheme Simulator Results Experiments Experimental Results Future Work

Experimental Setup

Motivation

1 Validate controller using local update by vision 2 Use sensed forces to control cutting tool

Setup

Introduction Simulation Control Scheme Simulator Results Experiments Experimental Results Future Work

Global Controller

Trajectory Generator Cutting Robot Force Deviation Vision Deviation Camera

I

Trajectory Generator Cartesian Error Pulling Robot Impedance Controller Cartesian Error

Global Control

1 Dual arm control for object separation 2 Cutting Robot using force and vision to cut along trajectory 3 Pulling robot uses impedance control to gradually open

cutting valley

Introduction Simulation Control Scheme Simulator Results Experiments Experimental Results Future Work

Global Controller

Trajectory Generator Cutting Robot Force Deviation Vision Deviation Camera

I

Trajectory Generator Cartesian Error Pulling Robot Impedance Controller Cartesian Error

Vision Control Vision control is used to compensate for deformation

1 Offline Trajectory estimation is incorrect 2 The pulling robot deforms the trajectory in an a priori

unknown manner

Introduction Simulation Control Scheme Simulator Results Experiments Experimental Results Future Work

Global Controller

Trajectory Generator Cutting Robot Force Deviation Vision Deviation Camera

I

Trajectory Generator Cartesian Error Pulling Robot Impedance Controller Cartesian Error

Impedance Control Pulling robot is used to open cutting valley

1 Used to restrain/fix the object on one side 2 Opening cutting valley reduces friction on blade further

decreasing the required forces

Introduction Simulation Control Scheme Simulator Results Experiments Experimental Results Future Work

Vision Controller

Description: Position of desired frame is obtained from image points Visual deviation is generated in Cartesian Space        pv

xi

pv

zi

pv

yi

pv

zi

1        = A   ui vi 1  

Introduction Simulation Control Scheme Simulator Results Experiments Experimental Results Future Work

Contents

1

Introduction Summary Motivation

2

Simulation Objectives Cell Modeling Cutting Strategy

3

Control Scheme Global Control Scheme Robot Control

4

Simulator Results Robotic Cell Behavior Graphical Results

5

Experiments Objectives Robot Control

6

Experimental Results Robot Behavior Graphical Results

7

Future Work Force Image Control Questions

Introduction Simulation Control Scheme Simulator Results Experiments Experimental Results Future Work

Results

Introduction Simulation Control Scheme Simulator Results Experiments Experimental Results Future Work

Visually Correct Trajectory

−0.7 −0.66 −0.62 −0.58 −0.54 −0.5 −0.46 −0.42 0.35 0.36 0.37 0.38 0.39 0.4 0.41 0.42 0.43

Passage 1 X(m) Y(m)

−0.7 −0.66 −0.62 −0.58 −0.54 −0.5 −0.46 −0.42 0.35 0.36 0.37 0.38 0.39 0.4 0.41 0.42 0.43

X(m) Y(m) Passage 13

−0.7 −0.66 −0.62 −0.58 −0.54 −0.5 −0.46 −0.42 0.35 0.36 0.37 0.38 0.39 0.4 0.41 0.42 0.43

X(m) Y(m) Passage 20 Offline Estimation Vision Extracted Curve Robot position

Robot Trajectory Following

Introduction Simulation Control Scheme Simulator Results Experiments Experimental Results Future Work

Contents

1

Introduction Summary Motivation

2

Simulation Objectives Cell Modeling Cutting Strategy

3

Control Scheme Global Control Scheme Robot Control

4

Simulator Results Robotic Cell Behavior Graphical Results

5

Experiments Objectives Robot Control

6

Experimental Results Robot Behavior Graphical Results

7

Future Work Force Image Control Questions

Introduction Simulation Control Scheme Simulator Results Experiments Experimental Results Future Work

Experimental Validation

Current Limitations Relies on an offline estimation of the cutting curve Valid for planar objects Sensitive to Calibration since Cartesian position is rebuilt Object height is known Pulling force learned offline

Introduction Simulation Control Scheme Simulator Results Experiments Experimental Results Future Work

Experimental Validation

Current Limitations Relies on an offline estimation of the cutting curve Valid for planar objects Sensitive to Calibration since Cartesian position is rebuilt Object height is known Pulling force learned offline Proposed Solution → IBVS Control Image directly, using image moments to obtain supplementary information i.e height angle etc.

Introduction Simulation Control Scheme Simulator Results Experiments Experimental Results Future Work

Questions?

Introduction Simulation Control Scheme Simulator Results Experiments Experimental Results Future Work

References I

Institut national de la statistique et des ´ etudes ´ economiques (INSEE), “Industrie agroalimentaire.” http://www.insee.fr/fr/themes/document.asp?ref id=T10F181, 2009. Accessed: 2013-04-22.

• L. Hinrichsen, “Manufacturing technology in the danish pig

slaughter industry,” Meat science, vol. 84, no. 2, pp. 271–275, 2010.

• H. Delingette, S. Cotin, and N. Ayache, “A hybrid elastic

model allowing real-time cutting, deformations and force-feedback for surgery training and simulation,” in Computer Animation, 1999. Proceedings, pp. 70–81, 1999.