Cooperative Haptics for Humanoid Robot Teleoperation Master Thesis - - PowerPoint PPT Presentation

Cooperative Haptics for Humanoid Robot Teleoperation

Master Thesis Presentation Jo˜ ao O. Barros joaoobarros@ua.pt

University of Aveiro – Department of Mechanical Engineering Scientific supervision: Prof. Dr. V´ ıtor Manuel Ferreira dos Santos

• Prof. Dr. Filipe Miguel Teixeira Pereira da Silva

December 11, 2014

Introduction V-REP Model Construction Experimental Setup Control Strategies Experiments and Results Conclusions

Outline

Introduction V-REP Model Construction Experimental Setup Control Strategies Experiments and Results Conclusions

Jo˜ ao O. Barros December 2014 Cooperative Haptics for Humanoid Robot Teleoperation 2 / 28

Introduction V-REP Model Construction Experimental Setup Control Strategies Experiments and Results Conclusions

What is haptics?

Haptic interaction with the world refers to sensing and manipulation using our sense of touch. Computer haptics technology interfaces the user with a virtual environment via the sense of touch by applying forces, vibrations, and/or motions to the user.

Jo˜ ao O. Barros December 2014 Cooperative Haptics for Humanoid Robot Teleoperation 3 / 28

Introduction V-REP Model Construction Experimental Setup Control Strategies Experiments and Results Conclusions

PHUA project

The main goal is the development and integration of hardware and software components in a functional low-budget platform, to perform studies in balance and locomotion tasks.

◮ An approach for kinesthetic teaching is proposed, in which the

user interactively demonstrates a specific motion task, while feeling the dynamics of the system through a haptic interface – tele-kinesthetic teaching.

Jo˜ ao O. Barros December 2014 Cooperative Haptics for Humanoid Robot Teleoperation 4 / 28

Introduction V-REP Model Construction Experimental Setup Control Strategies Experiments and Results Conclusions

PHUA project

The main goal is the development and integration of hardware and software components in a functional low-budget platform, to perform studies in balance and locomotion tasks.

◮ An approach for kinesthetic teaching is proposed, in which the

user interactively demonstrates a specific motion task, while feeling the dynamics of the system through a haptic interface – tele-kinesthetic teaching.

Jo˜ ao O. Barros December 2014 Cooperative Haptics for Humanoid Robot Teleoperation 4 / 28

Introduction V-REP Model Construction Experimental Setup Control Strategies Experiments and Results Conclusions

PHUA project

Robot’s current form

❼ Anthropometrically built ❼ 27 degrees-of-freedom ❼ Hybrid actuation system ❼ Force sensors ❼ Artificial vision system ❼ Inertial sensors

Jo˜ ao O. Barros December 2014 Cooperative Haptics for Humanoid Robot Teleoperation 5 / 28

Introduction V-REP Model Construction Experimental Setup Control Strategies Experiments and Results Conclusions

PHUA project

PHANToM OMNI haptic device

◮ The haptic device used was the PHANToM OMNI, a ground-

based haptic joystick.

Jo˜ ao O. Barros December 2014 Cooperative Haptics for Humanoid Robot Teleoperation 6 / 28

Introduction V-REP Model Construction Experimental Setup Control Strategies Experiments and Results Conclusions

Objectives

❼ Adaption/creation of a humanoid model in V-REP, and defin-

ition of its kinematic chains according to PHUA robotic plat- form;

❼ Definition of the force feedback developed towards the user; ❼ Setting up the communication between the two haptic devices; ❼ Basic teleoperation of the PHUA model in V-REP, with one

joystick;

❼ Teleoperation of the V-REP model in more complex tasks, with

two joysticks;

❼ Test and recording of motion parameters during the simulation

• f different locomotion patterns;

❼ Experiments in the real robot.

Jo˜ ao O. Barros December 2014 Cooperative Haptics for Humanoid Robot Teleoperation 7 / 28

Introduction V-REP Model Construction Experimental Setup Control Strategies Experiments and Results Conclusions

V-REP model construction - stages

1 CAD model import 2 Pure shapes extraction 3 Inertial parameters definition 4 Shape linkage (joints and force sensors); 5 Kinematic chains definition

Jo˜ ao O. Barros December 2014 Cooperative Haptics for Humanoid Robot Teleoperation 8 / 28

Introduction V-REP Model Construction Experimental Setup Control Strategies Experiments and Results Conclusions

V-REP model construction - stages

1 CAD model import 2 Pure shapes extraction 3 Inertial parameters definition 4 Shape linkage (joints and force sensors); 5 Kinematic chains definition

Jo˜ ao O. Barros December 2014 Cooperative Haptics for Humanoid Robot Teleoperation 8 / 28

Introduction V-REP Model Construction Experimental Setup Control Strategies Experiments and Results Conclusions

V-REP model construction - stages

1 CAD model import 2 Pure shapes extraction 3 Inertial parameters definition 4 Shape linkage (joints and force sensors); 5 Kinematic chains definition

Jo˜ ao O. Barros December 2014 Cooperative Haptics for Humanoid Robot Teleoperation 8 / 28

Introduction V-REP Model Construction Experimental Setup Control Strategies Experiments and Results Conclusions

V-REP model construction - stages

1 CAD model import 2 Pure shapes extraction 3 Inertial parameters definition 4 Shape linkage (joints and force sensors); 5 Kinematic chains definition

Jo˜ ao O. Barros December 2014 Cooperative Haptics for Humanoid Robot Teleoperation 8 / 28

Introduction V-REP Model Construction Experimental Setup Control Strategies Experiments and Results Conclusions

V-REP model construction - stages

1 CAD model import 2 Pure shapes extraction 3 Inertial parameters definition 4 Shape linkage (joints and force sensors); 5 Kinematic chains definition

Jo˜ ao O. Barros December 2014 Cooperative Haptics for Humanoid Robot Teleoperation 8 / 28

Introduction V-REP Model Construction Experimental Setup Control Strategies Experiments and Results Conclusions

CAD model import

◮ CAD model parts were rearranged and redefined to match the

real robot’s body links and DOFs.

Lower limbs parts Upper body parts

Jo˜ ao O. Barros December 2014 Cooperative Haptics for Humanoid Robot Teleoperation 9 / 28

Introduction V-REP Model Construction Experimental Setup Control Strategies Experiments and Results Conclusions

Pure shapes extraction

◮ Pure shapes are used for dynamic simulations.

Imported shape Triangle edit mode Grouped pure shapes

Jo˜ ao O. Barros December 2014 Cooperative Haptics for Humanoid Robot Teleoperation 10 / 28

Introduction V-REP Model Construction Experimental Setup Control Strategies Experiments and Results Conclusions

Approximated model for dynamic simulation

External appearance Optimized model

Jo˜ ao O. Barros December 2014 Cooperative Haptics for Humanoid Robot Teleoperation 11 / 28

Introduction V-REP Model Construction Experimental Setup Control Strategies Experiments and Results Conclusions

Joints and force sensors

Model kinematic chains Force sensors implementation

Jo˜ ao O. Barros December 2014 Cooperative Haptics for Humanoid Robot Teleoperation 12 / 28

Introduction V-REP Model Construction Experimental Setup Control Strategies Experiments and Results Conclusions

Kinematic chains definition

◮ All the elements of the mechanism are linked together to build

the legs and arms kinematic chains.

Jo˜ ao O. Barros December 2014 Cooperative Haptics for Humanoid Robot Teleoperation 13 / 28

Introduction V-REP Model Construction Experimental Setup Control Strategies Experiments and Results Conclusions

Hardware and software solutions

◮ ROS distributed system.

Jo˜ ao O. Barros December 2014 Cooperative Haptics for Humanoid Robot Teleoperation 14 / 28

Introduction V-REP Model Construction Experimental Setup Control Strategies Experiments and Results Conclusions

ROS modules’ interaction

◮ V-REP operates in association with the other ROS modules.

phantom_control V-REP robot_state haptic_feedback record_data

joint state/ pelvis position feet positions, sensor values, sensor positions, dummy positions CoP position, support polygon feedback force vector Jo˜ ao O. Barros December 2014 Cooperative Haptics for Humanoid Robot Teleoperation 15 / 28

Introduction V-REP Model Construction Experimental Setup Control Strategies Experiments and Results Conclusions

ROS modules’ interaction

◮ V-REP operates in association with the other ROS modules.

phantom_control V-REP robot_state haptic_feedback record_data

joint state/ pelvis position feet positions, sensor values, sensor positions, dummy positions CoP position, support polygon feedback force vector Jo˜ ao O. Barros December 2014 Cooperative Haptics for Humanoid Robot Teleoperation 15 / 28

Introduction V-REP Model Construction Experimental Setup Control Strategies Experiments and Results Conclusions

ROS modules’ interaction

◮ V-REP operates in association with the other ROS modules.

phantom_control V-REP robot_state haptic_feedback record_data

joint state/ pelvis position feet positions, sensor values, sensor positions, dummy positions CoP position, support polygon feedback force vector Jo˜ ao O. Barros December 2014 Cooperative Haptics for Humanoid Robot Teleoperation 15 / 28

Introduction V-REP Model Construction Experimental Setup Control Strategies Experiments and Results Conclusions

ROS modules’ interaction

◮ V-REP operates in association with the other ROS modules.

phantom_control V-REP robot_state haptic_feedback record_data

joint state/ pelvis position feet positions, sensor values, sensor positions, dummy positions CoP position, support polygon feedback force vector Jo˜ ao O. Barros December 2014 Cooperative Haptics for Humanoid Robot Teleoperation 15 / 28

Introduction V-REP Model Construction Experimental Setup Control Strategies Experiments and Results Conclusions

ROS modules’ interaction

◮ V-REP operates in association with the other ROS modules.

phantom_control V-REP robot_state haptic_feedback record_data

joint state/ pelvis position feet positions, sensor values, sensor positions, dummy positions CoP position, support polygon feedback force vector Jo˜ ao O. Barros December 2014 Cooperative Haptics for Humanoid Robot Teleoperation 15 / 28

Introduction V-REP Model Construction Experimental Setup Control Strategies Experiments and Results Conclusions

ROS modules’ interaction

◮ V-REP operates in association with the other ROS modules.

phantom_control V-REP robot_state haptic_feedback record_data

joint state/ pelvis position feet positions, sensor values, sensor positions, dummy positions CoP position, support polygon feedback force vector Jo˜ ao O. Barros December 2014 Cooperative Haptics for Humanoid Robot Teleoperation 15 / 28

Introduction V-REP Model Construction Experimental Setup Control Strategies Experiments and Results Conclusions

Haptic control loop

◮ The position command defines a closed loop between the V-

REP model and the PHANToM device(s).

Human Operator PHANToM joystick IK/Torque Actuation Haptic Rendering PHUA Model V-REP Dynamics

Jo˜ ao O. Barros December 2014 Cooperative Haptics for Humanoid Robot Teleoperation 16 / 28

Introduction V-REP Model Construction Experimental Setup Control Strategies Experiments and Results Conclusions

Inverse kinematics correspondence

◮ The robot’s end-effector will follow the joystick position.

Jo˜ ao O. Barros December 2014 Cooperative Haptics for Humanoid Robot Teleoperation 17 / 28

Introduction V-REP Model Construction Experimental Setup Control Strategies Experiments and Results Conclusions

Joint space correspondence

◮ A joint-by-joint control is implemented between the joysticks

and the robot legs.

Jo˜ ao O. Barros December 2014 Cooperative Haptics for Humanoid Robot Teleoperation 18 / 28

Introduction V-REP Model Construction Experimental Setup Control Strategies Experiments and Results Conclusions

Force feedback formulation

Stability deviation and instability approach

Jo˜ ao O. Barros December 2014 Cooperative Haptics for Humanoid Robot Teleoperation 19 / 28

Introduction V-REP Model Construction Experimental Setup Control Strategies Experiments and Results Conclusions

Force feedback formulation

Force components weighting

❋❘ =

1 η+1 · ❋1 + η η+1 · ❋2

(N) ❋ = F(s) ·

❈❖P ❈❖P

(N) η =

• ∆❋2

∆❋1

• Jo˜

ao O. Barros December 2014 Cooperative Haptics for Humanoid Robot Teleoperation 20 / 28

Introduction V-REP Model Construction Experimental Setup Control Strategies Experiments and Results Conclusions

Inverse kinematics control mode

Jo˜ ao O. Barros December 2014 Cooperative Haptics for Humanoid Robot Teleoperation 21 / 28

Introduction V-REP Model Construction Experimental Setup Control Strategies Experiments and Results Conclusions

Inverse kinematics control mode

Joint state evolution

Jo˜ ao O. Barros December 2014 Cooperative Haptics for Humanoid Robot Teleoperation 22 / 28

Introduction V-REP Model Construction Experimental Setup Control Strategies Experiments and Results Conclusions

Inverse kinematics control mode

CoP variation

Jo˜ ao O. Barros December 2014 Cooperative Haptics for Humanoid Robot Teleoperation 23 / 28

Introduction V-REP Model Construction Experimental Setup Control Strategies Experiments and Results Conclusions

Inverse kinematics control mode

Force rendered by the haptic device

Jo˜ ao O. Barros December 2014 Cooperative Haptics for Humanoid Robot Teleoperation 24 / 28

Introduction V-REP Model Construction Experimental Setup Control Strategies Experiments and Results Conclusions

Torque control mode

Jo˜ ao O. Barros December 2014 Cooperative Haptics for Humanoid Robot Teleoperation 25 / 28

Introduction V-REP Model Construction Experimental Setup Control Strategies Experiments and Results Conclusions

Torque control mode

Joint state evolution

Jo˜ ao O. Barros December 2014 Cooperative Haptics for Humanoid Robot Teleoperation 26 / 28

Introduction V-REP Model Construction Experimental Setup Control Strategies Experiments and Results Conclusions

Conclusions

❼ Simulating the teleoperation scenario in a virtual environment

provides many benefits to the operator;

❼ Dynamically-rich simulations were possible, with very satisfying

results;

❼ Occasional glitches due to a defective contact with the ground

were the main problem registered in this work;

❼ The dual PHANToM OMNI configuration was successfully im-

plemented, by means of a well designed ROS framework;

❼ The developed force generation algorithms were successful for

testing purposes, but this formulation still needs improvement, particularly in joint-by-joint control;

❼ When controlled in the dual joystick configuration, the V-REP

model offers a wide range of teleoperation possibilities.

Jo˜ ao O. Barros December 2014 Cooperative Haptics for Humanoid Robot Teleoperation 27 / 28

Introduction V-REP Model Construction Experimental Setup Control Strategies Experiments and Results Conclusions

Future work suggestions

❼ Foot plates construction should be reviewed; ❼ Force generation algorithms can be further developed in terms

• f mathematical formulation, using extra sensory information;

❼ Metrics of the user’s performance during the teleoperation should

be defined, since they are crucial in what concerns to the learn- ing process;

❼ Exploring new scenarios, as uneven terrains, and include ex-

ternal disturbances are within the next goals in simulation;

❼ Test typical gait patterns using a path planning strategy; ❼ A support bracket for the haptic joysticks should be designed, in

• rder to truly implement, and ease the bimanual teleoperation;

❼ Adaption of dual joystick configuration to the real PHUA plat-

form.

Jo˜ ao O. Barros December 2014 Cooperative Haptics for Humanoid Robot Teleoperation 28 / 28

Cooperative Haptics for Humanoid Robot Teleoperation

Master Thesis Presentation Jo˜ ao O. Barros joaoobarros@ua.pt

University of Aveiro – Department of Mechanical Engineering Scientific supervision: Prof. Dr. V´ ıtor Manuel Ferreira dos Santos

• Prof. Dr. Filipe Miguel Teixeira Pereira da Silva

December 11, 2014