Computer Vision and Control Control Computer Vision and for - PowerPoint PPT Presentation

Computer Vision and Control Control Computer Vision and for Autonomous Autonomous Robots Robots for Prof. Dr. Raul Rojas FU Berlin

Embodied Intelligence: A new Embodied Intelligence: A new Paradigm for AI Paradigm for AI - Intelligence Intelligence needs needs a a body body: : mechanics mechanics - Computer Computer vision vision in real time in real time - „Artificial Artificial Intelligence Intelligence is is the the art and art and science science „ - Energy Energy management management - of the the subconscious subconscious“ “ of - Local Local control control - - Communication Communication between between agents agents - - Coordination Coordination and and team team behavior behavior - - Adaptation and Adaptation and learning learning -

Robotic Soccer Soccer as AI as AI Robotic Benchmark Benchmark � RoboCup RoboCup started started with with IJCAI 1997 IJCAI 1997 � � I I - - Simulation Simulation league league � � II II – – Small Small size size league league � � III III- - Mid Mid- -size size league league � � IV IV- - Legged Legged league league � � V V – – Humanoid Humanoid league league �

Small- -Size Size Liga Liga Small 18 cm in diameter 4.5 by 5 meter field Five vs five

Lisbon 2004 2004 Lisbon

Kicking the distance Kicking the distance

Mid- -size size league league Mid four on field four 12 × 8 meters

Lisbon 2004 2004 Lisbon

Pressuring the goalie Pressuring the goalie

Our small- -size robots size robots Our small

Omnidirectional Design Design Omnidirectional

Omnidirectional Control Control Omnidirectional

Our mid mid- -size size robots robots Our Omnidirectional vision - Laptop for control - Firewire video camera

CAD Design CAD Design

FUXABOT: The The Hexapod Hexapod FUXABOT:

I Global vision I Global vision

Global vision vision Global wireless communication computer global camera

The world world is is colored colored The Team color ball

Projective Transformation Projective Transformation

Automatic camera calibration Automatic camera calibration

Illumination artifacts Illumination artifacts

Adaptive color maps Adaptive color maps

Tracking helps helps computer computer vision vision Tracking • the position of the ball is predicted • variable search frame • just a few pixels are read

Tracking the the robots robots Tracking We need the data of the future Data from the past t vision communication delay delay

Predict the the robot‘s robot‘s movement movement Predict positions ( x 1 , y 1 ) ( v x , v y, w ) 1 ( x 2 , y 2 ) ( x , y, θ ) predictor ( v x , v y, w ) 2 predictor ( x 3 , y 3 ) Position and orientation four ( v x , v y, w ) 3 frames in the future (100 ms) ( x 4 , y 4 ) ( v x , v y, w ) 4 commands

II Local vision II Local vision

Our first omnivision omnivision robots robots Our first

Spherical and parabolic Spherical and parabolic transformations transformations

The field seen with our mirror The field seen with our mirror

Locating the robot Locating the robot Parabolic M irror 500 400 distance 300 200 100 0 0 0.1 0.2 0.3 0.4 pixel distance

Expectation- -Maximization Maximization Expectation The model „attracts“ the cloud of points

Forces on real data

Precomputed resultant forces for each coordinate

Obstacle Detection

Obstacle Modelling

Obstacle Fusion

Obstacle Identification

III Reactive Behavior III Reactive Behavior

Reactive Behavior Behavior Control Control Reactive slow fast sensors behaviors actuators

Structure of a of a layer layer Structure Higher layer sensors effectors sensors behaviors actors Lower layer

Kicking reflex reflex Kicking Kicking reflex activated

Screenshot of control software Screenshot of control software

IV Learning and IV Learning and Coaching the robots Coaching the robots

Anpassbarkeit Anpassbarkeit

Raumfreiheit Raumfreiheit

Beispiel- -Eingabe Eingabe Beispiel

Learning to pass Learning to pass

Passing game game Passing

Team Play Team Play

The Goalie Goalie The

Goalie again Goalie again

Learning: robot : robot heal heal yourself yourself Learning

Invert the prediction Invert the prediction ( x 1 , y 1 ) ( v x , v y, w ) 1 ( x 2 , y 2 ) ( x , y, θ ) predictor ( v x , v y, w ) 2 predictor ( x 3 , y 3 ) desired position ( v x , v y, w ) 3 ( x 4 , y 4 ) ( v x , v y, w ) 4

One burnt motor One burnt motor

V Summary and V Summary and Outlook Outlook

FU Fighters FU Fighters � 1999 1999 Vizeweltmeister Vizeweltmeister � � 2000 2000 Europa- - und Vizeweltmeister und Vizeweltmeister Europa � � 2001 2001 Vierter Platz Vierter Platz � � 2002 2002 Europa- - und Vizeweltmeister und Vizeweltmeister Europa � � 2003 2003 Europameister Europameister � Dritter Platz (small small- -size size) ) Dritter Platz ( � � Halbfinalist (mid mid- -size size) ) Halbfinalist ( � � � 2004 2004 Weltmeister (small small- -size size) ) Weltmeister ( � Vierter Platz (mid mid- -size size) ) Vierter Platz ( � �

Small-Size Team Anna Egorova, Alexander Gloye, Mark Simon, Cüneyt Göktekin, Bastian Hecht, Achim Liers, Oliver Tenchio, Fabian Wiesel, Lina Ourima, Maria Jütte, Thomas Sunderman Susanne Schöttker-Söhl

Mid- -size size Team Team Mid Holger Freyther, Ketill Gunnarsson, Henning Heinold, Felix von Hundelshausen, Wolf Lindstrot, Marian Luft, Slav Petrov, Michael Schreiber, Frederik Zilly, Fabian Ruff, David Schneider, Markus Kettern

Detlef Müller ller und Feinwerktechnik und Feinwerktechnik Detlef M

Fritz- -Haber Haber- -Institut Institut Fritz � Georg Georg Heyne Heyne � � Peter Peter Zilske Zilske � � Torsten Torsten Vetter Vetter � � Ronald Ronald Nehring Nehring �