Computer Vision and Control Control Computer Vision and for - - PowerPoint PPT Presentation

▶
computer vision and control control computer vision and
SMART_READER_LITE
LIVE PREVIEW

Computer Vision and Control Control Computer Vision and for - - PowerPoint PPT Presentation

Jan 12, 2023 267 likes •901 views

Computer Vision and Control Control Computer Vision and for Autonomous Autonomous Robots Robots for Prof. Dr. Raul Rojas FU Berlin Embodied Intelligence: A new Embodied Intelligence: A new Paradigm for AI Paradigm for AI - Intelligence

slide-1
SLIDE 1

Computer Vision and Computer Vision and Control Control for for Autonomous Autonomous Robots Robots

  • Prof. Dr. Raul Rojas

FU Berlin

slide-2
SLIDE 2

Embodied Intelligence: A new Embodied Intelligence: A new Paradigm for AI Paradigm for AI

  • Intelligence

Intelligence needs needs a a body body: : mechanics mechanics

  • Computer

Computer vision vision in real time in real time

  • Energy

Energy management management

  • Local

Local control control

  • Communication

Communication between between agents agents

  • Coordination

Coordination and and team team behavior behavior

  • Adaptation and

Adaptation and learning learning

„ „Artificial Artificial Intelligence Intelligence is is the the art and art and science science

  • f
  • f the

the subconscious subconscious“ “

slide-3
SLIDE 3

Robotic Robotic Soccer Soccer as AI as AI Benchmark Benchmark

  • RoboCup

RoboCup started started with with IJCAI 1997 IJCAI 1997

  • I

I -

  • Simulation

Simulation league league

  • II

II – – Small Small size size league league

  • III

III-

  • Mid

Mid-

  • size

size league league

  • IV

IV-

  • Legged

Legged league league

  • V

V – – Humanoid Humanoid league league

slide-4
SLIDE 4

Small Small-

  • Size

Size Liga Liga

4.5 by 5 meter field Five vs five 18 cm in diameter

slide-5
SLIDE 5

Lisbon Lisbon 2004 2004

slide-6
SLIDE 6

Kicking the distance Kicking the distance

slide-7
SLIDE 7

Mid Mid-

  • size

size league league

field 12 × 8 meters four

  • n

four

slide-8
SLIDE 8

Lisbon Lisbon 2004 2004

slide-9
SLIDE 9

Pressuring the goalie Pressuring the goalie

slide-10
SLIDE 10

Our small Our small-

  • size robots

size robots

slide-11
SLIDE 11

Omnidirectional Omnidirectional Design Design

slide-12
SLIDE 12

Omnidirectional Omnidirectional Control Control

slide-13
SLIDE 13

Our Our mid mid-

  • size

size robots robots

Omnidirectional vision

  • Laptop for control
  • Firewire video camera
slide-14
SLIDE 14

CAD Design CAD Design

slide-15
SLIDE 15

FUXABOT: FUXABOT: The The Hexapod Hexapod

slide-16
SLIDE 16

I Global vision I Global vision

slide-17
SLIDE 17

Global Global vision vision

global camera computer wireless communication

slide-18
SLIDE 18

The The world world is is colored colored

Team color ball

slide-19
SLIDE 19

Projective Transformation Projective Transformation

slide-20
SLIDE 20

Automatic camera calibration Automatic camera calibration

slide-21
SLIDE 21

Illumination artifacts Illumination artifacts

slide-22
SLIDE 22

Adaptive color maps Adaptive color maps

slide-23
SLIDE 23

Tracking Tracking helps helps computer computer vision vision

  • just a few pixels are read
  • the position of the ball is predicted
  • variable search frame
slide-24
SLIDE 24

Tracking Tracking the the robots robots

t

vision delay communication delay

Data from the past We need the data of the future

slide-25
SLIDE 25

Predict Predict the the robot‘s robot‘s movement movement

(x1,y1) (x2,y2) (x3,y3) (x4,y4) (vx,vy,w)1 (vx,vy,w)2 (vx,vy,w)3 (vx,vy,w)4 positions commands predictor predictor

(x,y,θ )

Position and

  • rientation four

frames in the future (100 ms)

slide-26
SLIDE 26
slide-27
SLIDE 27

II Local vision II Local vision

slide-28
SLIDE 28

Our first Our first omnivision

  • mnivision robots

robots

slide-29
SLIDE 29

Spherical and parabolic Spherical and parabolic transformations transformations

slide-30
SLIDE 30

The field seen with our mirror The field seen with our mirror

slide-31
SLIDE 31

Locating the robot Locating the robot

Parabolic M irror

100 200 300 400 500 0.1 0.2 0.3 0.4 pixel distance distance

slide-32
SLIDE 32

Expectation Expectation-

  • Maximization

Maximization

The model „attracts“ the cloud of points

slide-33
SLIDE 33

Forces on real data

slide-34
SLIDE 34

Precomputed resultant forces for each coordinate

slide-35
SLIDE 35

Obstacle Detection

slide-36
SLIDE 36

Obstacle Modelling

slide-37
SLIDE 37

Obstacle Fusion

slide-38
SLIDE 38

Obstacle Identification

slide-39
SLIDE 39

III Reactive Behavior III Reactive Behavior

slide-40
SLIDE 40

Reactive Reactive Behavior Behavior Control Control

fast slow sensors behaviors actuators

slide-41
SLIDE 41

Structure Structure of a

  • f a layer

layer

Higher layer

sensors behaviors actors

effectors sensors

Lower layer

slide-42
SLIDE 42

Kicking Kicking reflex reflex

Kicking reflex activated

slide-43
SLIDE 43

Screenshot of control software Screenshot of control software

slide-44
SLIDE 44

IV Learning and IV Learning and Coaching the robots Coaching the robots

slide-45
SLIDE 45

Anpassbarkeit Anpassbarkeit

slide-46
SLIDE 46

Raumfreiheit Raumfreiheit

slide-47
SLIDE 47

Beispiel Beispiel-

  • Eingabe

Eingabe

slide-48
SLIDE 48

Learning to pass Learning to pass

slide-49
SLIDE 49

Passing Passing game game

slide-50
SLIDE 50

Team Play Team Play

slide-51
SLIDE 51

The The Goalie Goalie

slide-52
SLIDE 52

Goalie again Goalie again

slide-53
SLIDE 53

Learning Learning: robot : robot heal heal yourself yourself

slide-54
SLIDE 54
slide-55
SLIDE 55

Invert the prediction Invert the prediction

(x1,y1) (x2,y2) (x3,y3) (x4,y4) (vx,vy,w)1 (vx,vy,w)2 (vx,vy,w)3 (vx,vy,w)4 predictor predictor

(x,y,θ )

desired position

slide-56
SLIDE 56

One burnt motor One burnt motor

slide-57
SLIDE 57

V Summary and V Summary and Outlook Outlook

slide-58
SLIDE 58

FU Fighters FU Fighters

  • 1999

1999 Vizeweltmeister Vizeweltmeister

  • 2000

2000 Europa Europa-

  • und Vizeweltmeister

und Vizeweltmeister

  • 2001

2001 Vierter Platz Vierter Platz

  • 2002

2002 Europa Europa-

  • und Vizeweltmeister

und Vizeweltmeister

  • 2003

2003 Europameister Europameister

  • Dritter Platz (

Dritter Platz (small small-

  • size

size) )

  • Halbfinalist (

Halbfinalist (mid mid-

  • size

size) )

  • 2004

2004 Weltmeister ( Weltmeister (small small-

  • size

size) )

  • Vierter Platz (

Vierter Platz (mid mid-

  • size

size) )

slide-59
SLIDE 59

Small-Size Team Anna Egorova, Alexander Gloye, Mark Simon, Cüneyt Göktekin, Bastian Hecht, Achim Liers, Oliver Tenchio, Fabian Wiesel, Lina Ourima, Maria Jütte, Thomas Sunderman Susanne Schöttker-Söhl

slide-60
SLIDE 60

Mid Mid-

  • size

size Team Team

Holger Freyther, Ketill Gunnarsson, Henning Heinold, Felix von Hundelshausen, Wolf Lindstrot, Marian Luft, Slav Petrov, Michael Schreiber, Frederik Zilly, Fabian Ruff, David Schneider, Markus Kettern

slide-61
SLIDE 61

Detlef M Detlef Müller ller und Feinwerktechnik und Feinwerktechnik

slide-62
SLIDE 62

Fritz Fritz-

  • Haber

Haber-

  • Institut

Institut

  • Georg

Georg Heyne Heyne

  • Peter

Peter Zilske Zilske

  • Torsten

Torsten Vetter Vetter

  • Ronald

Ronald Nehring Nehring

slide-63
SLIDE 63