Autonomous and Human- Robot Collaborative Systems for Field - - PowerPoint PPT Presentation

▶
autonomous and human robot collaborative systems for
SMART_READER_LITE
LIVE PREVIEW

Autonomous and Human- Robot Collaborative Systems for Field - - PowerPoint PPT Presentation

Apr 11, 2024 417 likes •1.15k views

Autonomous and Human- Robot Collaborative Systems for Field Operations in Orchards, Greenhouses and Field Crops Avital Bechar Institute of Agricultural Engineering, ARO, Volcani Center, Israel 1 Overview Background The Agricultural

slide-1
SLIDE 1

1

Autonomous and Human- Robot Collaborative Systems

Avital Bechar

for Field Operations in Orchards, Greenhouses and Field Crops

Institute of Agricultural Engineering, ARO, Volcani Center, Israel

slide-2
SLIDE 2

2

Overview

 Background

 The Agricultural Research Organization  Agricultural productivity and production (robotics

perspectives)

 Characteristics of the agricultural domain (robotics

perspectives)

 Basic principles (AgRobots)  ARL activity  Conclusions

AgRA TC Webinar, March 2015 , 24

slide-3
SLIDE 3

3

Agricultural Research Organization

AgRA TC Webinar, March 2015 , 24

slide-4
SLIDE 4

4 AgRA TC Webinar, March 2015 , 24

  • Founded in 1921.
  • 1000 people: including 200 research

scientists and 220 graduate students.

  • 6 Institutes: Soil water and

environmental sciences; Plant protection; Animal Sciences; Plant sciences; Food sciences; and, Agricultural Engineering.

Agricultural Research Organization

slide-5
SLIDE 5

5 AgRA TC Webinar, March 2015 , 24

Institute of Agricultural Engineering

 The only research organization in Israel whose activities

encompass a wide range of engineering and technological topics relating to all aspects of agriculture.

 About 60 people, including 14 research scientists

slide-6
SLIDE 6

6 AgRA TC Webinar, March 2015 , 24

Institute of Agricultural Engineering

 Two departments:

 Sensing, information, and mechanization engineering  Production, growing and environmental engineering

a

slide-7
SLIDE 7

7

Agricultural production

 Cultivation and production processes in agriculture.  Affecting factors:

 crop characteristics and requirements,  the geographical/geological environments,  climatic conditions,  market demands  the farmer’s capabilities and means.

 Farm sizes increase and the number of farmers and

agricultural workers decreases.

 Human labor intensive and labor cost of 25-40%.

AgRA TC Webinar, March 2015 , 24

slide-8
SLIDE 8

8 AgRA TC Webinar, March 2015 , 24 (http://www.thadw.us/agricultural-employment-since- 1870 / )

slide-9
SLIDE 9

9 AgRA TC Webinar, March 2015 , 24

0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55

Metal Bolts Metal screw nuts Metal nails Metal Discs Plastic parts Rubber parts Wood parts Flower cuttings CV

CV of different materials

CV=σ/µ CV2 > CV1

slide-10
SLIDE 10

10 AgRA TC Webinar, March 2015 , 24

slide-11
SLIDE 11

11 AgRA TC Webinar, March 2015 , 24

slide-12
SLIDE 12

12

Unstructured Environments

  • Unknown a-priori
  • Unpredictable
  • Dynamic

AgRA TC Webinar, March 2015 , 24

slide-13
SLIDE 13

13

 The terrain, vegetation, landscape, visibility,

illumination and other atmospheric conditions are not well defined; vary, have inherent uncertainty, and generate unpredictable and dynamic situations.

AgRA TC Webinar, March 2015 , 24

Unstructured Environments

slide-14
SLIDE 14

14 AgRA TC Webinar, March 2015 , 24

Unstructured Objects

Variable and non-uniform: size shape color texture location

slide-15
SLIDE 15

15 AgRA TC Webinar, March 2015 , 24

  • +

+

Objects

  • +
  • +

Env.

Agr. Medical Space Under-water Military Industry

slide-16
SLIDE 16

16

Basic principles

 Main task: pruning, picking, harvesting, weeding...  Supporting tasks: localization, detection, navigation…  Mobility and steering  Sensing  Path planning and navigation  Manipulators and end effectors  Control  Autonomy and human-robot collaboration

AgRA TC Webinar, March 2015 , 24

slide-17
SLIDE 17

17

Autonomy/Human-Robot collaboration

AgRA TC Webinar, March 2015 , 24

 Autonomous robot are lack the capability to respond to

ill-defined, unknown, changing, and unpredicted events, such as occur in unstructured environments.

 Pareto principle: roughly 80% of a task is easy to adapt to

robotics and automation and 20% is difficult (Stentz et al., 2002).

slide-18
SLIDE 18

18

Hybrid Human-Robot Systems

AgRA TC Webinar, March 2015 , 24

Main Task

Supporting Task 1 Supporting Task 3 Supporting Task 4 Supporting Task 2

Subsystem 1 Subsystem 2

slide-19
SLIDE 19

19 AgRA TC Webinar, March 2015 , 24

slide-20
SLIDE 20

20 AgRA TC Webinar, March 2015 , 24

Lab members (current)

 2 PhD students (IE, CE-AgEng)  3 MSc students (ME, IE)  Mechanical Engineer  Electrical Engineer  Postdoc  Agronomist

slide-21
SLIDE 21

21 AgRA TC Webinar, March 2015 , 24

Projects (current)

 Autonomous greenhouse sprayer for specialty

crops (with BGU).

 A human-robot collaborative system for deciduous

tree selective pruning.

 a human-robot system for selective melon

collection (with Technion).

 an autonomous system for monitoring of diseases

in greenhouses (with BGU).

 Robotic sonar for yield estimation (with TAU).  Characterization of Agricultural Tasks for the

Design of a Minimalistic Robot (with Technion).

slide-22
SLIDE 22

22

Autonomous greenhouse sprayer

 Avital Bechar, Itamar Dar, Victor Bloch, Yael Edan,

Roee Finkelshtein, Guy Lidor, Ron Berenstein

AgRA TC Webinar, March 2015 , 24

slide-23
SLIDE 23

23

The motivation

AgRA TC Webinar, March 2015 , 24

slide-24
SLIDE 24

24

Plot geometry

100

m

115 170

AgRA TC Webinar, March 2015 , 24

slide-25
SLIDE 25

25 AgRA TC Webinar, March 2015 , 24

R R ∆

Sensing (Features)

slide-26
SLIDE 26

26

Features

Feature Formula Feature Formula

R Red h H/(H+S+V) G Green s S/(H+S+V) B Blue v V/(H+S+V) r R/(R+G+B) deltaH (H-S)+(S-V) g G/(R+G+B) deltaS (S-H)+(S-V) b B/(R+G+B) deltaV (V-S)+(V-H) deltaR (R-G)+(R-B) C1 R-G deltaG (G-R)+(G-B) C2 R-B deltaB (B-G)+(B-R) C3 G-B H Hue Real_ModHue S Saturation imag_ModHue V Value

) ( cos

) ( 2 2 1

2 2 2

{

GB RB RG B G R B G R − − − + + − − −

slide-27
SLIDE 27

27

 For all features

Find feature threshold value that maximizes the "splitting criterion“

 Among all features

Choose the one that maximizes the "splitting criterion“

Decision Tree - CART Breiman et al., 1984

AgRA TC Webinar, March 2015 , 24

slide-28
SLIDE 28

28

Total success

Movie 1 Level TS 2 Level TS 3 Level TS Movie 1 0.834 0.834 0.886 Movie 2 0.943 0.941 0.940 Movie 3 0.617 0.834 0.848 Movie 4 0.818 0.874 0.889 Movie 5 0.922 0.927 0.920 Movie 6 0.892 0.899 0.899 Movie 7 0.932 0.925 0.930 Average 0.851 0.891 0.902 Number of nodes 1 3 7

AgRA TC Webinar, March 2015 , 24

Decision tree

slide-29
SLIDE 29

29

Judges Vote (~ Majority rule)

 A customized CART variation, developed in this research  A “Judge” is single level CART (root node only)  Classification rule:

Judges

  • f

Number Vote Judges _ _ _

5 4 3 2 1 4 3 2 1 3 2 1 2 1 Vote (M) 5 5 5 5 5 4 4 4 4 3 3 3 2 2 Judges (N)

AgRA TC Webinar, March 2015 , 24

slide-30
SLIDE 30

30

Test set – "Judges Vote"

Variation 2/2 2/3 3/4 3/5 4/5 2 Level (3 features) Average 0.903 0.914 0.920 0.915 0.905 0.890 std 0.041 0.021 0.016 0.020 0.022 0.044

AgRA TC Webinar, March 2015 , 24

slide-31
SLIDE 31

31

Algorithm Evaluation Platform

Arduno CMP-03 Compass AX3500 - Dual 60A lifeCam NX-6000 180⁰

Lenovo R400

Servo SC-1256T

Motor DL-30 Encoder Optical E5

45 DAT A PWM 123

AgRA TC Webinar, March 2015 , 24

slide-32
SLIDE 32

32 AgRA TC Webinar, March 2015 , 24

slide-33
SLIDE 33

33

TXT1 Ein Yahav 261109 1st exp-fast.wmv

AgRA TC Webinar, March 2015 , 24

slide-34
SLIDE 34

34

Modification of a commercial sprayer

 An electric motor was installed on

the steering wheel controlled by a Roboteq controller.

 Installation of encoders on the

steering pivot/axle and the front wheels.

 PID control system.  Control system inputs: platform

steering angle; desired direction from the adaptive algorithm and bearing.

 Pure pursuit, carrot point 2m

AgRA TC Webinar, March 2015 , 24

slide-35
SLIDE 35

35

The ‘autonomous unit’

 Installed on the platform  Connected to sensors and

actuators

AgRA TC Webinar, March 2015 , 24

slide-36
SLIDE 36

36

Commercial Sprayer II

AgRA TC Webinar, March 2015 , 24

slide-37
SLIDE 37

37 AgRA TC Webinar, March 2015 , 24

slide-38
SLIDE 38

38

A H-R collaborative system for selective pruning

Avital Bechar, Victor Bloch, Roee Finkelshtain, Sivan Levi

AgRA TC Webinar, March 2015 , 24

slide-39
SLIDE 39

39

Objective

 Develop a human-robot integrated system for

tree pruning and shaping

 Design of a cutting tool  Develop a modelling technique  Development of human robot interface and

methodology

AgRA TC Webinar, March 2015 , 24

slide-40
SLIDE 40

40

Cutting tool alternatives

 Chain saw  Pruning shears  Laser  Water jet  Disc saw  Jigsaw

AgRA TC Webinar, March 2015 , 24

slide-41
SLIDE 41

41

Cutting tool design

 The cutting tool must be adapted to:

  • Tree dimensions, branch diameter and strength
  • Robot carrying ability, precision, energy source
  • Pruning technique: cutting angle, velocity
  • Tree structure: branch angles, depth inside the crown, obstacle

density, reaching ability

 Agronomical requirements:

  • Cutting angle 45°
  • Reduce risk of wounds
  • Cut disinfection (burned by high cutting speed)

AgRA TC Webinar, March 2015 , 24

slide-42
SLIDE 42

42

Cutting tool selection & modification for a robotic arm

 Energy source, type and consumption  Safety  Weight  Dimensions  Precision and accuracy  …

AgRA TC Webinar, March 2015 , 24

slide-43
SLIDE 43

43

High accuracy requirements

 Pruning shears: 3 directional dim. and 2 angular

  • dim. Total required accuracy in 5D.

 Disk saw: 1 directional dim. and 2 angular dim.

Total required accuracy in 3D

AgRA TC Webinar, March 2015 , 24

slide-44
SLIDE 44

44

Cutting tool design

10 20 30 40 50 60 70 80 3 6 9 12 15 18 21 24 27 30 33 36 Number of branches Branch Diameter [mm] 80% 82% 84% 86% 88% 90% 92% 94% 96% 98% 100% 3 6 9 12 15 18 21 24 27 30 33 36 Branch accumulative percentage Branch diameter [mm]

AgRA TC Webinar, March 2015, 24

slide-45
SLIDE 45

45

Cutting tool design

AgRA TC Webinar, March 2015, 24

slide-46
SLIDE 46

46

Tree Modeling

Simple and reliable method – mechanical

AgRA TC Webinar, March 2015 , 24

slide-47
SLIDE 47

47

Tree Modeling

(Linker et al., 2014)

AgRA TC Webinar, March 2015 , 24

slide-48
SLIDE 48

48

Path planning

Assuming cutting points given:

  • Find optimal reaching orientation
  • Solve robot navigation problem in 6 dimensional C-space
  • Find optimal order of cutting points

AgRA TC Webinar, March 2015 , 24

slide-49
SLIDE 49

49

Primal collaboration

 3D modelling of a tree a-priori  HO marks cutting point on model  Trajectory planning  Branch pruning

Drawbacks

 The need for a-priori 3D modelling  Long set up time  Computation power (time, cost)  Inaccurate  Lack of information  Not up to date

AgRA TC Webinar, March 2015 , 24

slide-50
SLIDE 50

50

Concept and task description

Stage Sense Reason Plan Act Sub-Task Images / model Cutting point detection Trajectory planning and control Branch pruning Control R H+ R R R

AgRA TC Webinar, March 2015 , 24

50

slide-51
SLIDE 51

51 AgRA TC Webinar, March 2015 , 24

slide-52
SLIDE 52

52 AgRA TC Webinar, March 2015 , 24

Brunch orientation (Two methods) Movements (joints and linear)

slide-53
SLIDE 53

53

 21 subjects  Age: 24 – 69  20 branches  Two types of marking (1 click and 2 clicks)  Two types of movements

Experiment

AgRA TC Webinar, March 2015 , 24

slide-54
SLIDE 54

54

Results

1 2 3 4 5 6

2 clicks 1 click

Time [sec]

click 1 click 2

a a b

α<<0.01

AgRA TC Webinar, March 2015 , 24

slide-55
SLIDE 55

55

Results

5 10 15 20 25

Cut sign move to scan scan move to cut return HR cycle time Robot cycle time

Average time [s] Joints Linearic

AgRA TC Webinar, March 2015 , 24

slide-56
SLIDE 56

56

Results

 Cutting point accuracy: 8-22 mm  Branch orientation accuracy: mean: 9.4º, med: 5.75º

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

1 2 3 4 5 6 7 8

2 4 6 8 10 12 14 16 18 20 22 More

Number of branches

Error [degree]

Frequency Cumulative %

AgRA TC Webinar, March 2015 , 24

slide-57
SLIDE 57

57

 Avital Bechar, Noa Schor, Sigal Berman, Aviv

Dombrovsky, Yigal Elad and Timea Ignat

AgRA TC Webinar, March 2015 , 24

A Diseases Monitoring Robot

slide-58
SLIDE 58

58

 A cart roves between crop rows.  Manipulator mounted on the cart is maneuvering into

a set of positions for sensing and detection.

 Sensors acquire data and fuse them to achieve high

precision.

 Early detection of two diseases: powdery mildew and

tomato spotted wilt virus (TSWV).

AgRA TC Webinar, March 2015 , 24

Scenario

slide-59
SLIDE 59

59

 No known algorithms for TSWV detection.  Detection of more than one threat has not been

attempted thus far.

 Development based on a holistic approach integrating

the design of both motion and perception.

AgRA TC Webinar, March 2015 , 24

Challenges

slide-60
SLIDE 60

60

 A robotic manipulator (MH5L, Motoman).  A custom-made end-effector.  Sensory apparatus.

AgRA TC Webinar, March 2015 , 24

Apparatus

Manipulator End-effector Laser sensor Multispectral camera Color camera

slide-61
SLIDE 61

61

TSWV Powdery mildew

AgRA TC Webinar, March 2015 , 24

Disease detection

slide-62
SLIDE 62

62 AgRA TC Webinar, March 2015 , 24

Motion planning and execution

slide-63
SLIDE 63

63 AgRA TC Webinar, March 2015 , 24

TSWV detection

slide-64
SLIDE 64

64 AgRA TC Webinar, March 2015 , 24

Disease detection

slide-65
SLIDE 65

65 AgRA TC Webinar, March 2015 , 24

Melons collecting robot

Avital Bechar, Moshe Karagoden, Ariel Weinstock, Moshe Mann, Sasha Katzman, Victor Bloch, Guy Lidor, Roee Finkelshtein, Itzhak Shmulevich

slide-66
SLIDE 66

66

 A Cartesian robot.

 Two cylindrical rails, toothed belt axis and end limit

switches;

 Two Stepper Motors;  Motor Controllers;  Programmable Logic Controller (PLC);  Frame (600 mm x1500mm).

 Vacuum operated Gripper

AgRA TC Webinar, March 2015 , 24

Apparatus

slide-67
SLIDE 67

67 67

Melons Picking Up Simulator

AgRA TC Webinar, March 2015 , 24

slide-68
SLIDE 68

68

 20 trials  In each trial, 3-7 objects  Density of 2-5 objects/m2  total collecting area of 4m X 0.5m  Cart velocity: 51 mm/sec  manipulator velocity: 800 mm/sec

AgRA TC Webinar, March 2015 , 24

Dynamic state experiment

slide-69
SLIDE 69

69

 Total success rate of 84%

 Due to technical and

communication problems

 Position error 7-10 mm at

reach location

 Collection pace: 7-8

  • bjects per minute

AgRA TC Webinar, March 2015 , 24

Results

slide-70
SLIDE 70

70

Conclusions

 There has been considerable progress.  Technical feasibility was shown.  Agricultural modifications or human integration.

AgRA TC Webinar, March 2015 , 24

slide-71
SLIDE 71

71

THANK YOU

Avital Bechar

avital@volcani.agri.gov.il