Autonomous and Human- Robot Collaborative Systems for Field - PowerPoint PPT Presentation

Autonomous and Human- Robot Collaborative Systems for Field Operations in Orchards, Greenhouses and Field Crops Avital Bechar Institute of Agricultural Engineering, ARO, Volcani Center, Israel 1

Overview  Background  The Agricultural Research Organization  Agricultural productivity and production (robotics perspectives)  Characteristics of the agricultural domain (robotics perspectives)  Basic principles (AgRobots)  ARL activity  Conclusions AgRA TC Webinar, March 2 24 , 2015

Agricultural Research Organization AgRA TC Webinar, March 3 24 , 2015

Agricultural Research Organization • Founded in 1921. • 1000 people: including 200 research scientists and 220 graduate students. • 6 Institutes: Soil water and environmental sciences; Plant protection; Animal Sciences; Plant sciences; Food sciences; and, Agricultural Engineering. AgRA TC Webinar, March 4 24 , 2015

Institute of Agricultural Engineering  The only research organization in Israel whose activities encompass a wide range of engineering and technological topics relating to all aspects of agriculture.  About 60 people, including 14 research scientists AgRA TC Webinar, March 5 24 , 2015

Institute of Agricultural Engineering  Two departments:  Sensing, information, and mechanization engineering  Production, growing and environmental engineering a AgRA TC Webinar, March 6 24 , 2015

Agricultural production  Cultivation and production processes in agriculture.  Affecting factors:  crop characteristics and requirements,  the geographical/geological environments,  climatic conditions,  market demands  the farmer’s capabilities and means.  Farm sizes increase and the number of farmers and agricultural workers decreases.  Human labor intensive and labor cost of 25-40%. AgRA TC Webinar, March 7 24 , 2015

(http://www.thadw.us/agricultural-employment-since- 1870 / ) AgRA TC Webinar, March 8 24 , 2015

CV of different materials 0.55 0.5 0.45 0.4 CV= σ / µ 0.35 0.3 CV CV2 > CV1 0.25 0.2 0.15 0.1 0.05 0 Metal Metal Metal Metal Plastic Rubber Wood Flower Bolts screw nuts nails Discs parts parts parts cuttings AgRA TC Webinar, March 9 24 , 2015

AgRA TC Webinar, March 10 24 , 2015

AgRA TC Webinar, March 11 24 , 2015

Unstructured Environments • Unknown a-priori • Unpredictable • Dynamic AgRA TC Webinar, March 12 24 , 2015

Unstructured Environments  The terrain, vegetation, landscape, visibility, illumination and other atmospheric conditions are not well defined; vary, have inherent uncertainty, and generate unpredictable and dynamic situations. AgRA TC Webinar, March 13 24 , 2015

Unstructured Objects Variable and non-uniform: size shape color texture location AgRA TC Webinar, March 14 24 , 2015

Industry Space Medical Agr. Under-water Military Env. + + - - Objects + + - - 15 AgRA TC Webinar, March 24 , 2015

Basic principles  Main task: pruning, picking, harvesting, weeding...  Supporting tasks: localization, detection, navigation…  Mobility and steering  Sensing  Path planning and navigation  Manipulators and end effectors  Control  Autonomy and human-robot collaboration AgRA TC Webinar, March 16 24 , 2015

Autonomy/Human-Robot collaboration  Autonomous robot are lack the capability to respond to ill-defined, unknown, changing, and unpredicted events, such as occur in unstructured environments.  Pareto principle: roughly 80% of a task is easy to adapt to robotics and automation and 20% is difficult (Stentz et al., 2002). AgRA TC Webinar, March 17 24 , 2015

Hybrid Human-Robot Systems Supporting Task 1 Supporting Supporting Main Task Task Task 4 2 Subsystem Subsystem Supporting 1 2 Task 3 AgRA TC Webinar, March 18 24 , 2015

AgRA TC Webinar, March 19 24 , 2015

Lab members (current)  2 PhD students (IE, CE-AgEng)  3 MSc students (ME, IE)  Mechanical Engineer  Electrical Engineer  Postdoc  Agronomist AgRA TC Webinar, March 20 24 , 2015

Projects (current)  Autonomous greenhouse sprayer for specialty crops ( with BGU ).  A human-robot collaborative system for deciduous tree selective pruning.  a human-robot system for selective melon collection ( with Technion ).  an autonomous system for monitoring of diseases in greenhouses ( with BGU ).  Robotic sonar for yield estimation ( with TAU ).  Characterization of Agricultural Tasks for the Design of a Minimalistic Robot ( with Technion ). AgRA TC Webinar, March 21 24 , 2015

Autonomous greenhouse sprayer  Avital Bechar, Itamar Dar, Victor Bloch, Yael Edan, Roee Finkelshtein, Guy Lidor, Ron Berenstein AgRA TC Webinar, March 22 24 , 2015

The motivation AgRA TC Webinar, March 23 24 , 2015

Plot geometry 170 100 m 115 AgRA TC Webinar, March 24 24 , 2015

Sensing (Features) ∆ R R AgRA TC Webinar, March 25 24 , 2015

Features Feature Formula Feature Formula R Red h H/(H+S+V) G Green s S/(H+S+V) B Blue v V/(H+S+V) r R/(R+G+B) deltaH (H-S)+(S-V) g G/(R+G+B) deltaS (S-H)+(S-V) b B/(R+G+B) deltaV (V-S)+(V-H) deltaR (R-G)+(R-B) C1 R-G deltaG (G-R)+(G-B) C2 R-B deltaB (B-G)+(B-R) C3 G-B − − H Hue Real_ModHue − 2 R G B { 1 cos ( ) + + − − − 2 2 2 2 ( R G B RG RB GB ) S Saturation imag_ModHue V Value 26

Decision Tree - CART Breiman et al., 1984  For all features Find feature threshold value that maximizes the "splitting criterion“  Among all features Choose the one that maximizes the "splitting criterion“ AgRA TC Webinar, March 27 24 , 2015

Decision tree Total success 1 Level 2 Level 3 Level Movie TS TS TS Movie 1 0.834 0.834 0.886 Movie 2 0.943 0.941 0.940 Movie 3 0.617 0.834 0.848 Movie 4 0.818 0.874 0.889 Movie 5 0.922 0.927 0.920 Movie 6 0.892 0.899 0.899 Movie 7 0.932 0.925 0.930 Average 0.851 0.891 0.902 Number of nodes 1 3 7 AgRA TC Webinar, March 28 24 , 2015

Judges Vote (~ Majority rule)  A customized CART variation, developed in this research  A “Judge” is single level CART (root node only)  Classification rule: Judges _ Vote Number _ of _ Judges Vote (M) 1 2 1 2 3 1 2 3 4 1 2 3 4 5 Judges (N) 2 2 3 3 3 4 4 4 4 5 5 5 5 5 AgRA TC Webinar, March 29 24 , 2015

Test set – "Judges Vote" Variation 2/2 2/3 3/4 3/5 4/5 2 Level (3 features) Average 0.903 0.914 0.915 0.905 0.890 0.920 std 0.041 0.021 0.016 0.020 0.022 0.044 AgRA TC Webinar, March 30 24 , 2015

Algorithm Evaluation Platform lifeCam NX-6000 Servo SC-1256T 45 Lenovo R400 PWM DAT A AX3500 - Dual 60A 180 ⁰ 123 CMP-03 Compass Arduno Encoder Optical Motor DL-30 AgRA TC Webinar, March E5 31 24 , 2015

AgRA TC Webinar, March 32 24 , 2015

TXT1 Ein Yahav 261109 1st exp-fast.wmv AgRA TC Webinar, March 33 24 , 2015

Modification of a commercial sprayer  An electric motor was installed on the steering wheel controlled by a Roboteq controller.  Installation of encoders on the steering pivot/axle and the front wheels.  PID control system.  Control system inputs: platform steering angle; desired direction from the adaptive algorithm and bearing.  Pure pursuit, carrot point 2m AgRA TC Webinar, March 34 24 , 2015

The ‘autonomous unit’  Installed on the platform  Connected to sensors and actuators AgRA TC Webinar, March 35 24 , 2015

Commercial Sprayer II AgRA TC Webinar, March 36 24 , 2015

AgRA TC Webinar, March 37 24 , 2015

A H-R collaborative system for selective pruning Avital Bechar, Victor Bloch, Roee Finkelshtain, Sivan Levi AgRA TC Webinar, March 38 24 , 2015

Objective  Develop a human-robot integrated system for tree pruning and shaping  Design of a cutting tool  Develop a modelling technique  Development of human robot interface and methodology AgRA TC Webinar, March 39 24 , 2015

Cutting tool alternatives  Chain saw  Pruning shears  Laser  Water jet  Disc saw  Jigsaw AgRA TC Webinar, March 40 24 , 2015

Cutting tool design  The cutting tool must be adapted to:  Tree dimensions, branch diameter and strength  Robot carrying ability, precision, energy source  Pruning technique: cutting angle, velocity  Tree structure: branch angles, depth inside the crown, obstacle density, reaching ability  Agronomical requirements:  Cutting angle 45°  Reduce risk of wounds  Cut disinfection (burned by high cutting speed) AgRA TC Webinar, March 41 24 , 2015

Cutting tool selection & modification for a robotic arm  Energy source, type and consumption  Safety  Weight  Dimensions  Precision and accuracy  … AgRA TC Webinar, March 42 24 , 2015

High accuracy requirements  Pruning shears: 3 directional dim. and 2 angular dim. Total required accuracy in 5D.  Disk saw: 1 directional dim. and 2 angular dim. Total required accuracy in 3D AgRA TC Webinar, March 43 24 , 2015