Robotic Apple Harvesting in Washington State Joe Davidson b & - - PowerPoint PPT Presentation

▶
robotic apple harvesting in washington state
SMART_READER_LITE
LIVE PREVIEW

Robotic Apple Harvesting in Washington State Joe Davidson b & - - PowerPoint PPT Presentation

Apr 23, 2023 314 likes •546 views

Robotic Apple Harvesting in Washington State Joe Davidson b & Abhisesh Silwal a IEEE Agricultural Robotics & Automation Webinar December 15 th , 2015 a Center for Precision and Automated Agricultural Systems (CPAAS) & b School of

slide-1
SLIDE 1

Supported By: National Institute of Food and Agriculture

Robotic Apple Harvesting in Washington State

Joe Davidsonb & Abhisesh Silwala

IEEE Agricultural Robotics & Automation Webinar December 15th, 2015

a Center for Precision and Automated Agricultural Systems (CPAAS) & b School of Mechanical and Materials Engineering,

Washington State University (WSU)

1

slide-2
SLIDE 2

Supported By: National Institute of Food and Agriculture

Acknowledgements

This work was funded by the United States Department of Agriculture – National Institute of Food and Agriculture (USDA- NIFA) through the National Robotics Initiative (NRI).

2

slide-3
SLIDE 3

Supported By: National Institute of Food and Agriculture

Presentation Overview

  • Motivation
  • Working environment
  • Design objectives
  • Hand picking analysis
  • System design
  • Preliminary field testing results
  • Future work
  • Questions

3

slide-4
SLIDE 4

Supported By: National Institute of Food and Agriculture

Research Motivation

  • Washington State fresh market apple industry in 2014

– 2.7 million metric tons of apples valued at $1.84 billion USD1 – Accounted for 70% of U.S. apple production

  • The WA fresh market apple harvest requires

– Employment of 30,000 additional workers – An estimated cost of $1,100 to $2,100 USD per acre per year2,3

  • Labor costs are rising and there is increasing uncertainty about the availability of

farm labor

  • Lack of mechanical harvesting for fresh market apples is a significant problem

1 USDA National Agricultural Statistics Service. (2014 Washington Agriculture Overview). Retrieved November 2, 2015, from

http://www.nass.usda.gov/Quick_Stats/Ag_Overview/stateOverview.php?state=WASHINGTON

2 Galinato, S., & Gallardo, R. K. (2011). 2010 Estimated Cost of Producing Pears in North Central Washington (FS031E). Retrieved January 13, 2013, from

http://extecon.wsu.edu/pages/Enterprise_Budgets.

3 Gallardo, R. K., Taylor, M., & Hinman, H. (2010). 2009 Cost Estimates of Establishing and Producing Gala Apples in Washington (FS005E). Retrieved January 7, 2013, from

http://extecon.wsu.edu/pages/Enterprise_Budgets.

4

slide-5
SLIDE 5

Supported By: National Institute of Food and Agriculture

Long-term Goal: Reduce dependence on the labor force for fresh market tree fruit harvesting

5

slide-6
SLIDE 6

Supported By: National Institute of Food and Agriculture

Working Environment

  • Commercial apple orchard located in Prosser, WA
  • Highly unstructured environment
  • Modern cultivation systems with formal tree architectures
  • “Fruit Wall” concept simplifies the task

6

slide-7
SLIDE 7

Supported By: National Institute of Food and Agriculture

Design Objectives

  • Cycle time < 6 sec
  • Detachment success > 90%
  • Fruit damage < 10%
  • Pick multiple apple varieties
  • Modular design that is cost-effective
  • Our Approach: An ‘undersensed’ design that executes

look-and-move fruit picking, is mechanically robust to position error, and replicates the human picking process

7

slide-8
SLIDE 8

Supported By: National Institute of Food and Agriculture

Initial Design Development: Manual Apple Picking

  • Fruit is grasped with a spherical power grasp4 with the index

finger applying pressure against the stem

  • No dexterous manipulation of the fruit with the fingers
  • To separate the apple from the branch, the hand moves the fruit

in a pendulum motion

4 Cutkosky, M. R. (1989) On Grasp Choice, Grasp Models, and the Design of Hands for Manufacturing Tasks. IEEE Transactions on Robotics and

Automation 5 (3): 269-279.

8

slide-9
SLIDE 9

Supported By: National Institute of Food and Agriculture

‘Undersensed’ Hand Picking5

  • Are there effective methods to pick fruit that do not require fruit
  • rientation and stem location?

9

5 Davidson, J., Silwal, A., Karkee, M., Mo, C., & Zhang, Q. (2015). Hand Picking Dynamic Analysis for

Undersensed Robotic Apple Harvesting. Transactions of the ASABE. (Under review)

slide-10
SLIDE 10

Representative Hand Picking Data

Supported By: National Institute of Food and Agriculture

10

slide-11
SLIDE 11

Mechanical Design

Supported By: National Institute of Food and Agriculture

  • Custom design
  • 7 degrees of freedom
  • Modular configuration

(Dynamixel Pro actuators)

11

slide-12
SLIDE 12

End-Effector Design

Supported By: National Institute of Food and Agriculture

  • Underactuation provides shape-adaptive

grasping

  • Passively compliant joints enhance

robustness to position error & unplanned collisions6

  • Grasping is executed in an open-loop

manner

  • Fabricated with additive manufacturing

12

6 Davidson, J., Silwal, A., Karkee, M., & Mo, C. (2015). Proof-of-Concept

  • f a Robotic Apple Harvester. Robotics and Autonomous Systems. (Under

review)

slide-13
SLIDE 13

Vision System7

Supported By: National Institute of Food and Agriculture

Color CCD Camera

PMD Camcube 3.0 (ToF, 3D camera)

Identification Fusion 1 2 3 Localization Camera Rig

13

7 Silwal, A., Gongal, A., & Karkee, M. (2014). Identification of Red Apples in Field Environment with

Over-the-Row Machine Vision System. Agricultural Engineering International: Agric Eng Intl (CIGR Journal), 16(4), 66-75.

slide-14
SLIDE 14

Experimental Setup8

Supported By: National Institute of Food and Agriculture

14

8 Silwal, A., Davidson, J., Karkee, M., Mo, C., Zhang, Q., & Lewis, K. (2015). Design, Integration and Testing of a Robotic

Apple Harvester. Journal of Field Robotics. (To be submitted)

slide-15
SLIDE 15

Hardware Architecture

Supported By: National Institute of Food and Agriculture

15

slide-16
SLIDE 16

Video

Supported By: National Institute of Food and Agriculture

16

slide-17
SLIDE 17

Vision Performance

Supported By: National Institute of Food and Agriculture

Actual vs. Recovered Image

17

slide-18
SLIDE 18

Task Timing & Vision Accuracy

Supported By: National Institute of Food and Agriculture

Vision Accuracy: Total # of Images = 54 Total Fruit Manual Count: 193 Total Fruit Identified: 193 Identification Accuracy = 100% Total Fruit in Workspace = 150 Average Fruit per Image = 4 Average Vison Time per Image = 6.3 s Average Vision time per Apple = 1.7 s

18

slide-19
SLIDE 19

Picking Results

Supported By: National Institute of Food and Agriculture

  • 127 of 150 fruits attempted were picked (approximately 85%)

– 8/127 – No stems – 33/127 – Spur attached to fruit – 86/127 – Stems attached to fruit

  • Misses fall into the following five general categories

1. Poorly thinned branch (aka “fruit pendulum”) – 7 instances 2. Finger grabbed adjacent obstruction – 3 3. Position and/or calibration error – 8 4. Fruit slipped from grasp – 2 5. Previous fruit stuck in hand - 3

  • No obvious evidence of bruising
  • Ideal fruit location is 3 – 6 in away from the trellis wire

19

slide-20
SLIDE 20

Picking Time

Supported By: National Institute of Food and Agriculture

  • Mean picking time – 6.01 sec/per fruit

– 1st fruit in a cycle: 6.22 sec – Remaining fruits in a cycle: 5.84 sec

  • Each task in the picking sequence was segregated into an

individual function and timed

– Motion planning computation: 0.15 sec – Approach: 2.14 sec – Grasp: 1.5 sec – Removal: 1.23 sec – Fruit release: 1 sec

0.5 1 1.5 2 2.5 Motion Planning Approach Grasp Removal Fruit Release

Picking time Time 20

slide-21
SLIDE 21

Future Work

Supported By: National Institute of Food and Agriculture

  • Higher level decision making based on detection of

trunks and trellis wires

  • Grasp planning based on visual input
  • Tactile sensor integration for detection of stem break,

missed fruits, etc.

21

slide-22
SLIDE 22

Questions???

Supported By: National Institute of Food and Agriculture

22