GENERATING 3D FRUIT MAPS FOR MODEL-BASED ASSESSMENT OF - PowerPoint PPT Presentation

GENERATING ¡3D ¡FRUIT ¡MAPS ¡FOR ¡ � ¡MODEL-­‑BASED ¡ASSESSMENT ¡OF ¡ROBOTIC ¡ ¡ FRUIT ¡HARVESTING ¡EFFICIENCY ¡ Stavros G. Vougioukas May 21, 2014

Motivation 3

Question No. 1 4 ¨ Can we build cost-effective fruit harvesting machines for existing tree architectures?

Question No. 2 ¡ 5 ¨ How much do different training systems affect mechanized harvesting efficiency? ¡

Work-cell automation ¡ 6 STEP 1 STEP 2

Orchard harvest mechanization ¡ 7 ¨ Directly to Step 2 : Design, build, evaluate… 1968 2008

Orchard harvest mechanization 8 ¨ Directly to Step 2 : Design, build, evaluate… 1985 2012

Limitations of existing approach ¡ 9 ¨ Development cycle : (Re)design, build, evaluate ¡ ¤ Since early on, the cycle relies on field testing ¤ Costly & slow (~1 cycle/year). ¤ Funding eventually runs out… ¡ Re-design Build ¡ Experiment ¡ platform ¡ Evaluate ¡

…more limitations ¡ 10 ¨ Experimental evaluations are not readily transferable: ¡ Machines Training systems & orchard layouts

Model-based design ¡ 11 Re-design Re-design ¡ Build ¡ Experiment ¡ Machine & orchard ¡ Evaluate ¡

‘Digital harvesting’ ¡ 12 Tree training system & orchard layout Design tool ¡ ¡ ¡ 3D fruit distributions ¡ Worker/robot kinematics Machine kinematics

Estimate 3D fruit distributions 13 φ ρ ϕ f ( , , ) h h ρ

Measuring fruit locations on trees ¡ 14 ¨ Very few attempts documented ¤ 1966: Citrus; Schertz & Brown ¤ 2006: Citrus; Lee & Rosa n String & plumb bob ¤ 1991: Citrus; Edan et al. n Manipulator & inverse kinematics ¤ 1994: Kiwi; Smith et al. n Surveying with theodolite ¨ Measurement rates < 1fruit/minute. ¡

New approach ¡ 15 ¨ Track picker’s hand position when fruit is grasped using ranging devices & trilaterate ¨ RCM400 from TimeDomain ¤ Center frequency: 4.3 GHz; Range: ~ 125 m (410 ft).

Methodology 16 16 ( ) 4 ( ) ∑ = − − + − + − * * * ˆ 2 2 2 2 ( x , y z , ) argmin r ( x bx ) ( y by ) ( z bz ) j j j ij j i j i j i x , y , z = i 1 j j j

RCM accuracy in free space 17 Range error is < 6.5 cm (95% confidence)

RCM accuracy in foliage 18 Range error is < 9.5 cm (95% confidence)

Trilateration errors 19 ¨ Geometric Dilution of Precision (GDOP). Trailer 95 th percentile (left) and mean (right) error in the fruit picking workspace.

Experimental results 20

Example: Bartlett Pears 21

Open-vase Bartlett pear trees 22

Pear yield distribution 23 Total: 7737 Average: 516 fruits per tree. Standard deviation, σ = 92.6 fruits.

Pear angular distribution 24 ρ H max max ∫ ∫ ϕ = ρ ϕ ρ ≈ α a ( ) f ( , , ) h d dh = ρ = h 0 0 ρ ϕ ≈ α ρ f ( , , ) h f ( , ) h d

Pear radial vs. height distribution 25 (m) ρ f ( , ) h d (m)

Pear height distribution ¡ 26 ρ π 2 = ∫ ∫ max (m) ρ ϕ ρ ϕ H h ( ) f ( , , ) h d d ϕ = ρ = 0 0

Pear radial distribution 27 π H 2 = ∫ ∫ max ρ ρ ϕ ϕ r ( ) f ( , , ) h d dh = = ϕ 0 h 0 (m)

High-density cling-peach trees ¡ 28

High-density cling-peach trees ¡ 29 (ft) (ft)

High-density cling-peach trees 30 Distance of fruits from trunk axis (ft)

High-density cling-peach trees 31 (ft)

Work in progress: Tree digitization and modeling 32

Next steps 33 ¨ Integration of tree models and fruits.

How can we use this? 34 Virtual fruit tree harvesting

Performance analysis and design ¡ 35 ¨ Picking efficiency; ¨ Picking throughput.

Harvesting simulations: Open-vase trees 36 q Robotic picking at high speeds will be difficult ; q Arms with reach of 8-10 ft would be too massive to be fast enough; severe branch interference; q Simulator will explore alternative multi-arm designs.

Harvesting simulations: High-density trees 37 ¨ Robot arms with reach of ~ 3ft can be fast (~ 1 reach-retrieve/s).

Design Issues 38 ¤ Could actuator arrays achieve high picking efficiency and speed? ¤ How many arms (~ $30k/arm)? ¤ What configuration? ¤ What sizes/work envelopes? ¤ How much do branches interfere?

What could the future bring? Machine Physical Virtual Model ¡ Build ¡ design ¡ Machine ¡ machine ¡ Field Testing ¡ Cultivation/ Physical Breeding ¡ training ¡ plants ¡ • Functional-structural plant models.

40 THANK YOU! ¡ Acknowledgements: ¡ ¡ Ø Co-­‑Pis ¡ o David ¡Slaughter ¡ o Fadi ¡Fathallah ¡ Ø Numerous ¡California ¡growers. ¡ ¡ Ø Farm ¡advisors: ¡ o Rachel ¡Elkins, ¡UCANR ¡Extension, ¡ ¡Lake ¡and ¡Mendocino ¡CounFes ¡ o Roger ¡Duncan, ¡UCANR ¡Extension, ¡Stanislaus ¡County ¡ o Janine ¡Hasey, ¡UC ¡Extension, ¡SuJer ¡& ¡Yuba ¡CounFes ¡ o Chuck ¡Ingels, ¡UCANR ¡Extension, ¡Sacramento ¡County ¡ ¡ Ø Students: ¡ Ø Jason ¡Wong, ¡Farangis ¡Khosro ¡Anjom, ¡Raj ¡Rajkishan. ¡ ¡ svougioukas@ucdavis.edu ¡