Bringing Aerial Robots Closer to Crops: Sensing, Sampling, and Safety - PowerPoint PPT Presentation

Bringing Aerial Robots Closer to Crops: Sensing, Sampling, and Safety nimbus.unl.edu Carrick Detweiler Computer Science and Engineering Department University of Nebraska‐Lincoln carrick@cse.unl.edu cse.unl.edu/~carrick nimbus.unl.edu

What are UAVs used for?

What are UAVs used for?

What are UAVs used for?

youtu.be/nm8MhcBkYDw Use Cases: Fly High

youtu.be/tFi8YbE9M6k Use Cases: Fly Low

Collect Samples?

Vision, CapabiliMes, and Requirements • Fly close to the environment • Interact with environment • Autonomy to increase success • Reliability to lower cost • Safety to enable adopMon

Talk Overview • Crop height esMmaMon & row following – MoMvaMon – System design – System verificaMon • Other Nimbus Lab projects – Aerial water sampling – Improving safety of robots – Wireless power transfer

MoMvaMon • Phenotyping trials – Many varieMes of plants – Aim: Improve understanding of water and nitrogen stressors during phenotyping trials • Measure crop height • AcMve Sensors • Research Challenges – Fly close (<1m) to crops – Follow rows autonomously Student: D. Anthony; Collaborators: S. Elbaum, A. Lorenz and R. Ferguson

Why Crop Height? • Crop height is a predictor of – Plant health – Stress – Yield • Rarely collected in agronomy research • Almost never used commercially

Crop Height Approach • Use small UAVs with laser scanner • Fly close – Be_er angles – Less expensive sensors • Challenges – Noisy scan data – Unstructured environment – Changing field condiMons – Near crop operaMon

System • Ascending Technologies Firefly • Hex‐rotor UAV • 600g payload • Hokuyo laser scanner – 5.6m, 36Hz Stalk • Onboard processing • Camera Leaves

Laser Scan Example

Crop Height Approach Frame • Noisy scans IMU CDF Xform • Individual plants have defined structure Ground Crop Laser • Reject noisy outliers Est. Est. using median filters • Kalman filter fuses IMU Crop Median Median and laser scan data Filter Height Filter w c w g Est. • Control based on ground distance and plant top distance PID Kalman Filter Control

Crop Height Measurement

1 Canopy 0.9 0.8 Crop Height Results 0.7 Ground 0.6 0.5 0.4 0.3 0.2 • Can idenMfy ground, Single Scan 0.1 Combined 0 0 1 2 3 4 5 top of crops, and Distance (m) 100 Estimated Crop Height (m) 90 2.5 True Height 80 intermediate canopy levels Upper Percentile 2 70 60 1.5 50 • Plan height within 3cm of 40 1 30 20 manual measurements 0.5 10 0 9 18 27 36 45 54 63 72 81 90 99 Lower Percentile • Autonomous height control • Challenges – Cannot follow rows with GPS

From Height EsMmaMon to Row Following • GPS accuracy poor • RTK GPS heavy, requires setup • 1‐2 rows in each phenotyping trial • Isolate rows from scan data • Exploit known informaMon about corn field 1 Crop Tops 1.5 Scan Range (m) 2 2.5 3 3.5 Ground 4 − 0.75 − 0.5 − 0.25 0 0.25 0.5 0.75 Scan Angle (rad)

Row LocalizaMon Algorithm 0 -1 -2 Z (m) -3 -4 -5 -6 -2 0 2 X (m) 0 0 -1 -1 -2 -2 Z (m) Z (m) -3 -3 -4 -4 -5 -5 -6 -6 -3 -2 -1 0 1 2 3 -3 -2 -1 0 1 2 3 X (m) X (m)

Row Following Results • Developed this past growing season (last tests 1 weeks ago) • Automated height control and row following 140 Feature 120 Particle filter • Stable in 10knt (5m/s) 100 winds Time (s) 80 • Comparison to ground 60 truth (video) in progress 40 20 0 -4 -3 -2 -1 0 1 2 3 4 Distance (m)

Future Work • Comparison to ground truth • EvaluaMon with different: – Growth stages – Winds – LighMng – Other row crops • Switching between rows • Measuring whole field

Talk Overview • Crop height esMmaMon & row following – MoMvaMon – System design – System verificaMon • Other Nimbus Lab projects – Aerial water sampling – Improving safety of robots – Wireless power transfer

UAV Water Sampling InteracMng with Environment • Goals – Collect water samples with UAV – ParMal Autonomy • Research Challenges – Staying safe and dry – IntegraMng ultrasound, alMmeter, and water sensor data – Easy and robust user interface Students: John‐Paul Ore, J Higgins Collaborator: S. Elbaum, A. Burgin, M. Hamilton, S. Thompson

Why Water Sampling?

Why Water Sampling? 19 May 2012 18 May 2012

How is it Done Now?

How is it Done Now? Grab Sampling

How is it Done Now? Fixed Samplers

How is it Done Now? Autonomous Systems J. Manley, A. Marsh, W. Cornforth, and C. Wiseman, “EvoluMon • of the Autonomous Surface Cral AutoCat,” Proceedings of Oceans 2000, MTS/IEEE Providence, RI, October, 2000. H. Ferreira, A. MarMns, A. Dias, C. Almeida, J. M. Almeida, E. P. • Silva, “ROAZ Autonomous Surface Vehicle Design and ImplementaMon”, Encontro Cienofico ‐ RobóMca 2006, Pavilhão MulM‐usos, Guimarães, Portugal, 28 Abril, 2006 R. Hine and P. McGillivary, “Wave powered autonomous surface • vessels as components of ocean observing systems,” Proceedings of PACON 2007, Honolulu, HI June 2007 Curcio, Joseph, John Leonard, and Andrew Patrikalakis. "SCOUT ‐ • A low cost autonomous surface platorm for research in cooperaMve autonomy." OCEANS, 2005. Proceedings of MTS IEEE. IEEE, 2005.

Limnologist Requirements

Limnologist Requirements • Three 20 ml water samples within 1 kilometer • Small and light enough for one scienMst • Reliable, cost‐effecMve, and safe • Sampler must not bias water properMes

Challenges • Altitude over water while sampling • Acquire and transport water • Avoid cross- contamination • Flying in wind • Ensuring safety • Autonomy

How Do We Sample Water? • UAV – AscTec Firefly • Electromechanical • Pump • Conductivity Sensors • Breakaway Mechanism • ‘Chassis’ & ‘Needle’ • Embedded System • Software • Altitude Control • Safety System • Autonomy

youtu.be/7mPbyXZpBws Autonomous Aerial Water Sampling

Micropump (10 grams)

Chassis, Servo, Flushing, Vials

Ultrasonic Sensors

ConducMvity Sensors ConducMvity Sensors

Experimental ValidaMon • Sample collecMon success rate • Comparison to manual methods • OperaMon in wind

OperaMon in Wind 100 90 Full Sampling Success Rate (%) Target 80 Sampling 70 AlRtude 0.72m 60 0.82m 50 0.92m 40 1.02m 30 1.12m 20 10 0 0‐2.7 2.7‐3.5 3.5‐4.5 4.5‐5.3 5.3+ Wind Speed (m/s) Total of 225 samples, at least 4 per data point

Methods Comparison: Temperature Transects • Adjustable length 4m tube with temp sensor • StaMc array vs. UAV • UC Berkeley Blue Oak Ranch Reserve

youtu.be/gL‐MahPaeCo Offu_ Air force Base Lake, Nebraska Zebra Mussel Veliger Sampling

Aerial Water Sampling • Findings – Successfully captured 100s of samples indoors – 100+ outdoors – Stable in 10‐15mph winds – Samples compare to grab samples • Uses – Sampling of hard to access locaMons – Temperature mapping – Invasive species (eDNA?) – Add conducMvity, Temp, DO, etc. – Chemical spills – Others?

Failures: Part of Field RoboMcs

youtu.be/rd04BMkHKcQ Failures: Part of Field RoboMcs

Improving Safety, Autonomy, and Reliability • Goals – Operate reliably under unpredictable condiMons – Avoid operaMons that may lead to loss of control • Research Challenges – Characterize known successful system operaMons – Synthesize those operaMons into a monitor that bounds UAV behavior Student: Hengle Jiang Collaborator: S. Elbaum

Failures in roboMcs • Hardware – Failed sensors or actuators – Intermi_ent errors • Environment condiMons – Wind, lighMng, obstacles, etc. • Users – Inpuyng incorrect commands – Do not understand system limits • Solware – Bugs – CommunicaMon/processing limits

youtu.be/VIAg8dM8TLk Improving Safety, Autonomy, and Reliability

Approach • Most faults show up as abnormal behavior in solware • Our approach: Inferred invariant monitoring – Implemented in ROS Program Invariant Monitor Traces Inference Synthesis Increased Modify Online Safety Behavior Monitoring

Improving Safety, Autonomy, and Reliability • Findings – Success rate for 7 unplanned Without Monitor scenarios increased 20% to 80% – Approach is applicable to any UAV task With Monitor – Need to idenMfy remedial acMons – Required for close interacMons with environment

Wireless Power Increased CapabiliMes • Goals – Collect data from sensors – Charge hard to access sensors – Place/retrieve sensors • Research Challenges – OpMmize power transfer and efficiency – Localize nodes to charge – Select nodes to recharge Students: B. Griffin, A. Mi_leider, J. Leng, N. Najeeb

Achievements: Wireless Power Increased CapabiliMes

Wireless Power Increased CapabiliMes • Findings – Resonant magneMc coupling extends range of transfer – Can transfer 10W – Can collect data while charging sensors – Removes requirement for large solar panels