Bringing Aerial Robots Closer to Crops: Sensing, Sampling, and Safety - - PowerPoint PPT Presentation

carrick@cse.unl.edu cse.unl.edu/~carrick nimbus.unl.edu

Bringing Aerial Robots Closer to Crops: Sensing, Sampling, and Safety nimbus.unl.edu

Carrick Detweiler Computer Science and Engineering Department University of Nebraska‐Lincoln

What are UAVs used for?

What are UAVs used for?

What are UAVs used for?

Use Cases: Fly High

youtu.be/nm8MhcBkYDw

Use Cases: Fly Low

youtu.be/tFi8YbE9M6k

Collect Samples?

Vision, CapabiliMes, and Requirements

• Fly close to the environment
• Interact with environment
• Autonomy to increase success
• Reliability to lower cost
• Safety to enable adopMon

Talk Overview

• Crop height esMmaMon & row following

– MoMvaMon – System design – System verificaMon

• Other Nimbus Lab projects

– Aerial water sampling – Improving safety of robots – Wireless power transfer

MoMvaMon

Student: D. Anthony; Collaborators: S. Elbaum, A. Lorenz and R. Ferguson

• Phenotyping trials

– Many varieMes of plants – Aim: Improve understanding of water and nitrogen stressors during phenotyping trials

• Measure crop height
• AcMve Sensors
• Research Challenges

– Fly close (<1m) to crops – Follow rows autonomously

Why Crop Height?

• Crop height is a predictor of

– Plant health – Stress – Yield

• Rarely collected in

agronomy research

• Almost never used

commercially

• Use small UAVs with

laser scanner

• Fly close

– Be_er angles – Less expensive sensors

• Challenges

– Noisy scan data – Unstructured environment – Changing field condiMons – Near crop operaMon

Crop Height Approach

System

• Ascending Technologies

Firefly

• Hex‐rotor UAV
• 600g payload
• Hokuyo laser scanner

– 5.6m, 36Hz

• Onboard processing
• Camera

Stalk Leaves

Laser Scan Example

Crop Height Approach

• Noisy scans
• Individual plants have

defined structure

• Reject noisy outliers

using median filters

• Kalman filter fuses IMU

and laser scan data

• Control based on

ground distance and plant top distance

Laser IMU Frame Xform CDF Crop Est.

Ground Est. Median Filter wc Median

Filter wg

Kalman Filter

Crop Height Est. PID Control

Crop Height Measurement

• Can idenMfy ground,

top of crops, and intermediate canopy levels

• Plan height within 3cm of

manual measurements

• Autonomous height control
• Challenges

– Cannot follow rows with GPS

1 2 3 4 5 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Distance (m)

Single Scan Combined

Canopy Ground

9 18 27 36 45 54 63 72 81 90 99 10 20 30 40 50 60 70 80 90 100

Lower Percentile Upper Percentile Estimated Crop Height (m)

0.5 1 1.5 2 2.5 True Height

Crop Height Results

• GPS accuracy poor
• RTK GPS heavy, requires setup
• 1‐2 rows in each phenotyping trial
• Isolate rows from scan data
• Exploit known informaMon about

corn field

From Height EsMmaMon to Row Following

−0.75 −0.5 −0.25 0.25 0.5 0.75 1 1.5 2 2.5 3 3.5 4

Scan Angle (rad) Scan Range (m)

Crop Tops Ground

Row LocalizaMon Algorithm

X (m)

• 2

2

Z (m)

• 6
• 5
• 4
• 3
• 2
• 1

X (m)

• 3
• 2
• 1

1 2 3

Z (m)

• 6
• 5
• 4
• 3
• 2
• 1

X (m)

• 3
• 2
• 1

1 2 3

Z (m)

• 6
• 5
• 4
• 3
• 2
• 1

Row Following Results

• Developed this past

growing season (last tests 1 weeks ago)

• Automated height control

and row following

• Stable in 10knt (5m/s)

winds

• Comparison to ground

truth (video) in progress

Distance (m)

• 4
• 3
• 2
• 1

1 2 3 4

Time (s)

20 40 60 80 100 120 140

Feature Particle filter

Future Work

• Comparison to ground truth
• EvaluaMon with different:

– Growth stages – Winds – LighMng – Other row crops

• Switching between rows
• Measuring whole field

Talk Overview

• Crop height esMmaMon & row following

– MoMvaMon – System design – System verificaMon

• Other Nimbus Lab projects

– Aerial water sampling – Improving safety of robots – Wireless power transfer

UAV Water Sampling InteracMng with Environment

• Goals

– Collect water samples with UAV – ParMal Autonomy

• Research Challenges

– Staying safe and dry – IntegraMng ultrasound, alMmeter, and water sensor data – Easy and robust user interface

Students: John‐Paul Ore, J Higgins Collaborator: S. Elbaum, A. Burgin, M. Hamilton, S. Thompson

Why Water Sampling?

18 May 2012 19 May 2012

Why Water Sampling?

How is it Done Now?

How is it Done Now? Grab Sampling

How is it Done Now? Fixed Samplers

How is it Done Now? Autonomous Systems

• J. Manley, A. Marsh, W. Cornforth, and C. Wiseman, “EvoluMon
• f the Autonomous Surface Cral AutoCat,” Proceedings of

Oceans 2000, MTS/IEEE Providence, RI, October, 2000.

• H. Ferreira, A. MarMns, A. Dias, C. Almeida, J. M. Almeida, E. P.

Silva, “ROAZ Autonomous Surface Vehicle Design and ImplementaMon”, Encontro Cienofico ‐ RobóMca 2006, Pavilhão MulM‐usos, Guimarães, Portugal, 28 Abril, 2006

• R. Hine and P. McGillivary, “Wave powered autonomous surface

vessels as components of ocean observing systems,” Proceedings of PACON 2007, Honolulu, HI June 2007

• Curcio, Joseph, John Leonard, and Andrew Patrikalakis. "SCOUT ‐

A low cost autonomous surface platorm for research in cooperaMve autonomy." OCEANS, 2005. Proceedings of MTS

• IEEE. IEEE, 2005.

Limnologist Requirements

Limnologist Requirements

• Three 20 ml water samples

within 1 kilometer

• Small and light enough for
• ne scienMst
• Reliable, cost‐effecMve, and

safe

• Sampler must not bias water

properMes

Challenges

• Altitude over water

while sampling

• Acquire and transport

water

• Avoid cross-

contamination

• Flying in wind
• Ensuring safety
• Autonomy

How Do We Sample Water?

• UAV – AscTec Firefly
• Electromechanical
• Pump
• Conductivity Sensors
• Breakaway Mechanism
• ‘Chassis’ & ‘Needle’
• Embedded System
• Software
• Altitude Control
• Safety System
• Autonomy

Autonomous Aerial Water Sampling

youtu.be/7mPbyXZpBws

Micropump (10 grams)

Chassis, Servo, Flushing, Vials

Ultrasonic Sensors

ConducMvity Sensors ConducMvity Sensors

Experimental ValidaMon

• Sample collecMon success rate
• Comparison to manual methods
• OperaMon in wind

OperaMon in Wind

10 20 30 40 50 60 70 80 90 100 0‐2.7 2.7‐3.5 3.5‐4.5 4.5‐5.3 5.3+ Full Sampling Success Rate (%) Wind Speed (m/s) 0.72m 0.82m 0.92m 1.02m 1.12m Target Sampling AlRtude Total of 225 samples, at least 4 per data point

Methods Comparison: Temperature Transects

• Adjustable length 4m tube with temp sensor
• StaMc array vs. UAV
• UC Berkeley Blue Oak Ranch Reserve

Offu_ Air force Base Lake, Nebraska Zebra Mussel Veliger Sampling

youtu.be/gL‐MahPaeCo

Aerial Water Sampling

• Findings

– Successfully captured 100s of samples indoors – 100+ outdoors – Stable in 10‐15mph winds – Samples compare to grab samples

• Uses

– Sampling of hard to access locaMons – Temperature mapping – Invasive species (eDNA?) – Add conducMvity, Temp, DO, etc. – Chemical spills – Others?

Failures: Part of Field RoboMcs

Failures: Part of Field RoboMcs

youtu.be/rd04BMkHKcQ

Improving Safety, Autonomy, and Reliability

• Goals

– Operate reliably under unpredictable condiMons – Avoid operaMons that may lead to loss of control

• Research Challenges

– Characterize known successful system operaMons – Synthesize those operaMons into a monitor that bounds UAV behavior

Student: Hengle Jiang Collaborator: S. Elbaum

Failures in roboMcs

• Hardware

– Failed sensors or actuators – Intermi_ent errors

• Environment condiMons

– Wind, lighMng, obstacles, etc.

• Users

– Inpuyng incorrect commands – Do not understand system limits

• Solware

– Bugs – CommunicaMon/processing limits

Improving Safety, Autonomy, and Reliability

youtu.be/VIAg8dM8TLk

Approach

• Most faults show up as abnormal behavior in solware
• Our approach: Inferred invariant monitoring

– Implemented in ROS Program Traces Invariant Inference Monitor Synthesis Online Monitoring Modify Behavior Increased Safety

Improving Safety, Autonomy, and Reliability

• Findings

– Success rate for 7 unplanned scenarios increased 20% to 80% – Approach is applicable to any UAV task – Need to idenMfy remedial acMons – Required for close interacMons with environment

Without Monitor With Monitor

Wireless Power Increased CapabiliMes

• Goals

– Collect data from sensors – Charge hard to access sensors – Place/retrieve sensors

• Research Challenges

– OpMmize power transfer and efficiency – Localize nodes to charge – Select nodes to recharge

Students: B. Griffin, A. Mi_leider,

• J. Leng, N. Najeeb

Achievements: Wireless Power Increased CapabiliMes

Wireless Power Increased CapabiliMes

• Findings

– Resonant magneMc coupling extends range of transfer – Can transfer 10W – Can collect data while charging sensors – Removes requirement for large solar panels

Summary

• UAVs can be used close to the ground and water

– Can already fly high and observe

• UAVs have disrupMve potenMal

– InteracMons with environment – Collect samples – Safety and reliability – Autonomy and ease of use

• Challenges

– Public percepMon (privacy, security, data access) – RegulaMons (FAA, airspace integraMon) – Fundamental challenges in roboMcs sensing and safety

NIMBUS Lab

• Nebraska Intelligent MoBile Unmanned Systems Lab

– Computer Science and Engineering Department – Grad Students, Undergrads – Co‐directed with Dr. SebasMan Elbaum

Carrick Detweiler Computer Science and Engineering Department University of Nebraska‐Lincoln

This work is supported in part by grants from NSF, USDA, Water for Food InsRtute, and ORED‐UNL. We would like to thank our collaborators:

• C. Allen, M. Brown, R. Ferguson, A. Lorenz, S. Thompson, M. Hamilton,

T.J. Fontaine, A. Burgin, and M. Waite.

Bringing Aerial Robots Closer to Crops: Sensing, Sampling, and Safety nimbus.unl.edu