Development of Control Architectures for Multi-Robot Agricultural - - PowerPoint PPT Presentation

▶
development of control architectures
SMART_READER_LITE
LIVE PREVIEW

Development of Control Architectures for Multi-Robot Agricultural - - PowerPoint PPT Presentation

Apr 08, 2024 326 likes •756 views

Development of Control Architectures for Multi-Robot Agricultural Field Production Systems Santosh K. Pitla, Ph.D. spitla2@unl.edu Assistant Professor Department of Biological Systems Engineering University of Nebraska-Lincoln IEEE RAS

slide-1
SLIDE 1

Development of Control Architectures for Multi-Robot Agricultural Field Production Systems

Santosh K. Pitla, Ph.D. spitla2@unl.edu Assistant Professor Department of Biological Systems Engineering University of Nebraska-Lincoln

1

IEEE RAS Agricultural Robotics & Automation Technical Committee

Webinar #013 December 2013

slide-2
SLIDE 2

Overview

  • Education and Research Background
  • Control Architectures
  • Individual Robot Control Architecture (IRCA)
  • Multi Robot System Control Architecture (MRSCA)
  • Autonomous Vehicle Platform (AVP) Development
  • Validation of Control Architectures
  • Post-Doctoral Research
  • Future Work at UNL
  • Q&A

2

slide-3
SLIDE 3

Education and Research Background

  • 2000 to 2004 – BS, Mechanical Engineering, Osmania University,

Hyderabad, India

  • Aug 2004 to May 2007 - MS, Mechanical Engineering and Biosystems and

Agricultural Engineering University of Kentucky, Lexington, KY

Thesis Title: Development of an Electro-Mechanical System to Identify and Map Soil Compaction

  • March 2007 to April 2012 – Research Engineer, Machine Systems

and Automation, BAE, University of Kentucky

  • Jan 2008 to Jan 2012- Ph.D. Program
  • May 2012 to Present, Post-Doc, The Ohio State University
  • OCT 2013 to Present, Assistant Professor and Advanced

Machinery Engineer, University of Nebraska-Lincoln

3

slide-4
SLIDE 4

Control Architectures

4

Weeding Robot (Madsen and Jakobsen, 2001) Autonomous Harvester (Pilarski et al., 2002) Citrus Fruit Harvesting Robot (Hannan et al., 2004)

Mato Grosso,Brazil

  • Un-manned agricultural machines
  • Road blocks
  • Cost
  • Safety
  • Intelligence
  • Control architecture work is underway

(Brooks, 1986; Arkin, 1990; Yavuz and Bradshaw, 2002; Blackmore et al., 2002; Torrie et al., 2002; and Mott et al., 2009)

  • Individual Robot Control Architecture
  • Multi-Robot System Control Architecture

Control Architectures Agricultural Machines Agricultural Robots

slide-5
SLIDE 5

Paradigm Shift (Big Vs. Small)

slide-6
SLIDE 6

Paradigm Shift (Big Vs. Small)

6

Spray material Application Variation (Luck and Pitla , 2010)

slide-7
SLIDE 7

Individual Robot Control Architecture (IRCA)

7

Individual Robot Control Architecture (IRCA)

Intelligence

slide-8
SLIDE 8

IRCA (continued)

  • Sensing Layer (SL)
  • Sensor Stack

Array of sensors that aid the robot in learning about the unknown environment

  • Wireless Communication Module (WCM)

Processes the information obtained wirelessly from remote or off-machine entities

The SL receives the environment data obtained from the sensor stack (on-machine) and the WCM (off-machine) and passes the information to the BL for further processing and filtering

slide-9
SLIDE 9

IRCA (continued)

  • Behavior Layer (BL)
  • Deliberative Behavior

High level decision making processes that require planning and algorithm execution

  • Reactive Behavior

Low level processes that do not require considerable computation but are crucial for safety and operability

  • By-Pass of Control

The switching of the control command generation source in response to the changing environment

slide-10
SLIDE 10

Individual Robot Control Architecture (IRCA)

10

FSM I Desired Control Commands FSM II FSM III FSM IV

slide-11
SLIDE 11

IRCA (continued)

FSML Simulation (MATLAB)

Inputs Scenario I Scenario II Scenario III Scenario IV Scenario IV dob (m) 5 5 3 5 5 TuF 1 1 SAd (deg) ±0.125 ±2.5

  • ±0.125

±0.125 SAr (deg)

  • ± 5
  • Eflag

1 ReF TrF

Five scenarios for FSM simulation

Input signals created using Signal Builder (MATLAB) corresponding to the five scenarios 11

slide-12
SLIDE 12

FSML created using the StateFlowChart tool

IRCA (continued)

FSML simulation

SIMULINK model created for FSML simulation

Internal States Trigger conditions

FSM I Cruise TuF~=1 && dob>=4 && Eflag ~=1 Slow TuF==1 && dob>=4 Safe Speed TuF>=0 && dob<4 && dob>=2 Dead dob<2 || Eflag==1 FSM II Navigate TuF>=0 && dob>=4 Safe Navigate TuF>=0 && dob<4 FSM III Lower (1) ReF~=1 && Eflag~=1&&TrF~=1&& dob>=4 Raise (0) ReF==1 || Eflag==1|| TrF==1|| dob<4 FSM IV ON (1) ReF~=1&&Eflag~=1&&TrF~= 1&&dob>=4 OFF (0) ReF==1||Eflag==1||TrF==1||d

  • b<4

12

Mutually Exclusive States Parallel States

slide-13
SLIDE 13

IRCA (continued)

FSM Simulation Results and Discussion

Active states of FSML (I to IV) in the default state Actives states in Scenario I Actives states in Scenario III 13

slide-14
SLIDE 14

IRCA (continued)

FSM Simulation Results and Discussion

Active states of FSML (I to IV) in Scenario I

FSM simulation outputs (desired control commands)

Scenarios Simulation time (s) Active Internal States (FSM I to IV) Desired Control Commands Scenario I 0-10 Cruise, Navigate, Lower, ON 4 km/h, ±0.125o, 1, 1 Scenario II 10-20 Slow, Navigate, Lower, ON 2 km/h, ± 2.5 o, 1, 1 Scenario III 20-30 Safe Speed, Safe Navigate, Raise, OFF 1 km/h, ± 5o, 0, 0 Scenario IV 30-40 Cruise, Navigate, Lower, ON 4 km/h, ±0.125o, 1, 1 Scenario V 40-50 Dead, Navigate, Raise, OFF 0 km/h, ±0.125o, 0, 0

14

slide-15
SLIDE 15

Multi-Robot System Control Architecture (MRSCA)

15

MRS planting in unique work zones (WZ I to WZ III)

No Cooperation Coordination Strategy (No Cooperation) Control Variables (m, r, c) Stand Alone Behavior (0,0,0)

Control Variables:

  • m = Mode (values: 0,1,2)
  • r = Role (values: 0,1,2)
  • c = Communication (values: 0,1)

No Cooperation Mode Stand Alone Role Transmit

slide-16
SLIDE 16

MRSCA (continued)

16

MRS performing baling and retrieval operations

Modest Cooperation

Coordination Strategy (Modest Cooperation) Control Variables (m, r, c) Leader (Baler) (1,1,0) Follower (Bale Spear) (1,2, 1)

slide-17
SLIDE 17

MRSCA (continued)

17

MRS performing Harvesting Operation

Absolute Cooperation

Coordination Strategy (Absolute Cooperation) Control Variables (m, r, c) Leader (Combine) (2,1, 0) Follower (GC-I, GC-II) (2,2, 1)

slide-18
SLIDE 18

MRSCA (continued)

18

slide-19
SLIDE 19

MRSCA (continued)

19

Coordination Strategy Control Variables (m, r, c) No Cooperation (0,0,0) Modest Cooperation (1,1,0), (1,2, 1) Absolute Cooperation (2,1, 0), (2,2, 1)

Global Information Module

slide-20
SLIDE 20

MRSCA (continued)

20

GIM

Role Coordination Wireless Communication StandAlone Leader Follower No Cooperation Modest Cooperation Absolute Cooperation Tx Rx Wait Unload Go To/ Retrieve Follow/ Load Unload Parallel Finite States Internal Finite States – Level I Internal Finite States – Level II

Hierarchy of the FSMs in GIM.

slide-21
SLIDE 21

MRSCA (continued)

Status Flag Definition High Low SA Raised when role of the robot is Stand Alone 1 L1 Raised when the robot is performing a leader role with the task Wait is active 1 L2 Raised when the robot is performing a leader role with the Unload task active 1 F1 Raised when the robot is performing a follower role and the task Goto is active 1 F2 Raised when the robot is performing a follower role and the task Follow/Load is active 1 F3 Raised when the robot is performing a follower role and the task Unload is active 1

21 Active States during Simulation in Stand Alone Mode

slide-22
SLIDE 22

MRSCA (Continued)

External Wireless Input

StatMsgF.mat To File1 StatMsgL.mat To File

Scope Leader Scope Follower

m r c BL

Inputs-IRCA (Leader)

m r c BD FE BW

Inputs-IRCA (Follower)

m r c BL SA L1 L2 F1 F2 F3

Global Information Module Baler (Leader)

m r c BD FE BW BLf SA L1 L2 F1 F2 F3

Global Information Module Bale Retriever (Follower)

SA L1 L2 F1 F2 F3 SAf L1f L2f F1f F2f F3f BL

22 Generic Flags Definition High Low BL, BLf Raised when the bale is ready to be retrieved 1 BD Raised when the Bale Retriever is closer to the hay bale to be retrieved 1 FE Raised when the Bale Retriever is close to the field edge for dropping of the bale 1 BW Raised when the bale is loaded on the Bale Retriever 1

slide-23
SLIDE 23

MRSCA (Continued)

Modest Cooperation (Baling – Bale Retrieving)

23

(a) (b) (c)

Active states of the Baler (Leader) Active tasks a) Go To; b) Load; and c) Unload of the Bale Retriever during the execution of the bale retrieving task

slide-24
SLIDE 24

MRSCA (Continued)

External Wireless Input (Follower 1) External Wireless Input (Follower 2)

StatMsgF2.mat To File2 StatMsgF1.mat To File1 StatMsgL.mat To File

Scope Leader Scope Follower2 Scope Follower1

m r c

  • d

PS FE

InputsF2 (IRCA )

m r c

  • d

PS FE

InputsF1 (IRCA)

m r c PS

Inputs (IRCA)

m r c PS SYNCL2 SYNCL1 SA L1 L2 F1 F2 F3

Global Information Module Leader

m r c

  • d

PS FE SA L1 L2 F1 F2 F3 SYNCF2

Global Information Module Follower2

m r c

  • d

PS FE SA L1 L2 F1 F2 F3 SYNCF1

Global Information Module Follower1

SA L1 L2 F1 F2 F3 SAf1 L1f1 L2f1 F1f1 F2f1 F3f1 Sf1 Sf1 SAf2 L1f2 L2f2 F1f2 F2f2 F3f2 Sf2 Sf2

Absolute Cooperation (Harvesting)

24 Flags Definition High Low PS Raised when the Grain Carts are full with grain or when the grain is available in the Combine for unloading 1 OD Raised when the Grain Cart is at a desired bearing (heading and location) relative to the Combine 1 SYNCF1/S YNCL1 Raised when Grain Cart I wants to synchronize with the Combine for the transfer of grain 1 SYNCF2/S YNCL2 Raised when Grain Cart II wants to synchronize with the Combine for the transfer of grain 1

slide-25
SLIDE 25

MRSCA (Continued)

25

slide-26
SLIDE 26

MRSCA (Continued)

26

(a) (b)

slide-27
SLIDE 27

MRSCA (Continued)

27

slide-28
SLIDE 28

Autonomous Vehicle Platform (AVP) Development

28

AVP framework: (a) solid model of basic frame, (b) fabricated AVP frame with mechanical components

Drive Motor Differential Flexible Coupling (a) (b) (c)

(a) Drive motor mounting and roller chain drive to differential at rear axle (b) 24 VDC steering actuator, (c) steering actuator and linkage mounted on the front axle of the AVP Ground speed sensor: (a) schematic and wiring diagram and (b) actual mounting location and configuration.

Infra-Red Sensor Orthogonal Steel Plate Oscillating DC Motor

(a) (b) (c) (d)

Infra-red sensor array construction: a) solid model, b) SHARP GP2Y0A700K NIR sensor, c) OEM 212 series oscillating motor drive, d) assembled sensor array mounted on the AVP

slide-29
SLIDE 29

AVP Development (continued)

MC012-010 Plus+1 microcontroller (Sauer Danfoss, MN)

Speed Controller Steering Controller System Controller IRF Controller IRR Controller CAN Bus Terminator Terminator IXF Module IXR Module

AVP distributed controller network topology Leaf Light HS (Kvaser, Sweden) CAN to RS232 gateway 9XTend PKG (B&B Electronics Manufacturing Co., Ottawa, IL) radio frequency modem Trimble AgGPS 132 (Trimble Navigation Ltd., Sunnyvale, CA) GPS engine and antenna

Task Computer for the AVP (Eee PC 1000, ASUSTeK Computer, Inc., Peitou, Taipei, Taiwan, R.O.C) 29

slide-30
SLIDE 30

AVP Development (continued)

50 A Relay Manual

24 V

Key Switch Steering Driver M+ M- V+ V- S + - Speed Driver M+ M- V+ V- S + - Auto

Fuse Block

10 A Signal Manual Signal Auto 30 A 5 A 5 A CAN PWR Wireless module FWD Bkwd Left Right Stop Stop System Controller Inputs 4.4 KΩ 4.4 KΩ Speed Controller Steering Controller

(a) (b)

AVP PDP (a) schematic of PDP, and (b) completed PDP installed on AVP 30

RF Modem GPS RS 232 RS 232 CAN to Serial Converter RS 232 Terminator Task Computer Drive Motor Steering Actuator IR Sensor Front IR Sensor Rear Speed Controller Steering Controller System Controller IRF Controller IRL Controller CAN Bus Terminator

Component Map of the AVP

VB.Net User Interface

GUIDE software used for programming the Micro Controllers

slide-31
SLIDE 31

AVP Development (continued)

31

Multi-Robot System (MRS) created by replicating the AVP

  • Completely electric - 24 VDC
  • Operates in Manual and Automatic modes
  • On-board rechargers for Lead-Acid Batteries
  • CAN based distributed controller network
  • Ability to override the automatic mode via manual commands in the

event of emergency

  • Three way AVP safety – 1) IR sensory arrays, 2) onboard emergency

stops, and 3) wireless remote stop.

  • Task computer, GPS and wireless communications for automatic
  • peration.
slide-32
SLIDE 32

Validation of IRCA

Deliberative and Reactive Behavior

32

20 21 22 23 24 25 26 27 28 29 30 5 10 15 20 25 30 35 40 Easting (m) Northing (m) AB Line Robot Path Obstacle A B

AVP tracking of AB line with obstacle in its path Speed States of the AVP

slide-33
SLIDE 33

Validation of MRSCA

Standalone Behavior

33

Automatic tracking of AB lines by three AVPs simulating planters.

slide-34
SLIDE 34

Validation of MRSCA (continued)

Modest Cooperation

34

20 22 24 26 28 30 32 5 10 15 20 25 30 35 40 45 Easting (m) Northing (m) Baler Path Bale Retriever Path A C B D F

Bale drop off location Field Edge

Motion paths of Baler and Bale Retriever AVPs

slide-35
SLIDE 35

Validation of MRSCA (continued)

Absolute Cooperation

35 Paths of Harvester, GC1 and GC2 AVPs during TS3.

slide-36
SLIDE 36

Post-Doctoral Research

36

Next Gen Autonomous Plat Form (AVP – II)

slide-37
SLIDE 37

Post-Doctoral Research

37

Controller Area Network (CAN) Data Acquisition from Field Machinery

ISO 11783 (Source: www.vector.com)

Anhydrous Applicator 16 row planter 12 row planter Sprayer

Data Acquisition from the ISO Diagnostic Port (Tractor: MX340)

slide-38
SLIDE 38

Post-Doctoral Research

38

CAN Data Acquisition

Screenshot of Vector CANalyzer Interface (Data collection from CASE IH MX340 Tractor) Decoded GPS CAN Data GPS CAN Message Time ID Data length D0 D1 D2 D3 D4 D5 D6 D7

0.044144 1 18FEF31Cx Rx d 8 E2 C8 3 95 F0 ED AE 4B

slide-39
SLIDE 39

Post-Doctoral Research

39

CAN Data Acquisition

Work Period Turning Dwell Period

slide-40
SLIDE 40

Future Work at UNL

Machine Automation and Agricultural Robotics for Row Crop and Bio Energy Production

40 Moving Biomass Bales from the field to On-Farm Bio Mass Processing Facility Spreading the by-product (fertilizer) in the field Soil Sampling, selective spraying, Weed Mapping, Crop Health MRS Planting or Spraying UAV (Source: www.Precision Hawk.com) AR Drone 2.0 (www.ardrone.parrot.com/)

slide-41
SLIDE 41

Thank You!! Questions?

42