Welcome Todays Agenda Welcome Remarks by President Karbhari Overview - - PowerPoint PPT Presentation
THE UNIVERSITY OF TEXAS AT ARLINGTON Forum: Controls and Uses of Autonomous Vehicles' Welcome Todays Agenda Welcome Remarks by President Karbhari Overview by Jim Grover Presentations by Frank Lewis Yan Wan Animesh Chakravarthy
THE UNIVERSITY OF TEXAS AT ARLINGTON
Forum: ‘Controls and Uses of Autonomous Vehicles'
Welcome Remarks by President Karbhari Overview by Jim Grover Presentations by ‒ Frank Lewis ‒ Yan Wan ‒ Animesh Chakravarthy ‒ Nick Gans Share your research initiatives Group Activity
Forum on Controls and Uses of Autonomous Vehicles
James P. Grover, Ph.D.
Interim Vice President for Research
http://www.ciobulletin.com/retail/amazon-prime- air-drones
Unmanned Aerial Vehicle (UAS) Unmanned Ground Vehicle (UGS)
http://theconversation.com/what-if- autonomous-vehicles-actually-make-us- more-dependent-on-cars-98498
Unmanned Surface and Underwater Vehicle (USV and UUV)
https://www.unmannedsyste mstechnology.com/wp- content/uploads/2013/12/C- Enduro-USV1.jpg https://www.dezeen.com/201 8/03/27/mit-reveal-life-like- soft-robotic-fish- documenting-marine-life- technology/
Precision Agriculture Aerial Taxi Cargo Transport Sports Coverage Emergency and Disaster Response Personal Assistance Traffic Surveillance Land Survey Infrastructure Health Monitoring Environmental Monitoring
UAV market is expected to grow US$ 51.85 billion by 2025 from US$ 11.45 billion in 2016. Global Autonomous Vehicles market accounted for $27.09 billion in 2017 and is expected to reach $615.02 billion by 2026 growing at a CAGR of 41.5%.
– Sensors – Communication – Control – Mechanical and electrical systems – Human-machine interaction – Security, privacy, certification – Data science – Machine learning – Embedded systems – Cyber-physical systems – Intelligent Transportation – Various application domains in civil engineering, business, biology, environmental science, urban planning, etc.
this domain from NSF, Lockheed, Ford, ONR, ARO, AFOSR, AFRL, FAA, etc.
such as AFRL Search and AI challenge
collaboration with local agencies and industries for successful technology transfer
in the near and future term: advanced autonomy, manned/unmanned interoperability, network security, human-machine collaboration
unmanned vehicles
– Highlights opportunities in transportation (people and goods), inspection, security and rescue, environmental monitoring – Highlights needs in intelligent infrastructure, safe navigation and control algorithms, advanced sensors/perception, human/machine information sharing, robustness/security
advancements in robotics and automation if instruction in robotics technologies is broadly available at all levels of the education system, from K-12, vocational, undergraduate and graduate programs.
states, including Texas, have enacted legislation related to autonomous vehicles
ground vehicles, defines owners and operators, preempts city legislation
Transportation Safety Administration released federal guidelines for Automated Driving Systems in 2016 and updated in 2019
legislation in committee
Forum on Controls and Uses of Autonomous Vehicles
Frank L. Lewis, Ph.D., NAI
Moncrief-O’Donnell Endowed Chair, UTA Research Institute Professor of Electrical Engineering, UTA
Talk available online at http://www.UTA.edu/UTARI/acs Activities in Automatic Control, Multi-agent Game Theory, Autonomous Intelligent Vehicles
Supported by : US NSF ONR, ARO, AFOSR
Yan Wan Nick Gans Ali Davoudi Patrik Kolaric Yusuf Kartal Victor Lopez
Moncrief-O’Donnell Chair, UTA Research Institute (UTARI) The University of Texas at Arlington, USA with
F.L. Lewis
National Academy of Inventors
UTA Research Institute Autonomous Systems Lab Patrik Kolaric Yusuf Kartal Victor Lopez
UTARI Autonomous Systems Lab
Human/Autonomous Systems interactions: Interactive Control of multiple UAV / UGV Multiagent decision for autonomous driverless vehicles Autonomous navigation and motion planning for UAV / UGV Human Interfaces using voice control, gesture control Synchronization and collective control in multi-agent autonomous teams Neurocognitive Reinforcement Learning for autonomous control
UAV platforms- Parrot AR Drone quadcopters
DR Robot Jaguar 4x4 DR Robot V4
Human User Interfaces
Current funding of $1.5 M in ONR and NSF grants
US Army Relevance. Decision and Control of Multi-Agent Teams. Received leveraging funding from US Army TARDEC, Ground Vehicle Robotics . Working with Dr. Dariusz Mikulski and Dr. Greg Hudas. Develop efficient learning mechanisms for military teams of humans and autonomous systems. Study Risk and Trust methods to develop coalitions in changing environments. Single User control of Multiple UAV/UGV. US Navy Collaborations. Work with Brian Holm-Hansen at ONR, Gary Hewer at NAWS China Lake, Dr. Wei Kang at US Naval Postgraduate School in Monterey. Reinforcement Learning for Improved Control of Unmanned Aerial Vehicles. Autonomous Decision for Multi-body Intelligent Systems US Air Force Relevance of Research in Coordination and Control of Multiple UAV Teams. Received leveraging funding from Kevin Bollino at AFOSR EAORD Europe. New methods for human coordination of multiple UAV systems. Trust-based learning mechanisms for improved performance and risk reduction of autonomous systems. Dual-Use Tech Transfer to Industry: Current Contracts Boeing Defense Space & Security. Adaptive Controller for Phantom Ray Unmanned Aircraft. Various Companies. Bio-inspired Learning for Data-driven Industrial Process Control. Dual-Use Tech Transfer to Electric Power Microgrids: Multi-agent control of distributed renewable generation- Resilient distributed protocols for improved response of microgrids. Applications to Army Bases as microgrids.
Current Funding
Cognitive Information,” Ford Contract for 3 years, $150K. April 2019-April 2022
contract from Lockheed Martin Advanced Technology Labs, Feb.-Dec. 2019.
Reinforcement Learning Games,” ARO grant, $30,000, April-June 2019.
based and Data-driven Real-time Optimization and Control for Uncertain Networked Systems, NSF grant, $220,000, September 2018-August 2020.
Grant, $815,000, June 2018-May 2022.
Microgrids,” ONR grant, $449,000, March 2017-March 2020.
Textbooks in Aircraft Control, Autonomous Systems, Machine Learning, Robotics, Multi-agent Systems
7 US Patents in Control Using Machine Reinforcement Learning, Multiagent Cooperative Control, Neural Adaptive Control, Intelligent Resource Assignment
400 journal papers
The Issues about Intelligent Feedback Control for Autonomous Vehicles
Automatic Feedback Control Systems must have provable stability, performance guarantees, and robustness These are not required in computer science applications MILSPEC Military Handling Qualities requirements for Aircraft Controllers Commercial aircraft stability augmentation and autopilots need guarantees Computer Science holds a wealth of Techniques such as Machine Learning, Game Theory, Nash equilibrium, Multi-agent decision, Deep Neural Networks How can we use CS machine learning, multiagent decision and still guarantee stability ??
UAV Control applications at Boeing Defense Space & Security Kevin Wise and Eugene Lavretsky Highly reliable adaptive uncertainty approximation compensators for flight control applications: unmanned aircraft – Phantom Ray
UAV Applications of our Machine Learning Neural Adaptive Control Technology
Our Neural Adaptive Controller is Currently Flown on Boeing Phantom Ray Unmanned Aircraft
Patent-
"Backlash Compensation Using Neural Network," U.S. Patent 6,611,823, awarded 26 Aug. 2003.
Dynamics and Control of Quadrotor UAV
AR Drone Parrot Crazyflie 3D Robotics Octocopter
Rotorcraft UAV
Angular position – attitudes Position – navigational states The Quadrotor States
Roll Pitch yaw
x y z X φ θ ψ =
Body Axes Vs. Earth-fixed axes
Quadrotor UAV Control Framework
ϕ θ ψ
τ τ τ τ = Lift torques
u
Only 4 control inputs
Control Problem - An Underactuated System
Position subsystem Attitude subsystem
Quadrotor Dynamics consists of TWO COUPLED Lagrange Dynamical Systems New Backstepping Control Design Technique Control Problem – Two interacting dynamical systems An Underactuated System 6 states but only 4 control inputs
Neural Adaptive Backstepping Flight Controller- 2 Control loops
Attitude Control Inner Loop Position Control Outer Loop Machine Learning Neural Adaptive Compensator for Dynamic Effects Control Allocation Unit Problem- built into the UAV. Can only implement PID controller
Problem- Inner attitude control loop is built into the UAV Can only implement PID controller
Every living organism improves its control actions based on rewards received from the environment The resources available to living organisms are usually meager. Nature uses optimal control.
1. Apply a control. Evaluate the benefit of that control. 2. Improve the control policy. RL finds optimal policies by evaluating the effects of suboptimal policies
Optimality in Biological Systems
Optimality Provides an Organizational Principle for Behavior
Charles Darwin showed that Optimal Control over long timescales. Is responsible for Natural Selection of Species
Why are Biological Systems Resilient?
Cell Homeostasis
The individual cell is a complex feedback control system. It pumps ions across the cell membrane to maintain homeostatis, and has
Cellular Metabolism
Permeability control of the cell membrane
http://www.accessexcellence.org/RC/VL/GG/index.html
Optimality in Biological Systems
Why are Biological Systems Resilient?
Multi-player Game Solutions IEEE Control Systems Magazine, Dec 2017
methodology for online adaptation to optimal feedback controller using integral reinforcement learning,” US patent 9,134,707 issued 15 Sept. 2015.
Bring Machine Learning into Feedback Control - Reinforcement Learning for Improved Optimal Adaptive Control
DDO – Online Data-Driven Optimization
Work with Dan Levine in Neurocognitive Psychology for Controls
Doya, Kimura, Kawato 2001
Limbic system
Motor control 200 Hz theta rhythms 4-10 Hz
Deliberative evaluation control
Reinforcement Learning Optimal Adaptive Controller
Draguna Vrabie Actor-Critic Machine Learning Structure For Continuous-time Systems Standard Adaptive Controller Supervisory Reinforcement Learning Controller Optimal Feedback Control- Minimum Resources Minimum fuel Minimum time Maximum efficiency
Charles Darwin showed that Nature Uses Reinforcement Learning Optimal Control for the Evolution of Species. The Resources available to natural biological systems are usually meager
integral reinforcement learning,” US patent 9,134,707 issued 15 Sept. 2015.
x u V
ZOH T
x Ax Bu = +
System
T T
x Qx u Ru ρ = +
Critic ActorK
−
T T
Optimal engine controllers based on RL for Caterpillar, Ford (Zetec engine), and GM
Applications of Our Algorithms to Auto Engine Control
Student S. Jagannathan, NAI 17 US patents 8-10% improvement in fuel efficiency and a drastic reduction in NOx (90%), HC (30%) and CO (15%) by operating with adaptive exhaust gas recirculation.
Savings to Caterpillar were over $65,000 per component.
Patent 6,064,997, awarded 16 May 2000.
F.L. Lewis, H. Zhang, A. Das, K. Hengster-Movric, Cooperative Control of Multi-Agent Systems: Optimal Design and Adaptive Control, Springer- Verlag, 2013 Multiple Interacting UAV Swarms Formations Need a Distributed Decision &Control Strategy
Multi-Agent Interacting Systems
Shan Zuo, Y.D. Song, F.L. Lewis, and A. Davoudi,, “Time-Varying Output Formation-Containment of General Linear Homogeneous and Heterogeneous Multi-Agent Systems,” IEEE Transactions on Control of Network Systems, vol. 6, no. 2, pp. 537-548, June 2019. Ci Chen, Kan Xie, Frank L. Lewis, Shengli Xie, and Ali Davoudi, “Fully Distributed Resilience for Adaptive Exponential Synchronization of Heterogenous Multi-Agent Systems Against Actuator Faults,” IEEE Trans. Automatic Control, vol. 64, no. 8, pp. 3347-3354, Aug. 2019
Leader or root node Followers
Communication Graph Formation of Multiple UAV Multiple Interacting Autonomous Vehicles
Interacting Dynamical Systems Desired Formation Topology
Leader
Cyber/physical System - The way we communicate can either Limit or Enhance the Way we
Each agent only sees its local neighbors
Synchronization of Multiple Quadrotor UAV
Single User controls Formation of 3 UAV Quadrotors
Submarine Control Surfaces
Unmanned Underwater Vehicles Robust Distributed Formation Controller
Kinematic Equation Dynamics Motion Equation UUV Dynamics consists of TWO COUPLED Lagrange Dynamical Systems
Hao Liu, Yanhu Wang, and Frank L. Lewis, “Robust Distributed Formation Controller Design for a Group of Unmanned Underwater Vehicles,” IEEE Transactions on Systems, Man, and Cybernetics: Systems, to appear 2019.
1. Victor G. Lopez, F.L. Lewis, Yan Wan, Edgar N. Sanchez, and Lingling Fan, “Solutions for Multiagent Pursuit-Evasion Games on Communication Graphs: Finite-Time Capture and Asymptotic Behaviors,” IEEE Transactions on Automatic Control, to appear, 2019. 2. Hao Liu, Yu Tian, F.L Lewis, Yan Wan, K.P. Valavanis, “Robust Formation Tracking Control for Multiple Quadrotors under Aggressive Maneuvers, Automatica, to appear 2019. 3. Shan Zuo, Y.D. Song, F.L. Lewis, and A. Davoudi,, “Time-Varying Output Formation-Containment of General Linear Homogeneous and Heterogeneous Multi-Agent Systems,” IEEE Transactions on Control of Network Systems, vol. 6, no. 2, pp. 537-548, June 2019. 4. Kan Xie, Ci Chen, Frank L. Lewis, and Shengli Xie, “Adaptive Compensation for Nonlinear Time-varying Multi-Agent Systems with Actuator Failures and Unknown Control Directions,” IEEE Transactions on Cybernetics, vol. 49, no. 5, pp. 1780-1790, May 2019. 5.
Systems Under Attacks on Sensors and Actuators,” IEEE Transactions on Cybernetics, to appear 2019. 6. Victor Lopez and F.L. Lewis, “Dynamic Multiobjective Control for Continuous-time Systems using Reinforcement Learning,” IEEE
7. Jinna Li, Tianyou Chai, F.L. Lewis, Jinliang Ding, Yi Jiang, “Off-policy interleaved Q-learning: optimal control for affine nonlinear discrete-time systems,” IEEE Transactions on Neural Networks and Learning Systems, vol. 30, no. 5, pp. 1308-1320, May 2019. 8. Bahare Kiumarsi, K. Vamvoudakis, H. Modares, and F.L. Lewis, “Optimal and Autonomous Control Using Reinforcement Learning: A Survey,” IEEE Trans. Neural Networks and Learning Systems, vol. 29, no. 6, pp. 2042-2061, June 2018.
A few of our Recent Papers in Autonomous Vehicles and Multi-agent Intelligent Systems
Two opposing teams of MAS in Graphical Zero-Sum games. Protagonists (blue) try to minimize a value function, whereas antagonists (orange) want to maximize it. The result is a local Nash saddle-point equilibrium at each node and also global Nash between the two teams.
Opposing Teams and Malicious Adversaries’ on Networks
We study many sorts of Autonomous Vehicle and Social Behaviors in communication networks
0k
z
Adversarial Leader Group
ik
h
ik
g
0k
x
Leader Group Social Network with weak links
Disturbance inputs of social agents are subverted by adversarial leaders to mount a coordinated attack on a social network. The social net has cut-sets of weak links (shown dashed in red), and so has the potential to be fractured into disconnected subgroups.
Infiltration of Weakly Linked Splinter Groups in Social Networks
Evader set V Pursuer Set U
Multiple evaders (blue) are networked by interaction graph Ge. Multiple pursuers (red) are networked by graph Gp. Pursuer and evader interactions are captured by bipartite graph Gep (dashed edges).
Multi-agent Pursuit-Evasion Games
To appear in IEEE Transactions on Automatic Control
Work of Victor Lopez
Real-time Multiplayer Games for Autonomous Traffic Decision Ford Contract – Dimitar Filev and Subramanya Naeshrao
Each agent has dynamics- e.g. moving vehicles Each agent tries to optimize its own performance function Applications of multiagent systems in autonomous vehicle systems Lane changing problem Intersection Problem Freeway merging problem Platoon convoys Changing platoons Work of Victor Lopez
RL for Human-Robot Interaction (HRI)
1.
Transactions on Cybernetics, vol. 46, no. 3, pp. 655-667, 2016. 2.
Design and Adaptive Inverse Filtering" IEEE Transactions on Control Systems Technology, vol. 25, no. 1, pp. 278-285, Jan. 2017. 3.
robot interaction,” Control Theory and Technology, vol. 14, no. 1, pp. 1-15, Feb. 2016.
PR2 meets Isura
Forum on Controls and Uses of Autonomous Vehicles
Yan Wan, Ph.D.
Associate Professor, Electrical Engineering
MathWorks, National Instruments, NCSU, U of Washington, WPI, and MIT Lincoln lab, 2013-2017
https://www.youtube.com/watch?v=Yi_dK4iRCA4&t=34s
AFRL Search and AI Challenge, 1st place in
teams who scored top five in all competition runs, April 2019 https://www.youtube.com/watch?v=fOq56R7Dx Dk
including five from NSF, and four from other agencies and industries.
Aerial Networks in an Uncertain Airspace, $147,240, 08/30/2019-08/31/2021, Sole PI (100%).
Co-PI (50%), collaboration with Frank Lewis (PI).
Estimation, Optimal Learning, Scalable Uncertainty Sampling, and Time-critical Communication, $815,019, 06/01/2018- 05/31/2022, Co-PI (50%), collaboration with Frank Lewis (PI).
Control for Uncertain Networked Systems, $220,010 (project total $300,000), 09/15/2018-08/31/2020, Co-PI (33%), collaboration with Frank Lewis (PI) and Ali Davoudi (Co-PI), and TAMU-CC.
total: $998,803), 09/01/2017 - 08/31/2020, Lead Institution PI (100%), collaboration with TAMU-CC, UNT, and UPRM.
(20%), collaboration with Atilla Dogan (PI), Kamesh Subbarao, Manfred Huber, and Brian Huff.
$442,538, 06/01/2015-05/31/2020, Sole PI (100%).
09/31/2019, Co-PI (50%), collaboration with Frank Lewis (PI).
Co-PI (50%), collaboration with Frank Lewis (PI).
concerned with multiple spatiotemporal scales:
communication
measurement and goal function for control
communication model, and distributed reinforcement learning is used for adaptive optimal control.
technology transfer in the safety-critical emergency response application.
Mathworks, and many other universities and industries
Forum on Controls and Uses of Autonomous Vehicles
Animesh Chakravarthy, Ph.D.
Associate Professor, Mechanical and Aerospace Engineering
Forum on Controls and Uses of Autonomous Vehicles
Nicholas Gans, Ph.D.
Division Head - Automation and Intelligent Systems, UTARI
Advancing I ntelligent Systems TRLs Through Federal Funded R&D at UT Arlington Research I nstitute
tools to with direct application to real-world problems.
vehicles for surveillance
physical and cognitive load
target detection and recognition
custom swarm sir vehicles
monitoring
care and social interaction
roads and infrastructure
disabled
prosthetics
Industry Society Defense
Sector Domains Applications
applications such as intelligent transportation, emergency response, infrastructure monitoring and agriculture
environments & vision-based distributed formation control
Research in Multi-agent, Multi-system Autonomy funded by DoD Research Labs and Offices
and Manned, Unmanned Teaming utilizing AI/ML and AR
Applied Unmanned Systems Developments Through I ndustry Partnership at UTA Research I nstitute
for UAV-based Identification of UAVs – STTR Phase II invited and submitted
UAVs for Surveying, I nspection and Environmental Monitoring
Hurricane Harvey in 2017
UAV Flying Underneath Bridge