Where the Intermediate is the Big Step Intralogistics with Safe and - - PowerPoint PPT Presentation

▶
where the intermediate is the big step
SMART_READER_LITE
LIVE PREVIEW

Where the Intermediate is the Big Step Intralogistics with Safe and - - PowerPoint PPT Presentation

Aug 30, 2022 125 likes •654 views

Where the Intermediate is the Big Step Intralogistics with Safe and Scalable Fleets of Autonomously Operating Vehicles in Shared Spaces Achim J. Lilienthal contact: www.mrolab.eu/achim-lilienthal achim.lilienthal@oru.se Intra- Logistics

slide-1
SLIDE 1

Intralogistics with Safe and Scalable Fleets of Autonomously Operating Vehicles in Shared Spaces

Where the Intermediate is the Big Step

Achim J. Lilienthal contact:

www.mrolab.eu/achim-lilienthal achim.lilienthal@oru.se

slide-2
SLIDE 2

Intralogistics with Safe and Scalable Fleets of Autonomously Operating Vehicles in Shared Spaces

Where the Intermediate is the Big Step

Intra- Logistics with Integrated Automatic Deployment

www.iliad-project.eu

H2020 project funded by the EC Start: Jan 2017 Duration: 48 months Funding: 7M€ Partners: 9

slide-3
SLIDE 3

Intralogistics with Safe and Scalable Fleets of Autonomously Operating Vehicles in Shared Spaces

Where the Intermediate is the Big Step

Intra- Logistics with Integrated Automatic Deployment

H2020 project funded by the EC Start: Jan 2017 Duration: 48 months Funding: 7M€ Partners: 9

www.iliad-project.eu

slide-4
SLIDE 4

ILIAD in a nutshell

Why the intermediate is the big step

slide-5
SLIDE 5

Concept image

More quickly changing market needs, unforeseeable trends, shorter product life cycles, …

slide-6
SLIDE 6

Flexible intralogistics needed

  • Today’s highly automated goods-to-man solutions
  • require dedicated warehouses,
  • are unsuitable for fresh food, bulky goods, etc.
  • ILIAD’s aim is to provide solutions for flexible intralogistics for the transition to

automation in shared spaces.

slide-7
SLIDE 7

Therefore we need intralogistic systems that are F-RE-SE–QUD–SA-EFF! (1) highly flexible, (2) rock-solid reliable, (3) self-optimising, (4) quickly deployable and (5) safe yet (6) efficient in environments shared with humans.

Flexible intralogistics needed

slide-8
SLIDE 8

Approach

Safety and efficiency by long-term learning and prediction of patterns Easy deployment with semantic mapping On-line, self-optimising fleet management Flexible manipulation Human safety-aware AGV fleets: safe and legible motion planning, human detection and tracking, mutual communication of intent

slide-9
SLIDE 9

Platform (CitiTruck)

Early prototype (APPLE platform at Hannover fair 2015)

slide-10
SLIDE 10

Demonstrators

slide-11
SLIDE 11

Demonstrators

  • Demos at key stakeholders from food distribution sector.
  • Distribution of fresh food products particularly

challenging:

  • sensitive products,
  • short shelf life,
  • rapid response to consumer needs.
  • Food industry largest manufacturing sector in EU:

4.2 million jobs.

slide-12
SLIDE 12

Orkla Foods

Not always well-defined aisle environment Heterogeneous and brittle (and sometimes large or heavy) items to pick.

slide-13
SLIDE 13

NCFM

  • National Centre for Food Manufacturing
slide-14
SLIDE 14

ASDA

Photo credit: Redirack Photo credit: Adrian Welsh

slide-15
SLIDE 15

Easy deployment, long-term operation

Learning, modelling and exploiting flow information

slide-16
SLIDE 16

Learn, model and use dynamics

  • Learn and map how things usually move.
  • Statistical multi-modal flow model.
  • Use for socially compliant motion planning:

less "annoying", safer and more efficient operation.

slide-17
SLIDE 17

Learn and model dynamics

  • Learn and map how things usually move.
  • Extensive research on mapping geometric structure.
  • Environment typically defined by spatial movement patterns.

FP7 EU project SPENCER http://www.spencer.eu/

slide-18
SLIDE 18

Learn and model dynamics

  • Learn and map how things usually move.
  • Extensive research on mapping geometric structure.
  • Environment typically defined by spatial movement patterns.
  • Using this information can
  • lead to safer and socially more acceptable robot trajectories;
  • allow to plan energy efficient paths for flying robots;
  • improve gas distribution mapping.
slide-19
SLIDE 19

Learn and model dynamics

  • Learn and map how things usually move.
  • Extensive research on mapping geometric structure.
  • Environment typically defined by spatial movement patterns.
  • Statistical multi-modal flow model CLiFF map

(Circular–Linear Flow Field map) – A probabilistic approach for general flow mapping

Enabling Flow Awareness for Mobile Robots in Partially Observable Environments.

  • T. P. Kucner, M. Magnusson, E. Schaffernicht, V. Hernandez Bennetts, and A. J. Lilienthal.

RA-L 2017 (2:2, pp. 1093-1100) / ICRA 2017

slide-20
SLIDE 20

Learn and model dynamics

  • Learn and map how things usually move.
  • Extensive research on mapping geometric structure.
  • Environment typically defined by spatial movement patterns.
  • Statistical multi-modal flow model CLiFF map
  • Local elements are probability distributions of observations V = (θ, ρ)
  • One circular (orientation θ) and one linear (speed ρ) random variable.
slide-21
SLIDE 21

Learn and model dynamics

  • Learn and map how things usually move.
  • Extensive research on mapping geometric structure.
  • Environment typically defined by spatial movement patterns.
  • Statistical multi-modal flow model CLiFF map
  • Local elements are probability distributions of observations (θ, ρ)
  • One circular (orientation θ) and one linear (speed ρ) random variable.
  • Semi-wrapped Gaussian mixtures

Learning CLiFF maps

  • Init: Mean Shift (MS) to determine number of clusters and their positions
  • Use MS clusters to initialize Expectation Maximisation (EM)
slide-22
SLIDE 22

Learn and model dynamics

  • Learn and map how things usually move.
  • Extensive research on mapping geometric structure.
  • Environment typically defined by spatial movement patterns.
  • Statistical multi-modal flow model CLiFF map
  • Local elements are probability distributions of observations (θ, ρ)
  • One circular (orientation θ) and one linear (speed ρ) random variable.
  • Field of semi-wrapped Gaussian mixtures
slide-23
SLIDE 23

Learn and model dynamics

  • Learn and map how things usually move.
  • Extensive research on mapping geometric structure.
  • Environment typically defined by spatial movement patterns.
  • Statistical multi-modal flow model CLiFF map
  • Local elements are probability distributions of observations (θ, ρ)
  • One circular (orientation θ) and one linear (speed ρ) random variable.
  • Field of semi-wrapped Gaussian mixtures
slide-24
SLIDE 24

Learn and model dynamics

  • Learn and map how things usually move.
  • Extensive research on mapping geometric structure.
  • Environment typically defined by spatial movement patterns.
  • Statistical multi-modal flow model CLiFF map
  • Local elements are probability distributions of observations (θ, ρ)
  • One circular (orientation θ) and one linear (speed ρ) random variable.
  • Field of semi-wrapped Gaussian mixtures
slide-25
SLIDE 25

Learn and model dynamics

  • Learn and map how things usually move.
  • Extensive research on mapping geometric structure.
  • Environment typically defined by spatial movement patterns.
  • Statistical multi-modal flow model CLiFF map
  • Field of semi-wrapped Gaussian mixtures
  • Observations in robotics are often sparse

?

slide-26
SLIDE 26

Learn and model dynamics

  • Learn and map how things usually move.
  • Extensive research on mapping geometric structure.
  • Environment typically defined by spatial movement patterns.
  • Statistical multi-modal flow model CLiFF map
  • Field of semi-wrapped Gaussian mixtures
  • Observations in robotics are often sparse

?

slide-27
SLIDE 27

Learn and model dynamics

  • Learn and map how things usually move.
  • Extensive research on mapping geometric structure.
  • Environment typically defined by spatial movement patterns.
  • Statistical multi-modal flow model CLiFF map
  • Field of semi-wrapped Gaussian mixtures
  • Observations in robotics are often sparse
  • => Data imputation to build dense maps from sparse measurements
  • Monte Carlo Imputation (MC)
  • sampling virtual observations from the surrounding
  • tends to preserve multimodal characteristics and keep sharp transitions
  • Nadaraya Watson Imputation (NW)
  • Weighted extrapolation (distance kernel)
  • smooths data and models introduces gradual changes
  • MC performed better than NW on pedestrian data (and is less sensitive to

kernel size) – but results may differ in different applications

slide-28
SLIDE 28

Learn and model dynamics

  • Learn and map how things usually move.
  • Extensive research on mapping geometric structure.
  • Environment typically defined by spatial movement patterns.
  • Statistical multi-modal flow model CLiFF map
  • Field of semi-wrapped Gaussian mixtures
  • Observations in robotics are often sparse
  • => Data imputation to build dense maps from sparse measurements
  • Summary
  • It is possible to accurately represent multimodal (even turbulent) flow

using CLiFF map.

  • It is possible to reconstruct a dense representation based on

sparsely distributed observations.

slide-29
SLIDE 29

Using dynamics models

  • Use for socially compliant motion planning:

less "annoying", safer and more efficient operation.

Kinodynamic Motion Planning on Gaussian Mixture Fields.

  • L. Palmieri, T. Kucner, M. Magnusson, A. J. Lilienthal, and K. O. Arras

ICRA 2017

slide-30
SLIDE 30

Using dynamics models

  • Socially compliant motion planning using CLiFF map
  • Mobile robot motion planning approach: CLiFF-RRT*
  • CLiFF map – provides learned perception prior
  • RRT* motion planner
  • asymptotically optimal sampling-based motion planner
  • considers the robot's kinematic and its non-holonomic constraints

Kinodynamic Motion Planning on Gaussian Mixture Fields.

  • L. Palmieri, T. Kucner, M. Magnusson, A. J. Lilienthal, and K. O. Arras

ICRA 2017

slide-31
SLIDE 31

Using dynamics models

  • Socially compliant motion planning using CLiFF map
  • Mobile robot motion planning approach: CLiFF-RRT*
  • CLiFF map – provides learned perception prior
  • RRT* motion planner
  • => CLiFF map guides sampling in the RRT* planner
  • low costs for paths that comply to the directions of CLiFF-map mixture

components and high costs for paths in opposite directions

slide-32
SLIDE 32

Using dynamics models

  • Socially compliant motion planning using CLiFF map
  • Mobile robot motion planning approach: CLiFF-RRT*
  • CLiFF map – provides learned perception prior
  • RRT* motion planner
  • => CLiFF map guides sampling in the RRT* planner
  • Results are very encouraging
  • Planner generates reasonable ("socially compliant") trajectories
slide-33
SLIDE 33

Using dynamics models

  • Socially compliant motion planning using CLiFF map
  • Mobile robot motion planning approach: CLiFF-RRT*
  • CLiFF map – provides learned perception prior
  • RRT* motion planner
  • => CLiFF map guides sampling in the RRT* planner
  • Results are very encouraging
  • Planner generates reasonable ("socially compliant") trajectories
  • Planner is very efficient (significantly faster than RRT and RRT*) /

very fast convergence

  • Generates higher quality paths (in terms of smoothness and length)

RRT* path CLiFF-RRT* path

slide-34
SLIDE 34

Easy deployment, long-term operation

Ongoing work

slide-35
SLIDE 35

Spatiotemporal mapping

  • Representations and inference for compression of past

experience and prediction of future states with confidence intervals.

  • Learn where and when activities happen.

FreMEn (Krajnik et al., ECMR 2015)

slide-36
SLIDE 36

Predicting patterns

  • Combine inputs from mapping and tracking.
  • Actively update knowledge – plan where and when to

collect data.

FreMEn (Krajnik et al., ECMR 2015)

slide-37
SLIDE 37

Reliability-aware loc. & maps

  • Combining metrics to assess quality of scan registration.
  • Detect mapping errors using learned structural cues.
slide-38
SLIDE 38

Auto-calibration

  • Unsupervised monitoring and (re-)calibration of sensors.
  • Find regions with high information for calibration.

6 DOF – x,y,z,roll,pitch,yaw Optimized timing offset 7 DOF – x,y,z,roll,pitch,yaw,dt

slide-39
SLIDE 39

Safe and human-aware

  • peration

Ongoing work

slide-40
SLIDE 40

Human safety

  • Study human safety in shared environments.
  • Connect injury biomechanics to safe control and

planning.

  • Associate vehicle dynamics to injury safety database.
  • Extend Safe Motion Unit paradigm to vehicle-human and

vehicle-vehicle interaction.

  • Shape vehicle velocity based on
  • humans and vehicles in the environment,
  • environment observability,
  • predictive braking models.
slide-41
SLIDE 41

Human-aware fleets

  • Increase human-robot cooperation safety & efficiency.
  • Detect, track, and analyse people.
slide-42
SLIDE 42

Human-aware fleets

  • Increase human-robot cooperation safety & efficiency.
  • D

etect, track, and analyse people.

slide-43
SLIDE 43

Human-aware fleets

  • Increase human-robot cooperation safety & efficiency.
  • Detect, track, and analyse people.
slide-44
SLIDE 44

Human-aware fleets

  • Increase human-robot cooperation safety & efficiency.
  • Detect, track, and analyse people.
  • => Retenua AB
slide-45
SLIDE 45

Human-aware fleets

  • Increase human-robot cooperation safety & efficiency.
  • Detect, track, and analyse people.
  • Recognise human intentions.
  • Visually communicate robot intentions.
  • Socially normative motion planning.
slide-46
SLIDE 46

Manipulation

Ongoing work

slide-47
SLIDE 47

Manipulation

  • Innovative end-effectors.
  • Perception for dense packets, and plastic wrapping.
  • Control for unwrapping, picking, palletising.
  • Optimise package positions on pallets.
slide-48
SLIDE 48

Fleet management

Ongoing work

slide-49
SLIDE 49

Fleet management

  • Integrated task allocation, motion planning, and coordination.
  • Guaranteed deadlock-free operation.
  • Continuously revise w.r.t. changing requirements.
slide-50
SLIDE 50

Intralogistics with Safe and Scalable Fleets of Autonomously Operating Vehicles in Shared Spaces

Where the Intermediate is the Big Step

Intra- Logistics with Integrated Automatic Deployment

Thanks for your attention!

slide-51
SLIDE 51

Intralogistics with Safe and Scalable Fleets of Autonomously Operating Vehicles in Shared Spaces

Where the Intermediate is the Big Step

Intra- Logistics with Integrated Automatic Deployment

www.iliad-project.eu

H2020 project funded by the EC Start: Jan 2017 Duration: 48 months Funding: 7M€ Partners: 9

This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 732737.

Thanks for your attention!

slide-52
SLIDE 52

Intralogistics with Safe and Scalable Fleets of Autonomously Operating Vehicles in Shared Spaces

Where the Intermediate is the Big Step

Achim J. Lilienthal contact:

www.mrolab.eu/achim-lilienthal achim.lilienthal@oru.se