On Algorithmic Decision Procedures in Emergency Response Systems in - - PowerPoint PPT Presentation

▶
on algorithmic decision procedures in emergency response
SMART_READER_LITE
LIVE PREVIEW

On Algorithmic Decision Procedures in Emergency Response Systems in - - PowerPoint PPT Presentation

Feb 21, 2023 265 likes •681 views

On Algorithmic Decision Procedures in Emergency Response Systems in Smart and Connected Communities Geoffrey Pettet 1 , Ayan Mukhopadhay 2 , Mykel Kochenderfer 2 , Yevgeniy Voroybeychik 3 , Abhishek Dubey 1 1 Vanderbilt University, 2 Stanford

slide-1
SLIDE 1

World-class, interdisciplinary research with global impact.

Institute for Software Integrated Systems

On Algorithmic Decision Procedures in Emergency Response Systems in Smart and Connected Communities

1Vanderbilt University, 2Stanford University, 3Washington University in St Louis

Sponsored by National Science Foundation, Center for Automotive Research at Stanford (CARS), and Tennessee Department of Transportation

Geoffrey Pettet1, Ayan Mukhopadhay2, Mykel Kochenderfer2, Yevgeniy Voroybeychik3, Abhishek Dubey1

slide-2
SLIDE 2

World-class, interdisciplinary research with global impact.

Institute for Software Integrated Systems

Motivation and Background

1

slide-3
SLIDE 3

World-class, interdisciplinary research with global impact.

Institute for Software Integrated Systems

The emergency response problem

The problem: Respond Efficiently to all incidents spread over a large geographic area with limited resources.

All traffic incidents

  • ccurring in Davidson

County In January 2018, with a sliding window of ~12 hours

2

slide-4
SLIDE 4

World-class, interdisciplinary research with global impact.

Institute for Software Integrated Systems

Proactive Emergency Response

Online Demand Estimation Models[1] Anticipatory Stationing of Resources Optimal Dispatch[1] Active Learning and Improvement Mechanisms

[1] Ayan Mukhopadhyay, Geoffrey Pettet, Chinmaya Samal, Abhishek Dubey, and Yevgeniy Vorobeychik. 2019. An online decision- theoretic pipeline for responder dispatch. In Proceedings of the 10th ACM/IEEE International Conference on Cyber-Physical Systems (ICCPS ’19). Association for Computing Machinery, New York, NY, USA, 185–196. DOI:https://doi.org/10.1145/3302509.3311055 3

slide-5
SLIDE 5

World-class, interdisciplinary research with global impact.

Institute for Software Integrated Systems

Proactive Emergency Response

Online Demand Estimation Models[1] Anticipatory Stationing of Resources Optimal Dispatch[1] Active Learning and Improvement Mechanisms

[1]

Prediction Actual Tree Search (MCTS)

Previous Work

[1] Ayan Mukhopadhyay, Geoffrey Pettet, Chinmaya Samal, Abhishek Dubey, and Yevgeniy Vorobeychik. 2019. An online decision- theoretic pipeline for responder dispatch. In Proceedings of the 10th ACM/IEEE International Conference on Cyber-Physical Systems (ICCPS ’19). Association for Computing Machinery, New York, NY, USA, 185–196. DOI:https://doi.org/10.1145/3302509.3311055 4

slide-6
SLIDE 6

World-class, interdisciplinary research with global impact.

Institute for Software Integrated Systems

Proactive Emergency Response

Active Learning and Improvement Mechanisms

“Rebalancing”

Advantages over decision making at time of dispatch:

  • Ample time to make decision
  • Avoids legal and moral questions
  • Proactive
  • Larger decision space => more

room for gains

Online Demand Estimation Models[1] Anticipatory Stationing of Resources Optimal Dispatch[1]

Focus of Paper

5

slide-7
SLIDE 7

World-class, interdisciplinary research with global impact.

Institute for Software Integrated Systems

System Model and Assumptions

6

slide-8
SLIDE 8

World-class, interdisciplinary research with global impact.

Institute for Software Integrated Systems

Region segmented into a grid with equally sized cells Incident arrival model “Depots” – subset of cells where agents can wait

https://www.nashville.gov

System em Model el – Assu Assumption ions

Historical data (incidents, traffic, weather, etc.)

7

slide-9
SLIDE 9

World-class, interdisciplinary research with global impact.

Institute for Software Integrated Systems

System Model – Multi Agent SMDP

States

  • Continuous state

space

  • Discrete states of

interest:

  • Incident occurrence
  • Responder

availability

  • Rebalancing

triggered

Actions

  • Directing agents to

valid cells:

  • Response: pending

incident locations

  • Rebalancing: depots

Transitions

  • Time between

incidents

  • Incident Service

time

  • Computation time
  • Travel time

Rewards

  • Balance-
  • minimizing response

times

  • minimizing distance

traveled

  • Ic: Grids waiting for service
  • Rc: Agent states
  • Ec: Environmental Factors

Current State sc

sc+1

Action set

σc sc-1 Ic+1 , Rc+1 , Ec+1 σc+1 Ic+2 , Rc+2 , Ec+2 sc+2 σc-1 σc+2

*SMDP diagram simplified for demonstration

8

slide-10
SLIDE 10

World-class, interdisciplinary research with global impact.

Institute for Software Integrated Systems

Problem Definition

Given: System state, predicted spatial-temporal incident distribution Return: Action recommendation set that maximizes expected reward Reward can be fine-tuned

9

slide-11
SLIDE 11

World-class, interdisciplinary research with global impact.

Institute for Software Integrated Systems

Approaches to Solving SMDP

Policy iteration (Dispatch)[1]

  • Will converge to best

dispatch policy eventually

  • Slow – must estimate state

transition probabilities

MCTS (Dispatch)

  • Anytime algorithm
  • Not scalable to

dynamic balancing

Greedy Heuristic Search Multi- Agent Monte Carlo Tree Search

[1] Ayan Mukhopadhyay, Zilin Wang, and Yevgeniy Vorobeychik. 2018. A Decision Theoretic Framework for Emergency Responder Dispatch. In Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent Systems (AAMAS ’18). International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC, 588–596.

SimTrans[1] Queueing Theory A1 A3 A2

New Work

10

slide-12
SLIDE 12

World-class, interdisciplinary research with global impact.

Institute for Software Integrated Systems

Approach 1: Greedy Search with Queue-Heuristic

11

slide-13
SLIDE 13

World-class, interdisciplinary research with global impact.

Institute for Software Integrated Systems

Approach 1: Queue Theory Heuristic Search

μ

Waiting Area Server

Grid Cell incident rate := υ Incidents Placed in Waiting Queue Ambulances ‘serve’ incidents with mean rate := μ

Intuition

12

slide-14
SLIDE 14

World-class, interdisciplinary research with global impact.

Institute for Software Integrated Systems

Approach 1: Queue Theory Heuristic Search

Server M/M/c Queue Formulation Multiple Servers

Intuition

μ

13

slide-15
SLIDE 15

World-class, interdisciplinary research with global impact.

Institute for Software Integrated Systems

Approach 1: Queue Theory Heuristic Search

μ

Server M/M/c Queue Formulation Multiple Servers

Intuition

Queue Response time: Avg time in system

14

slide-16
SLIDE 16

World-class, interdisciplinary research with global impact.

Institute for Software Integrated Systems

Approach 1: Queue Theory Heuristic Search “Multi Class, Multi Server Queue Formulation”

μ1 μ2 μk

15

slide-17
SLIDE 17

World-class, interdisciplinary research with global impact.

Institute for Software Integrated Systems

Approach 1: Queue Theory Heuristic Search

μ1 μ2 μk

υ2

υ1

2

υ22 υ3

2

How to determine split of rates?

16

slide-18
SLIDE 18

World-class, interdisciplinary research with global impact.

Institute for Software Integrated Systems

Approach 1: Queue Theory Heuristic Search

  • For each cell, distribute rate among depots inversely proportional to the

distance from the cell to the depot

  • Closer depots => higher portion of rate
  • Solve System of Linear equations above for each cell
  • υgd is the fraction of arrival rate for cell g that is shared by depot d

17

slide-19
SLIDE 19

World-class, interdisciplinary research with global impact.

Institute for Software Integrated Systems

Approach 1: Queue Theory Heuristic Search

  • To score a particular allocation of agents:
  • Must consider travel times => not memoryless, so model explicitly
  • ϒ represents collection of split rates
  • Score πϒ => sum across all cells and depots
  • Estimated (queue) response times (waiting + service time)
  • Travel time from depot to cell

18

slide-20
SLIDE 20

World-class, interdisciplinary research with global impact.

Institute for Software Integrated Systems

Approach 1: Queue Theory Heuristic Search

  • Depot selection: Greedy Search
  • One by one select depot that

minimizes πϒ

  • Add to chosen set
  • Re-split rates and calculate

new scores with each new depot placed

  • Continue until the number of

depots chosen is the same as number of agents

  • Assign agents to chosen depots

by minimizing distance traveled (Linear Program)

19

slide-21
SLIDE 21

World-class, interdisciplinary research with global impact.

Institute for Software Integrated Systems

Approach 1: Overview

Choose depots via greedy search

  • Repeat until # chosen depots == #

agents:

  • Split incident rates across depots
  • Score allocations
  • Add depot that minimizes score

Assign agents to chosen depots

  • Minimize distance traveled
  • LP, Greedy Search, etc.

Choose depots via greedy search Assign agents to chosen depots

20

slide-22
SLIDE 22

World-class, interdisciplinary research with global impact.

Institute for Software Integrated Systems

Approach 1: Overview

Choose depots via greedy search

  • Repeat until # chosen depots == #

agents:

  • Split incident rates across depots
  • Score allocations
  • Add depot that minimizes score

Assign agents to chosen depots

  • Minimize distance traveled
  • LP, Greedy Search, etc.

Choose depots via greedy search Assign agents to chosen depots

Disadvantages:

  • Doesn’t take internal system state into account
  • Ignores dynamic incident rate distribution

Advantage:

  • Computationally efficient

21

slide-23
SLIDE 23

World-class, interdisciplinary research with global impact.

Institute for Software Integrated Systems

Approach 2: Multi-Agent Monte Carlo Tree Search (MMCTS)

22

slide-24
SLIDE 24

World-class, interdisciplinary research with global impact.

Institute for Software Integrated Systems

Approach 2: MMCTS

Claes, Daniel, et al. "Decentralised online planning for multi-robot warehouse commissioning." AAMAS'17: PROCEEDINGS OF THE 16TH INTERNATIONAL CONFERENCE ON AUTONOMOUS AGENTS AND MULTIAGENT SYSTEMS. 2017.

Inspiration

Typical Warehouse Model

*Improved on state of the art, particularly in cases with large state space

23

slide-25
SLIDE 25

World-class, interdisciplinary research with global impact.

Institute for Software Integrated Systems

Approach 2: MMCTS

Standard MCTS; Action space limited to relevant actions for the Agent

Monolithic State Space State space split for each agent

24

slide-26
SLIDE 26

World-class, interdisciplinary research with global impact.

Institute for Software Integrated Systems

Approach 2: MMCTS

Standard MCTS; Action space limited to relevant actions for the Agent

Extensions needed for domain…

  • How to approximate behavior of other agents in

ERM Domain?

  • How to enforce global constraints?

25

slide-27
SLIDE 27

World-class, interdisciplinary research with global impact.

Institute for Software Integrated Systems

Approach 2: MMCTS

Approximate Agent behavior

  • 1) Naïve policy: other agents

do not rebalance

  • 2) Informed policy: use

queue-based heuristic formulated in approach 1! Enforcing Global Constraints

  • Centralized filter
  • Ensure that
  • incidents are responded to
  • Depots aren’t filled over

capacity

  • Uses greedy action

assignment based on returned rewards

26

slide-28
SLIDE 28

World-class, interdisciplinary research with global impact.

Institute for Software Integrated Systems

Approach 2: MMCTS

Approximate Agent behavior

  • 1) Naïve policy: other agents

do not rebalance

  • 2) Informed policy: use

queue-based heuristic formulated in approach 1! Enforcing Global Constraints

  • Centralized filter
  • Ensure that
  • incidents are responded to
  • Depots aren’t filled over

capacity

  • Uses greedy action

assignment based on returned rewards

27

slide-29
SLIDE 29

World-class, interdisciplinary research with global impact.

Institute for Software Integrated Systems

Approach 2: MMCTS Reward Structure

  • Accounts for -
  • Incident dispatch -> response time
  • Balancing -> distance traveled

28

slide-30
SLIDE 30

World-class, interdisciplinary research with global impact.

Institute for Software Integrated Systems

Approach 2: MMCTS Reward Structure

  • Accounts for…
  • Incident dispatch -> response time
  • Balancing -> distance traveled

Discounted Travel Time

29

slide-31
SLIDE 31

World-class, interdisciplinary research with global impact.

Institute for Software Integrated Systems

Approach 2: MMCTS Reward Structure

  • Accounts for…
  • Incident dispatch -> response time
  • Balancing -> distance traveled

Average Distance Traveled For Re-balancing

30

slide-32
SLIDE 32

World-class, interdisciplinary research with global impact.

Institute for Software Integrated Systems

Approach 2: MMCTS Reward Structure

  • Accounts for…
  • Incident dispatch -> response time
  • Balancing -> distance traveled

Average Distance Traveled For Re-balancing; [>0]

31

slide-33
SLIDE 33

World-class, interdisciplinary research with global impact.

Institute for Software Integrated Systems

Experiments and Discussion

32

slide-34
SLIDE 34

World-class, interdisciplinary research with global impact.

Institute for Software Integrated Systems

Experimental Configuration

Davidson County: Nashville Fire Department Administration Area

  • 26 Responders (Agents)
  • 36 Depots
  • Incident model training set: 35858

traffic incidents occurring in 2018

  • Entire system evaluated on 2728

incidents occurring in January, 2019

33

slide-35
SLIDE 35

World-class, interdisciplinary research with global impact.

Institute for Software Integrated Systems

Experimental Configuration

Radius of Influence (RoI)

  • Only depots within a cell’s RoI

are considered when splitting rates in heuristic score

  • Encourages even agent

distribution

  • Reduces computation time

RoI := 3 cells

34

slide-36
SLIDE 36

World-class, interdisciplinary research with global impact.

Institute for Software Integrated Systems

Results – Greedy Heuristic Search

Observations

  • Radius of Influence (RoI)

has significant impact

  • Best RoI => significant

impact on tail of response time distribution

  • <1 mile moved per

rebalancing step on average

35

slide-37
SLIDE 37

World-class, interdisciplinary research with global impact.

Institute for Software Integrated Systems

Results – MMCTS w/ Incident Model

Observations

  • Distance-reward weight ->

large impact on amount traveled

  • Lookahead Horizon and

Rebalance Period -> impact

  • n response time

distribution

*Other hyperparameters same as M-1

36

slide-38
SLIDE 38

World-class, interdisciplinary research with global impact.

Institute for Software Integrated Systems

Results – MMCTS w/ Oracle

Observations

  • Large potential

improvement

  • Despite increase in

distance moved, Queue rebalancing shows little improvement over static

  • More distance traveled

than queue heuristic approach

37

slide-39
SLIDE 39

World-class, interdisciplinary research with global impact.

Institute for Software Integrated Systems

Results – MMCTS vs Heuristic

MMCTS Queue Heuristic

Observations

  • Similar best-case

performance for each approach

38

slide-40
SLIDE 40

World-class, interdisciplinary research with global impact.

Institute for Software Integrated Systems

Key Takeaways

EMS Specific

  • Both approaches improve on baseline
  • MMCTS w/ oracle demonstrates entanglement with efficacy of incident model
  • MMCTS is more configurable than heuristic, but more sensitive to

hyperparameter choices

General:

  • Planning performance dependent on quality of underlying event prediction

models

  • Imperative to understand needs and constraints of target domain for it to be

implemented

  • Computational capacity of agents has evolved -> should use

Not feasible for real system

39

slide-41
SLIDE 41

World-class, interdisciplinary research with global impact.

Institute for Software Integrated Systems

Contact Info

  • Presenter – Geoffrey Pettet:
  • Geoffrey.a.pettet@Vanderbilt.edu
  • Collaborators:
  • Ayan Mukhopadhay
  • Mykel Kochenderfer,
  • Yevgeniy Voroybeychik,
  • Abhishek Dubey

40