Time Predictions in Uber Eats Zi Wang@Uber QCon New York 2019 - - PowerPoint PPT Presentation

▶
time predictions in uber eats
SMART_READER_LITE
LIVE PREVIEW

Time Predictions in Uber Eats Zi Wang@Uber QCon New York 2019 - - PowerPoint PPT Presentation

Jun 05, 2023 412 likes •956 views

Time Predictions in Uber Eats Zi Wang@Uber QCon New York 2019 June 2019 Agenda 1. ML in Uber Eats Goals & Challenges ML Platform @ Uber 2. How Time Predictions Power Dispatch System 3. Deep Dive in Time Predictions Food

slide-1
SLIDE 1

June 2019

Time Predictions in Uber Eats

Zi Wang@Uber QCon New York 2019

slide-2
SLIDE 2
  • 1. ML in Uber Eats
  • Goals & Challenges
  • ML Platform @ Uber
  • 2. How Time Predictions Power Dispatch System
  • 3. Deep Dive in Time Predictions
  • Food Preparation Time Prediction
  • Delivery Time Estimation
  • Travel Time Estimation
  • 4. Q&A

Agenda

slide-3
SLIDE 3

ML in Uber Eats

slide-4
SLIDE 4
  • Goals & Challenges
  • ML Platform @ Uber

Agenda

slide-5
SLIDE 5

~ 8B

Gross Bookings for 2018

> 500 Cities

Our Scale

> 220,000 Restaurant Partners

slide-6
SLIDE 6

Make eating well effortless, every day, for everyone. Our Mission

slide-7
SLIDE 7

Goals & Challenges

Predicting the Future Network Efficiency Food Discovery

Reliable Affordable Effortless

slide-8
SLIDE 8

Eyeball ETD prediction ETD prediction ETA prediction Prep-time prediction

eater browsing

  • rder

dispatch delivery-partner food delivery-partner delivery-partner food created arrival ready begins trip arrival dropped-off

slide-9
SLIDE 9

ML Platform @ Uber

slide-10
SLIDE 10
slide-11
SLIDE 11

Feature Report

slide-12
SLIDE 12

Model Accuracy Report

slide-13
SLIDE 13

How Time Predictions Power Dispatch System

slide-14
SLIDE 14
  • Overview of Dispatch System
  • Evolution via Time Predictions

○ Dispatch System w/o Time Predictions ○ Dispatch System w/ Time Predictions

Agenda

slide-15
SLIDE 15

Make Demand-Supply Matching Decisions

Challenges

  • Solve an NP-Hard problem with a large problem space within seconds
  • Improve efficiency without compromising delivery quality
  • Eater & Restaurant & Delivery Partner
slide-16
SLIDE 16
  • Fast drop-off
  • Low delivery fee
  • 24/7

Eater Restaurant Partner

  • Short wait time
  • Low Unfulfillment

Delivery Partner

  • Short wait time
  • Smart route planning
  • Quick hand-off

Eater & Restaurant & Delivery Partner

slide-17
SLIDE 17

Matching Algorithm:

An Augmented Vehicle Routing Problem (VRP)

slide-18
SLIDE 18

Dispatch System w/o Time Predictions

slide-19
SLIDE 19

Scheduled pick-up time Order Created 9:00 8:30 do not dispatch a delivery-partner dispatch a delivery-partner Fixed 7 mins

When to Dispatch?

8:53 1st dispatch attempt ...

slide-20
SLIDE 20

How to Dispatch? (Greedy)

  • Jobs dispatched

independently without considering other jobs.

slide-21
SLIDE 21

Before...

restaurant-partner delivery-partner eater marketplace

  • Where is my food?
  • Food is cold
  • How much longer do I have to wait?
slide-22
SLIDE 22

Dispatch System w/ Time Predictions

slide-23
SLIDE 23

Predicted pick-up time Order created 9:00 8:56 8:30 do not dispatch a driver dispatch a driver ETA 8:50 1st dispatch attempt ...

When to Dispatch?

... nth dispatch attempt

slide-24
SLIDE 24

How to Dispatch? (Global)

  • All jobs and supplies

are considered at the same time.

slide-25
SLIDE 25
  • Then we solve the

entire set of jobs and supplies as a single global optimization problem.

slide-26
SLIDE 26

Greedy Global

#2 #1

slide-27
SLIDE 27
slide-28
SLIDE 28

After...

restaurant delivery partner eater marketplace

  • Fast delivery times
  • Accurate ETD estimations
  • Track food location
  • Prevent delivery partners from waiting around
  • Prevent food waiting for delivery partners
  • Track delivery partner’s location
  • Dispatch delivery partners at the right time
  • Maintain supply/demand, prevent surge
  • Reduce waiting at restaurants
  • Maximize earning potential
  • Be aware of estimated travel time
slide-29
SLIDE 29

Deep Dive in Time Predictions

slide-30
SLIDE 30
  • Food Preparation Time Prediction
  • Delivery Time Estimation
  • Travel Time Estimation

Agenda

slide-31
SLIDE 31

Food Preparation Time Prediction

slide-32
SLIDE 32

Why is Predicting Food Prep-time Difficult?

  • 1) True restaurant prep-time is unknown!

○ Example: We need to infer true prep-time in a retrospective manner based on restaurants and delivery partners’ signals.

  • 2) Prediction with limited signals

○ Example: The busyness in the actual restaurant is unknown

slide-33
SLIDE 33

How Did We Use ML to Solve the Problem?

  • Feature engineering
  • ML Model
  • Feedback Loop
slide-34
SLIDE 34

Feature Engineering

  • Historical features

○ Avg prep-time for 1 week, ...

  • Real-time (Contextual) features

○ Time of day, day of week, order size, location, ...

  • Near real-time features

○ Avg prep-time for last 10 mins, ...

slide-35
SLIDE 35

Representation Learning

slide-36
SLIDE 36

Sensor Signals

slide-37
SLIDE 37

Conditional Random Field Model

37

slide-38
SLIDE 38

Data preparation pipelines push data into the Feature Store tables and training data repositories.

Time of day, day of week,

  • rder size,

location, ...

Sensing & Perception Bluetooth Data

Feature Engineering (Cont’d) - Data Pipeline

Preparation time

slide-39
SLIDE 39

ML Model

  • Model: Gradient boosting decision trees (XGBoost)
  • Historical features
  • Realtime (Contextual)s features
  • Near real-time features
slide-40
SLIDE 40

Hyperparameter tuning

Image source: www.nature.com/articles/nature14541

slide-41
SLIDE 41

Model Training

Model training jobs use Feature Store and training data repository data sets to train models and then push them to the model repository.

slide-42
SLIDE 42

Model Training (Cont’d) - Model Deployment

slide-43
SLIDE 43

Model Training (Cont’d) - Make Predictions

slide-44
SLIDE 44

Historical features

E.g. average prep-time in last week

Near real time features

E.g. average prep-time in last few minutes

Real time features

E.g. order size, time of day

Michelangelo model training Production model (GBDT) Predicted prep-time Updated Data Online prediction Offline training

ML Model with Feedback Loop

slide-45
SLIDE 45
  • Ground truth exploration

○ Experiment in restaurants ○ ...

  • Improving ML model

○ Feature engineering ■ Exploration of places, weather, and event data ■ Model partitioning ■ ... ○ Leverage ensemble learning (stacking) ○ Collaboration with AI Labs on more deep learning models

Future Improvements

slide-46
SLIDE 46

Delivery Time Estimation

slide-47
SLIDE 47

Eater-facing ETD

EATER RESTAURANT DELIVERY PARTNER

0) eyeball 1) order 10) food created received 2) order 6) food accepted ready 4) accepted trip 8) begun trip 11) ended trip 3) dispatched 5) arrived 7) departed 9) arrived restaurant-leg delivery-leg

  • bservable state

not-observable state

slide-48
SLIDE 48

Why is predicting ETD difficult?

EATER RESTAURANT DELIVERY PARTNER

0) eyeball 1) order 10) food created received 2) order 6) food accepted ready 3) dispatched 5) arrived 7) departed 9) arrived 4) accepted trip 8) begun trip 11) ended trip restaurant-leg delivery-leg

  • bservable state

not-observable state delay early pick request

slide-49
SLIDE 49

Travel Time Estimation

slide-50
SLIDE 50

Rider Rider - Request Ride Driver Rider - On Trip

slide-51
SLIDE 51

Credits

Teams @ Uber

Special thanks to:

  • Engineers
  • Data Scientists
  • Product managers
  • Product Ops
  • Data Analysts
slide-52
SLIDE 52

THANK YOU Q & A

slide-53
SLIDE 53