Classifying logistic vehicles in cities using Deep learning Salma - - PowerPoint PPT Presentation

Schlumberger-Private

Salma Benslimane, Simon Tamayo, Arnaud De La Fortelle Mines ParisTech – PSL Research University, Center for Robotics, Paris 75006, France Chair of Urban Logistics – City of Paris, La Poste, ADEME, Groupe Pomona, Renault

Classifying logistic vehicles in cities using Deep learning

1

15th World conference on Transport Research - 2019 Mumbai, 26-31 May 2019 To cite this work: Salma Benslimane, Simon Tamayo, Arnaud de La Fortelle. Classifying logistic vehicles in cities using Deep learning. World Conference on Transport Research, May 2019, Mumbai, India. ⟨hal-02144606⟩

Schlumberger-Private

OUTLINE

• Context
• Workflow
• Vehicle categorization
• Data acquisition
• Deep Learning Model
• Future Work

2

Schlumberger-Private

Context

Increase of small freight transportation vehicles inside cities Current use of costly, inaccurate physical sensors for vehicle counting Absence of precise information about congestion, pollution and freight effect on city

3

Schlumberger-Private

Which policies should Paris city implement as a response to e-commerce effect and structural changes on logistics and transportation?

Need for quantitative information about:

Number of vehicle Classification of freight vehicles’ by type Video based diagnosis E-commerce last mile delivery inside the city City planning and policies

Research question

4

Schlumberger-Private

Demo

5

Schlumberger-Private

Workflow

Counting and classifying logistic vehicles

6

Vehicle classification Data acquisition Data preprocessing Model training and evaluation

Select and define categories to classify Query images of the defined categories (balanced) Clean scrapped images and use a CNN to crop them Train a classifier for logistic vehicles detection

Schlumberger-Private

Vehicle categorization

4 categories for freight transportation in Paris

7

Heavy-duty

Text

Medium-duty

Text

Light-duty

Text

Non-Logistic

Text

Particularly used in France, not present in existing Datasets

GVM > 3.5 t H > 3 m GVM ≤ 3.5 t 2 m < H ≤ 3m GVM ≤ 3.5 t H ≤ 2 m GVM ≤ 2.7 t H ≤ 2 m

Schlumberger-Private

Data acquisition

Web scraping for balanced and personalized dataset

8

Search Engine (Google) ○

Define models per category

○

Scrape images per model and

• rganize them in categories

Results ○

Dataset of 4 categories

Limitations ○

Corrupt files

○

No vehicles in images

○

Background noise

○

Same angles of view

3D CAD website (hum3D) ○

Query categories of vehicles

○

Manually classify into caregories

Results ○

Dataset of 4 categories

Limitations ○

Synthetic images

○

No background noise

○

Unbalanced dataset Balanced, multiple view, diverse dataset with reduced manual annotation

Schlumberger-Private

• Cleaning the dataset from Google:

○ Deleting corrupt files ○ Correcting the extensions ○ Deleting images not containing vehicles using existing object classification neural netwoks

• Cleaning the entire dataset:

○ Delete noise of the background by applying convolutional neural network YOLO to the images and cropping only the vehicle

Dataset cleaning

9 Illustration of image processing by using DarkNet classifier to classify and crop the vehicle from the image

Schlumberger-Private

2 methods: ○

Train last layer (feature extraction) ○ Initialize weights of network with pretrained model on ImageNet (weight initialization)

Model training

Transfer learning

10

Schlumberger-Private

Transfer learning

Results

11

72,000 images CNN Model 90% Training 10% Testing CNN Model Accuracy Inception CNN 86.2 % MobileNet CNN 90.2 %

Schlumberger-Private

Transfer learning: Results

12

Feature extraction Weights initialization

Inception V3 MobileNetV2 Inception V3 MobileNetV2

Accuracy

86.2 % 90.1 % 91.8 % 90.47 %

Running time

194s 65s 60 minutes 44 minutes

MobileNet feature extraction normalized confusion matrix

Result of training Inception V3 and MobileNet V2 on the 72 000 images dataset Light-duty vehicle Passenger vehicle

Schlumberger-Private

Demo

13 Video demo of the pipeline output

Detect and classify logistic vehicles from a video feed -> count vehicle by type Summary

Schlumberger-Private

Future Work

• Higher category refinement in non-logistic

categories (adding SUVs, PickUps, etc.)

• Increase Database size & vehicle images’

angle of view

• Coordinating counting vehicles between

intersections

14

14

Schlumberger-Private

Thank you

15

Questions?

Schlumberger-Private 16

Context

Value of B2C e-commerce sales in France from 2012 to 2018 (in billion U.S. dollars)

Impact of e-commerce on rapid increase of freight transportation inside the cities Use of smaller capacity vehicles (SUVs, Vans) for last miles delivery Evolving size and structure of freight transportation fleet

Schlumberger-Private

Classifier Architecture

17

Category 2 Models.txt Category n Models.txt Category 2 Raw Images Category n Raw Images Category 1 Models.txt Category 1 Raw Images Category 2 Processed Images Category n Processed Images Category 1 Processed Images

MODEL CATEGORIZATION

Select and define categories to classify

WEB SCRAPING

Query images of the defined categories in balanced numbers

PROCESSING & CROPPING

Clean scrapped images and use a CNN to crop them

NETWORK TRAINING

Train a classifier for logistic vehicles detection

Schlumberger-Private

Large trucks Heavy-duty Medium-duty Remorques Camions Fourgons Intermediate duty Light-duty Non-logistic Fourgonnettes VUL Passenger

Initial vehicle categorization

18

Considering 6 categories of vehicles (5 logistics):

Careful: High overlap between classes

Note: this categorization is not used in existing dataset => existing datasets cannot be directly used for training

Schlumberger-Private

Heavy-duty Medium-duty Light-duty Non logistic

GVM > 3.5 t H > 3 m GVM ≤ 3.5 t 2 m < H ≤ 3m GVM ≤ 3.5 t H ≤ 2 m GVM ≤ 2.7 t H ≤ 2 m

Vehicle categorization

19

Considering 4 categories of vehicles (3 logistics) with less overlap:

Schlumberger-Private

Data acquisition:

Web scraping for balanced and personalized dataset

20

by model from a Search Engine (Google)

○ Define models per category ○ Scrape images per model and

• rganize them in categories

Results

○ Dataset of X categories

Limitations

○ Corrupt files ○ Mismatch of name extension and type ○ No vehicles in images ○ Background noise ○ Same angles of view

by subcategory from 3D CAD website (hum3D)

○ Query categories of vehicles ○ Manually classify into subtypes

Results

○ Dataset of X categories

Limitations

○ Synthetic images ○ No background noise ○ Unbalanced dataset

Aim

• Construct a balanced dataset with minimum manual labellisation by querying images of vehicles

Schlumberger-Private

• 2 methods explored:

○

Train last layer (feature extraction) ○ Initialize weights of network with pretrained model on ImageNet (weight initialization)

Transfer learning

21 Figure 18: Representation of transfer learning architecture

Schlumberger-Private

Transfer learning: Results

22

6 CATEGORY DS 6000 images 4 CATEGORY DS 4000 images 4 CATEGORY DS 60 000 images 4 CATEGORY UNBALANCED DS 150 000 images 4 CATEGORY DS 72 000 images

74 %

• 82 %

88 % 93 % 96 % 86 %

• 86 %

90 %

Figure 19: Results of iterations for training convolutional networks on generated datasets

Schlumberger-Private

Extended video

23