rich feature hierarchies for accurate object detection
play

Rich feature hierarchies for accurate object detection and semantic - PowerPoint PPT Presentation

Sep 18, 2023 •329 likes •704 views

Rich feature hierarchies for accurate object detection and semantic segmentation Ross Girshick, Je ff Donahue, Trevor Darrell, Jitendra Malik UC Berkeley Tech Report @ http://arxiv.org/abs/1311.2524 Detection & Segmentation input

  1. Rich feature hierarchies for accurate object detection and semantic segmentation Ross Girshick, Je ff Donahue, Trevor Darrell, Jitendra Malik UC Berkeley � Tech Report @ http://arxiv.org/abs/1311.2524

  2. Detection & Segmentation input person background motorbike person motorbike

  3. PASCAL VOC Example PASCAL VOC images

  4. Dominant detection methods 1. Part-based sliding window methods (HOG) DPM Poselets 2. Region-proposal classifiers (SIFT++ BoW) Russell et al. 2006 Gu et al. 2009 van de Sande et al. 2011 > “selective search”

  5. PASCAL VOC epochs (detection) 2007-2010 The Moore’s law years � 2010-2011 The year of kitchen sinks (or the all-too-soon end of Moore’s law) � 2011-2012 Stagnation (no new features le fu , juice all squeezed from context) � 2013– Learning rich features?

  6. ImageNet LSVRC’12 winner UToronto “SuperVision” CNN Krizhevsky, Sutskever, and Hinton. ImageNet Classification with Deep Convolutional Neural Networks. NIPS 2012. � cf. LeCun et al. Neural Comp. ’89 & Proc. of the IEEE ‘98

  7. Impressive ImageNet results! Task: 1000-way whole-image classification metric: classification error rate (lower is better) But... does it generalize to other datasets and tasks? See: Donahue, Jia, et al. DeCAF Tech Report. � Much debate at ECCV’12

  8. Objective Understand if the SuperVision CNN can be made to work as an object detector.

  9. Object detection system R-CNN: “Regions with CNN features” warped region aeroplane ? no. . . . person? yes. . . . CNN tvmonitor? no. 1 . Input 2 . Extract region 3 . Compute 4 . Classify image proposals (~2k) CNN features regions (With a few minor tweaks: semantic segmentation) (e.g. selective search)

  10. Training 1. Pre-train CNN for image classification train CNN large auxiliary dataset (ImageNet)

  11. Training 1. Pre-train CNN for image classification train CNN large auxiliary dataset (ImageNet)

  12. Training 1. Pre-train CNN for image classification train CNN large auxiliary dataset (ImageNet) 2. Fine-tune CNN on target dataset and task fine-tune CNN (optional) small target dataset (PASCAL VOC)

  13. Training 1. Pre-train CNN for image classification train CNN large auxiliary dataset (ImageNet) 2. Fine-tune CNN on target dataset and task fine-tune CNN (optional) small target dataset (PASCAL VOC)

  14. Training 1. Pre-train CNN for image classification train CNN large auxiliary dataset (ImageNet) 2. Fine-tune CNN on target dataset and task fine-tune CNN (optional) small target dataset (PASCAL VOC) 3. Train linear predictor for detection CNN features region proposals per class ~2000 warped SVM windows / image small target training labels dataset (PASCAL VOC)

  15. Training labels 3. Train linear predictor for detection CNN features region proposals per class ~2000 warped SVM windows / image training labels small target dataset (PASCAL VOC) labeling protocol positives = ground truth negatives = max IoU < 0.3

  16. CNN features for detection warped region region pool 5 : 6 x 6 x 256 = 9216-dim 6.4% / 15% non-zero � fc 6 : 4096-dimensional 71.2% / 20% nz � fc 7 : 4096-dimensional 100% / 20% nz

  17. Results VOC 2007 VOC 2010 reference DPM v5 (Girshick et al. 2011) 33.7% 29.6% UVA sel. search (Uijlings et al. 2012) 35.1% Regionlets (Wang et al. 2013) 41.7% 39.7% R-CNN pool 5 40.1% R-CNN fc 6 43.4% R-CNN fc 7 42.6% R-CNN FT pool 5 42.1% R-CNN FT fc 6 47.2% R-CNN FT fc 7 48% 43.5% metric: mean average precision (higher is better)

  18. Results VOC 2007 VOC 2010 DPM v5 (Girshick et al. 2011) 33.7% 29.6% UVA sel. search (Uijlings et al. 2012) 35.1% Regionlets (Wang et al. 2013) 41.7% 39.7% pre-trained R-CNN pool 5 40.1% only R-CNN fc 6 43.4% R-CNN fc 7 42.6% R-CNN FT pool 5 42.1% R-CNN FT fc 6 47.2% R-CNN FT fc 7 48% 43.5% metric: mean average precision (higher is better)

  19. Results VOC 2007 VOC 2010 DPM v5 (Girshick et al. 2011) 33.7% 29.6% UVA sel. search (Uijlings et al. 2012) 35.1% Regionlets (Wang et al. 2013) 41.7% 39.7% R-CNN pool 5 40.1% R-CNN fc 6 43.4% R-CNN fc 7 42.6% fine-tuned R-CNN FT pool 5 42.1% R-CNN FT fc 6 47.2% R-CNN FT fc 7 48% 43.5% metric: mean average precision (higher is better)

  20. Results — update VOC 2007 VOC 2010 DPM v5 (Girshick et al. 2011) 33.7% 29.6% UVA sel. search (Uijlings et al. 2012) 35.1% Regionlets (Wang et al. 2013) 41.7% 39.7% pre-trained R-CNN pool 5 40.1% 44.0% only R-CNN fc 6 43.4% 46.2% R-CNN fc 7 42.6% 43.5% R-CNN FT pool 5 42.1% R-CNN FT fc 6 47.2% R-CNN FT fc 7 48% 43.5% metric: mean average precision (higher is better)

  21. CV and DL together Good features are not enough! warped region aeroplane ? no. . . . person? yes. . . . CNN tvmonitor? no. 1 . Input 2 . Extract region 3 . Compute 4 . Classify image proposals (~2k) CNN features regions Computer Deep Computer Vision Learning Vision

  22. Top bicycle FPs (AP 62.5%)

  23. Top bird FPs (AP 41.4%)

  24. False positive types: cat DPM voc − release5: cat CNN FT fc7: cat 100 100 percentage of each type percentage of each type 80 80 60 60 40 40 Loc Loc Sim Sim 20 20 Oth Oth BG BG 0 0 25 100 400 1600 6400 25 100 400 1600 6400 total false positives total false positives AP 56.3% AP 23.0% Analysis so fu ware from: D. Hoiem, Y. Chodpathumwan, and Q. Dai. “Diagnosing Error in Object Detectors.” ECCV , 2012.

  25. Visualizing features > What does pool 5 learn? > Recap: > pool 5 : max-pooled output of last conv. layer > 6 x 6 spatial structure (with 256 channels) > receptive field size 163 x 163 (of 224 x 224) 6 6 256 unit position receptive field

  26. Visualization method > Select a unit in pool 5 > Run it as a detector > Show top-scoring regions > Non-parametric, lets unit “speak for itself” � � � (Used ~10 million held-out regions.)

  27. pool5 feature: (3,3,42) (top 1 − 96) 0.9 0.8 0.8 0.7 0.7 0.7 0.7 0.7 0.7 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5

  28. pool5 feature: (3,4,80) (top 1 − 96) 0.9 0.8 0.8 0.8 0.7 0.7 0.6 0.6 0.6 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4

  29. pool5 feature: (4,5,110) (top 1 − 96) 0.8 0.8 0.8 0.7 0.7 0.7 0.7 0.7 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.3 0.3 0.3

  30. pool5 feature: (3,5,129) (top 1 − 96) 0.9 0.9 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6

  31. pool5 feature: (4,2,26) (top 1 − 96) 0.8 0.8 0.8 0.7 0.7 0.7 0.7 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5

  32. pool5 feature: (3,3,39) (top 1 − 96) 0.8 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation Authors: Ross

Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation Authors: Ross

Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation Authors: Ross Girshick, Jeff Donahue, Trevor Darrell, Jitendra Malik Presented by Huihuang Zheng Problem: Object Detection Regionlets SegDPM (2013) Selective

499 views • 19 slides
Convolutional Feature Maps Elements of efficient (and accurate) CNN-based object detection

Convolutional Feature Maps Elements of efficient (and accurate) CNN-based object detection

Convolutional Feature Maps Elements of efficient (and accurate) CNN-based object detection Kaiming He Microsoft Research Asia (MSRA) Overview of this section Quick introduction to convolutional feature maps Intuitions: into the black

786 views • 41 slides
3D Object Proposals using Stereo Imagery for Accurate Object Class Detection Xiaozhi Chen,

3D Object Proposals using Stereo Imagery for Accurate Object Class Detection Xiaozhi Chen,

3D Object Proposals using Stereo Imagery for Accurate Object Class Detection Xiaozhi Chen, Kaustav Kundu, Yukun Zhu, Huimin Ma, Sanja Fidler and Raquel Urtasun Presentation by Jungwook Lee Why use proposals? - Smart proposal generation methods

287 views • 25 slides
Bounding Box Regression With Uncertainty for Accurate Object Detection 1 Carnegie Mellon

Bounding Box Regression With Uncertainty for Accurate Object Detection 1 Carnegie Mellon

Bounding Box Regression With Uncertainty for Accurate Object Detection 1 Carnegie Mellon University 2 Megvii Yihui He 1 , Chenchen Zhu 1 , Jianren Wang 1 , Marios Savvides, 2 Xiangyu Zhang Ambiguity: inaccurate labelling MS-COCO Ambiguity:

713 views • 32 slides
S9243 Fast and Accurate Object Detection Floris Chabert , Solutions Architect with PyTorch and

S9243 Fast and Accurate Object Detection Floris Chabert , Solutions Architect with PyTorch and

S9243 Fast and Accurate Object Detection Floris Chabert , Solutions Architect with PyTorch and TensorRT Prethvi Kashinkunti , Solutions Architect March 19 2019 OVERVIEW Topics What &amp; Why? Problem Our solution How? Architecture

1.78k views • 34 slides
Feature Extraction Manos Baltsavias, Haris Papasaika OVERVIEW APPLICATIONS - Object detection

Feature Extraction Manos Baltsavias, Haris Papasaika OVERVIEW APPLICATIONS - Object detection

Feature Extraction Manos Baltsavias, Haris Papasaika OVERVIEW APPLICATIONS - Object detection and localisation (2D measurement) - Matching (3D information) - Image analysis and scene interpretation , e.g. Edgels -&gt; edge segments -&gt;

1.6k views • 84 slides
Feature Selection Matters for Anchor-Free Object Detection Chenchen Zhu Carnegie Mellon

Feature Selection Matters for Anchor-Free Object Detection Chenchen Zhu Carnegie Mellon

Carnegie Mellon Feature Selection Matters for Anchor-Free Object Detection Chenchen Zhu Carnegie Mellon University 04/29/2020 Carnegie Mellon Overview Background Motivation Feature Selection in Anchor-Free Detection General

466 views • 42 slides
Feature-level Phase Detection for Execution Trace Using Object Cache Yui Watanabe, Takashi Ishio,

Feature-level Phase Detection for Execution Trace Using Object Cache Yui Watanabe, Takashi Ishio,

Feature-level Phase Detection for Execution Trace Using Object Cache Yui Watanabe, Takashi Ishio, Katsuro Inoue Osaka University, Japan July 21, 2008 WODA 2008 1 Software Engineering Laboratory, Department of Computer Science, Graduate School

202 views • 19 slides
Object Detection Sanja Fidler CSC420: Intro to Image Understanding 1 / 48 Object Detection The

Object Detection Sanja Fidler CSC420: Intro to Image Understanding 1 / 48 Object Detection The

Object Detection Sanja Fidler CSC420: Intro to Image Understanding 1 / 48 Object Detection The goal of object detection is to localize objects in an image and tell their class Localization: place a tight bounding box around object Most

1.17k views • 91 slides
Automated Feature Extraction Automated Feature Extraction for Object Recognition for Object

Automated Feature Extraction Automated Feature Extraction for Object Recognition for Object

Automated Feature Extraction Automated Feature Extraction for Object Recognition for Object Recognition I.Levner and V.Bulitko http://ircl.cs.ualberta.ca Outline Outline Motivation System Overview Feature Extraction Problem

601 views • 15 slides
A Review on Salient Object Detection Feng Lin Salient Object Detection Target Detect and

A Review on Salient Object Detection Feng Lin Salient Object Detection Target Detect and

A Review on Salient Object Detection Feng Lin Salient Object Detection Target Detect and segment salient objects in natural scenes a) good detection b) high resolution c) computational efficiency Metric F-score MAE (mean

1.12k views • 39 slides
Detection, Segmentation Overview Object Detection deer cat Object Detection as Classification

Detection, Segmentation Overview Object Detection deer cat Object Detection as Classification

CS6501: Deep Learning for Visual Recognition Detection, Segmentation Overview Object Detection deer cat Object Detection as Classification deer? cat? CNN background? Object Detection as Classification deer? cat? CNN background?

577 views • 28 slides
Lecture 11: Object detection Contains slides from S. Lazebnik, R. Girshick, B. Hariharan 1

Lecture 11: Object detection Contains slides from S. Lazebnik, R. Girshick, B. Hariharan 1

Lecture 11: Object detection Contains slides from S. Lazebnik, R. Girshick, B. Hariharan 1 Object detection with bounding boxes What? Where? Object detection Source: R. Girshick 2 Evaluating an object detector At test time,

2.51k views • 56 slides
From image classification to object detection Image classification Object detection Image source

From image classification to object detection Image classification Object detection Image source

From image classification to object detection Image classification Object detection Image source Slides from L. Lazebnik What are the challenges of object detection? Images may contain more than one class, multiple instances from the same

1.11k views • 57 slides
Making Object Detection Work in Medical Imaging Idan Bassuk Segmentation Detection

Making Object Detection Work in Medical Imaging Idan Bassuk Segmentation Detection

Making Object Detection Work in Medical Imaging Idan Bassuk Segmentation Detection Classification 4 5 Mature technology 6 Mature, Accurate and Fast 7 Changing the world, one killer app at a time 8 Changing the world, one killer app at

570 views • 39 slides
Object based feature extraction of Google based feature extraction of Google Object Earth

Object based feature extraction of Google based feature extraction of Google Object Earth

International Electronic Conference on Sensors and Applications 1 16 June 2014 Object based feature extraction of Google based feature extraction of Google Object Earth i imagery for mapping termite mounds in magery for mapping termite

279 views • 9 slides
14:332:231 DIGITAL LOGIC DESIGN Ivan Marsic, Rutgers University Electrical &amp; Computer

14:332:231 DIGITAL LOGIC DESIGN Ivan Marsic, Rutgers University Electrical &amp; Computer

14:332:231 DIGITAL LOGIC DESIGN Ivan Marsic, Rutgers University Electrical &amp; Computer Engineering Fall 2013 Lecture #9: Combinational Logic Design Practices Signal Names and Active Levels Signal names are chosen to be descriptive

378 views • 13 slides
Unit 1 Circuit Basics KVL, KCL, Ohm's Law LED Outputs Buttons/Switch Inputs 1.2 VOLTAGE AND

Unit 1 Circuit Basics KVL, KCL, Ohm's Law LED Outputs Buttons/Switch Inputs 1.2 VOLTAGE AND

1.1 Unit 1 Circuit Basics KVL, KCL, Ohm's Law LED Outputs Buttons/Switch Inputs 1.2 VOLTAGE AND CURRENT 1.3 Current and Voltage - - - Charge is measured in units of Coulombs - - - - Current Amount of charge flowing

547 views • 28 slides
Unit 1 Circuit Basics KVL, KCL, Ohm's Law LED Outputs Buttons/Switch Inputs 1.2 VOLTAGE AND

Unit 1 Circuit Basics KVL, KCL, Ohm's Law LED Outputs Buttons/Switch Inputs 1.2 VOLTAGE AND

1.1 Unit 1 Circuit Basics KVL, KCL, Ohm's Law LED Outputs Buttons/Switch Inputs 1.2 VOLTAGE AND CURRENT 1.3 Current and Voltage - - - Charge is measured in units of Coulombs - - - - Current Amount of charge flowing

906 views • 31 slides
A Framework for Comparing Models for Adaptive Testing Jill-Jnn Vie February 19, 2016 Models

A Framework for Comparing Models for Adaptive Testing Jill-Jnn Vie February 19, 2016 Models

A Framework for Comparing Models for Adaptive Testing Jill-Jnn Vie February 19, 2016 Models for Adaptive Testing Framework, Experiment, Results NEW! Adaptive Submodularity Models for Adaptive Testing Computerized Adaptive Testing (CAT)

669 views • 30 slides
Return of the Devil in the Details: Delving Deep into Convolutional Nets Ken Chatfield, Karen

Return of the Devil in the Details: Delving Deep into Convolutional Nets Ken Chatfield, Karen

Return of the Devil in the Details: Delving Deep into Convolutional Nets Ken Chatfield, Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman Visual Geometry Group, Department of Engineering Science, Univesity of Oxford Hilal E. Akyz 1 2

908 views • 39 slides
The Global Atmosphere Watch (GAW) Reactive Gases Measurement Network Martin Schultz 1 , Hajime

The Global Atmosphere Watch (GAW) Reactive Gases Measurement Network Martin Schultz 1 , Hajime

The Global Atmosphere Watch (GAW) Reactive Gases Measurement Network Martin Schultz 1 , Hajime Akimoto 2 , Jan Bottenheim 3 , Brigitte Buchmann 4 , Ian Galbally 5 , Stefan Gilge 6 , Detlev Helmig 7 , Hiroshi Koide 8 , Alastair Lewis 9 , Paul Novelli

1.04k views • 27 slides
Continuous Improvement Toolkit QFD (Quality Function Deployment) Continuous Improvement Toolkit .

Continuous Improvement Toolkit QFD (Quality Function Deployment) Continuous Improvement Toolkit .

Continuous Improvement Toolkit QFD (Quality Function Deployment) Continuous Improvement Toolkit . www.citoolkit.com The Continuous Improvement Map Managing Deciding &amp; Selecting Planning &amp; Project Management* Risk PDPC Decision

690 views • 11 slides
Completeness of Queries over SQL Databases Werner Nutt and Simon Razniewski Introduction }

Completeness of Queries over SQL Databases Werner Nutt and Simon Razniewski Introduction }

Completeness of Queries over SQL Databases Werner Nutt and Simon Razniewski Introduction } Data Quality research investigates how good data is } Dimensions of Data Quality are: } Correctness } Timeliness } Completeness 2

894 views • 30 slides

More recommend