Context for Efficient Recognition from Large-Scale Topographic Map - - PowerPoint PPT Presentation

▶
context for efficient recognition from
SMART_READER_LITE
LIVE PREVIEW

Context for Efficient Recognition from Large-Scale Topographic Map - - PowerPoint PPT Presentation

Oct 03, 2022 360 likes •1.25k views

ICC 2017 Washington D.C, USA July 02-07, 2017 Implementing the Concept of Geographic Context for Efficient Recognition from Large-Scale Topographic Map Series Johannes H. Uhl 1 Research Team: Stefan Leyk 1 , Yao-Yi Chiang 2 , Weiwei Duan 3 ,

slide-1
SLIDE 1

Johannes H. Uhl1 Research Team: Stefan Leyk1, Yao-Yi Chiang2, Weiwei Duan3, Vinil Jain2, Dan Feldman2, Craig Knoblock3

Implementing the Concept of Geographic Context for Efficient Recognition from Large-Scale Topographic Map Series

ICC 2017 Washington D.C, USA July 02-07, 2017

1 Department of Geography

University of Colorado Boulder

3 Computer Science Department

University of Southern California

2 Spatial Sciences Institute

University of Southern California

slide-2
SLIDE 2

A Case Study and Outlook

Outline

Map Processing: Impact & Challenges Geographic Context & Map Processing

slide-3
SLIDE 3

I MAP PROCESSING: IMPACT & CHALLENGES II THE PRINCIPLE OF GEOGRAPHIC CONTEXT III CASE STUDY:

Recognition of Buildings and Urban Areas in Historical Topographic Maps

A Case Study and Outlook

Outline

Map Processing: Impact & Challenges Geographic Context & Map Processing

slide-4
SLIDE 4

I Map Processing: Impact & Challenges

Map Processing: Impact & Challenges

slide-5
SLIDE 5

(b) Royal Prussian Surveying Unit, Map of Western Russia, 1915, 1:100K (a) Military Geographical Institute, Poland 1930, 1:25K (d) Swiss Federal Topographic Bureau, Swiss Topographic Map (Siegfried Map), 1912, 1:25K (c) Imperial and Royal Military Geographical Institute, Austria, Map

  • f the Austrian-Hungarian Monarchy

and foreign map pages, Russia, 1878, 1:75K

The Impact of Map Processing

Map Processing: Impact & Challenges

Preserving unique witnesses of the past unlocking geographic information

slide-6
SLIDE 6

The Impact of Map Processing

Map Processing: Impact & Challenges

slide-7
SLIDE 7
  • Map processing = Recognition + Extraction
  • Pattern recognition, computer vision, machine learning...
  • Creating GIS-readable data from scanned map archives
  • Retrospective Landscape Analysis
  • Historians, Geographers, Demographers, Landscape

Ecologists, etc…

The Impact of Map Processing

Map Processing: Impact & Challenges

slide-8
SLIDE 8
  • Map processing = Recognition + Extraction
  • Pattern recognition, computer vision, machine learning...
  • Creating GIS-readable data from scanned map archives
  • Retrospective Landscape Analysis
  • Historians, Geographers, Demographers, Landscape

Ecologists, etc…

The Impact of Map Processing

Map Processing: Impact & Challenges

slide-9
SLIDE 9

Current Challenges in Map Processing

Map Processing: Impact & Challenges

slide-10
SLIDE 10

Current Challenges in Map Processing

Map Processing: Impact & Challenges

slide-11
SLIDE 11
  • Complexity, graphical quality, data volume
  • User interaction  Low levels of automation

in information extraction

Current Challenges in Map Processing

Map Processing: Impact & Challenges

slide-12
SLIDE 12
  • Complexity, graphical quality, data volume
  • User interaction  Low levels of automation

in information extraction

Current Challenges in Map Processing

Map Processing: Impact & Challenges

slide-13
SLIDE 13
  • Complexity, graphical quality, data volume
  • User interaction  Low levels of automation

in information extraction

Current Challenges in Map Processing

Map Processing: Impact & Challenges

slide-14
SLIDE 14

Current Challenges in Map Processing

Map Processing: Impact & Challenges

Map recognition involving user interaction:

slide-15
SLIDE 15

Current Challenges in Map Processing

Map Processing: Impact & Challenges

Label Learn Extract

Map recognition involving user interaction:

slide-16
SLIDE 16

Current Challenges in Map Processing

Map Processing: Impact & Challenges

Label Learn Extract

Map recognition involving user interaction:

slide-17
SLIDE 17

Current Challenges in Map Processing

Map Processing: Impact & Challenges

Label Learn Extract

Map recognition involving user interaction:

shape, color & gradient descriptors

slide-18
SLIDE 18

Current Challenges in Map Processing

Map Processing: Impact & Challenges

Label Learn Extract

Map recognition involving user interaction:

shape, color & gradient descriptors

slide-19
SLIDE 19

Current Challenges in Map Processing

Map Processing: Impact & Challenges

Label Learn Extract

Map recognition involving user interaction:

How to overcome user labeling to achieve higher levels of automation?

shape, color & gradient descriptors

slide-20
SLIDE 20

II The Principle of Geographic Context

Effective use of external (geographic) data for improved information extraction from maps

Geographic Context & Map Processing

slide-21
SLIDE 21
  • Map series in digital archives
  • Large data volume
  • Dependent editions with

incremental change (updates)

  • Overlap in content to guide

learning?

Geographic Context?

1950

Geographic Context & Map Processing

slide-22
SLIDE 22
  • Map series in digital archives
  • Large data volume
  • Dependent editions with

incremental change (updates)

  • Overlap in content to guide

learning?

Geographic Context?

1964 1950

Geographic Context & Map Processing

slide-23
SLIDE 23
  • Map series in digital archives
  • Large data volume
  • Dependent editions with

incremental change (updates)

  • Overlap in content to guide

learning?

Geographic Context?

1964 1950

Geographic Context & Map Processing

slide-24
SLIDE 24
  • Map series in digital archives
  • Large data volume
  • Dependent editions with

incremental change (updates)

  • Overlap in content to guide

learning?

Geographic Context?

2012 1964 1950

Geographic Context & Map Processing

slide-25
SLIDE 25
  • Map series in digital archives
  • Large data volume
  • Dependent editions with

incremental change (updates)

  • Overlap in content to guide

learning?

Geographic Context?

2012 1964 1950

Geographic Context & Map Processing

slide-26
SLIDE 26
  • Map series in digital archives
  • Large data volume
  • Dependent editions with

incremental change (updates)

  • Overlap in content to guide

learning?

Geographic Context?

2012 1964 1950

Geographic Context & Map Processing

slide-27
SLIDE 27
  • Map series in digital archives
  • Large data volume
  • Dependent editions with

incremental change (updates)

  • Overlap in content to guide

learning?

  • Generic (not independent)

ancillary data representing feature

  • f interest
  • Know “where to expect” the

feature of interest

Geographic Context?

2012 1964 1950

Geographic Context & Map Processing

slide-28
SLIDE 28

(1) Creating contextual information

  • Geometry
  • Attributes

Information Extraction & Geographic Context

Geographic Context & Map Processing

slide-29
SLIDE 29

(1) Creating contextual information

  • Geometry
  • Attributes

Information Extraction & Geographic Context

Gazetteer

Admin Records (x,y)

Geographic Context & Map Processing

Vector data (VGI…)

slide-30
SLIDE 30

(1) Creating contextual information

  • Geometry
  • Attributes

Information Extraction & Geographic Context

Gazetteer

Admin Records (x,y)

Geographic Context & Map Processing

Vector data (VGI…)

slide-31
SLIDE 31

(1) Creating contextual information

  • Geometry
  • Attributes

(2) Adaptive graphics sampling

  • Collect spatially constrained graphics examples
  • Assume overlap: map & context

Information Extraction & Geographic Context

Gazetteer

Admin Records (x,y)

Geographic Context & Map Processing

Vector data (VGI…)

slide-32
SLIDE 32

(1) Creating contextual information

  • Geometry
  • Attributes

(2) Adaptive graphics sampling

  • Collect spatially constrained graphics examples
  • Assume overlap: map & context

Information Extraction & Geographic Context

Gazetteer

Admin Records (x,y)

Geographic Context & Map Processing

Vector data (VGI…)

slide-33
SLIDE 33

(1) Creating contextual information

  • Geometry
  • Attributes

(2) Adaptive graphics sampling

  • Collect spatially constrained graphics examples
  • Assume overlap: map & context

Information Extraction & Geographic Context

Gazetteer

Admin Records (x,y)

Geographic Context & Map Processing

Vector data (VGI…)

slide-34
SLIDE 34

(1) Creating contextual information

  • Geometry
  • Attributes

(2) Adaptive graphics sampling

  • Collect spatially constrained graphics examples
  • Assume overlap: map & context

(3) Compute feature descriptors: Create knowledge base

  • Shape, color, texture descriptors
  • To be used in learning and extraction

Information Extraction & Geographic Context

Gazetteer

Admin Records (x,y)

Geographic Context & Map Processing

Vector data (VGI…)

slide-35
SLIDE 35

(1) Creating contextual information

  • Geometry
  • Attributes

(2) Adaptive graphics sampling

  • Collect spatially constrained graphics examples
  • Assume overlap: map & context

(3) Compute feature descriptors: Create knowledge base

  • Shape, color, texture descriptors
  • To be used in learning and extraction

Information Extraction & Geographic Context

Gazetteer

Admin Records (x,y)

Geographic Context & Map Processing

Step (1) and (2): Eliminate user interaction

Vector data (VGI…)

slide-36
SLIDE 36

III Case Study

Geographic context for automated map symbol recognition: Buildings and Urban Areas

Case Study: Building and Urban Area Extraction

slide-37
SLIDE 37

The Experiment

Case Study: Building and Urban Area Extraction

slide-38
SLIDE 38

The Experiment

Case Study: Building and Urban Area Extraction

slide-39
SLIDE 39

The Experiment

Case Study: Building and Urban Area Extraction

slide-40
SLIDE 40

The Experiment

Case Study: Building and Urban Area Extraction

slide-41
SLIDE 41

The Experiment

Geographic Context Buildings

Case Study: Building and Urban Area Extraction

slide-42
SLIDE 42

The Experiment

Geographic Context Buildings

Case Study: Building and Urban Area Extraction

slide-43
SLIDE 43

The Experiment

Geographic Context Buildings

Case Study: Building and Urban Area Extraction

slide-44
SLIDE 44

The Experiment

Geographic Context Buildings

Spatial offsets Temporal inconsistencies Generalization effects

Case Study: Building and Urban Area Extraction

slide-45
SLIDE 45

The Experiment

Geographic Context Buildings

Spatial offsets Temporal inconsistencies Generalization effects

Case Study: Building and Urban Area Extraction

  • Preprocessing
  • Graphics sampling
  • Sample cleaning
  • Learning
  • Recognition
  • Extracted buildings

& urban areas

slide-46
SLIDE 46

Discrepancies between contextual and map data

Case Study: Building and Urban Area Extraction

slide-47
SLIDE 47

Discrepancies between contextual and map data

Case Study: Building and Urban Area Extraction

slide-48
SLIDE 48

Discrepancies between contextual and map data

Temporal inconsistencies Generalization effects

Case Study: Building and Urban Area Extraction

slide-49
SLIDE 49

Spatial offsets

Discrepancies between contextual and map data

Temporal inconsistencies Generalization effects

Case Study: Building and Urban Area Extraction

slide-50
SLIDE 50

Spatial offsets

Discrepancies between contextual and map data

Temporal inconsistencies Generalization effects Distortions introduced during georeferencing

Case Study: Building and Urban Area Extraction

slide-51
SLIDE 51

Spatial offsets

Discrepancies between contextual and map data

Temporal inconsistencies Generalization effects Spatial offsets Distortions introduced during georeferencing

Case Study: Building and Urban Area Extraction

slide-52
SLIDE 52

Guided Graphics Sampling

Case Study: Building and Urban Area Extraction

…using image processing / computer vision techniques

“Cleaning” the samples…

slide-53
SLIDE 53

Clean graphic samples for learning process

Case Study: Building and Urban Area Extraction

slide-54
SLIDE 54

Clean graphic samples for learning process

Case Study: Building and Urban Area Extraction

t-distributed stochastic neighbor embedding (t-SNE) plots for visual quality assessment

slide-55
SLIDE 55

Clean graphic samples for learning process

Case Study: Building and Urban Area Extraction

t-distributed stochastic neighbor embedding (t-SNE) plots for visual quality assessment

slide-56
SLIDE 56

Clean graphic samples for learning process

Case Study: Building and Urban Area Extraction

t-distributed stochastic neighbor embedding (t-SNE) plots for visual quality assessment

slide-57
SLIDE 57

Feature extraction based on convolutional neural networks

Case Study: Building and Urban Area Extraction

slide-58
SLIDE 58

Feature extraction based on convolutional neural networks

Case Study: Building and Urban Area Extraction

slide-59
SLIDE 59

Feature extraction based on convolutional neural networks

Case Study: Building and Urban Area Extraction

Input data 42x42x3 Convolution + ReLU 20x5x5 MaxPooling 2x2 Convolution + ReLU 50x15x15 MaxPooling 2x2 Fully connected + ReLU (500) Fully connected + SoftMax (3) = Class scores (urban, single bldg., no building)

slide-60
SLIDE 60

Feature extraction based on convolutional neural networks

Case Study: Building and Urban Area Extraction

Input data 42x42x3 Convolution + ReLU 20x5x5 MaxPooling 2x2 Convolution + ReLU 50x15x15 MaxPooling 2x2 Fully connected + ReLU (500) Fully connected + SoftMax (3) = Class scores (urban, single bldg., no building)

LeNet architecture

slide-61
SLIDE 61

Preliminary Experimental Results

Case Study: Building and Urban Area Extraction

slide-62
SLIDE 62

Preliminary Experimental Results

Buildings No buildings Urban area

Case Study: Building and Urban Area Extraction

slide-63
SLIDE 63

Preliminary Experimental Results

Buildings No buildings Urban area No building Single building urban

Case Study: Building and Urban Area Extraction

slide-64
SLIDE 64

Preliminary Experimental Results

Label w/ max probability, prediction using stride=20

No building Single building urban

Case Study: Building and Urban Area Extraction

slide-65
SLIDE 65

Preliminary Experimental Results

Label w/ max probability, prediction using stride=20

No building Single building urban Percentage of correctly classified (PCC) = 0.81 Kappa index = 0.66 Normalized Mutual Information (NMI) = 0.46

Case Study: Building and Urban Area Extraction

slide-66
SLIDE 66

Preliminary Experimental Results

Label w/ max probability, prediction using stride=20

No building Single building urban Percentage of correctly classified (PCC) = 0.81 Kappa index = 0.66 Normalized Mutual Information (NMI) = 0.46

Case Study: Building and Urban Area Extraction

Class Precision Recall No buildings 0.98 0.70 Urban area 0.85 0.98 Individual buildings 0.06 0.99

slide-67
SLIDE 67

Preliminary Experimental Results

Label w/ max probability, prediction using stride=20

No building Single building urban Percentage of correctly classified (PCC) = 0.81 Kappa index = 0.66 Normalized Mutual Information (NMI) = 0.46

Case Study: Building and Urban Area Extraction

Class Precision Recall No buildings 0.98 0.70 Urban area 0.85 0.98 Individual buildings 0.06 0.99

slide-68
SLIDE 68

Discussion

  • Availability of contextual geographic data + machine learning:
  • Great potential for fully automatic map recognition
  • External (but not independent) contextual information:
  • Efficiently guides graphics sampling
  • Elimination of user intervention:
  • Necessary to exploit large volumes of digital historical map archives
slide-69
SLIDE 69

Acknowledgements

US National Science Foundation award IIS 1563933 to the University of Colorado at Boulder and IIS 1564164 to the University of Southern California “Exploiting Context in Cartographic Evolutionary Documents to Extract and Build Linked Spatial-temporal Datasets”

slide-70
SLIDE 70

Additional material

slide-71
SLIDE 71

Guided Graphics Sampling

Case Study: Building and Urban Area Extraction

slide-72
SLIDE 72

Guided Graphics Sampling

Step 1 Positive sub-images Urban areas No urban areas Positive samples (urban areas) Building centroids Sub-image cropping

Case Study: Building and Urban Area Extraction

slide-73
SLIDE 73

Guided Graphics Sampling

Step 2 Step 1 Positive sub-images Urban areas No urban areas Positive samples (urban areas) Building centroids

Positive samples (Individual buildings)

discard Individual buildings No individual buildings Sub-image cropping

Case Study: Building and Urban Area Extraction

slide-74
SLIDE 74

Guided Graphics Sampling

Step 2 Step 1 Positive sub-images Urban areas No urban areas Positive samples (urban areas) Building centroids Negative areas

Positive samples (Individual buildings)

Negative samples (no buildings) discard Buffering Negative area creation Individual buildings No individual buildings Sub-image cropping Sub-image cropping Random locations Step 3

Case Study: Building and Urban Area Extraction

slide-75
SLIDE 75

Guided Graphics Sampling

Step 1 Positive sub-images Urban areas No urban areas Positive samples (urban areas) Building centroids Sub-image cropping

Case Study: Building and Urban Area Extraction

slide-76
SLIDE 76

Guided Graphics Sampling

Step 1 Positive sub-images Urban areas No urban areas Positive samples (urban areas) Building centroids Sub-image cropping

green/red colour ratio

  • f the dominant colour

< 0.8?

Case Study: Building and Urban Area Extraction

slide-77
SLIDE 77

Guided Graphics Sampling

Step 1 Positive sub-images Urban areas No urban areas Positive samples (urban areas) Building centroids Sub-image cropping

green/red colour ratio

  • f the dominant colour

< 0.8?

Case Study: Building and Urban Area Extraction

slide-78
SLIDE 78

Guided Graphics Sampling

Step 2 Step 1 Positive sub-images Urban areas No urban areas Building centroids

Positive samples (Individual buildings)

discard Individual buildings No individual buildings Sub-image cropping

Case Study: Building and Urban Area Extraction

slide-79
SLIDE 79

Guided Graphics Sampling

Step 2 Step 1 Positive sub-images Urban areas No urban areas Building centroids

Positive samples (Individual buildings)

discard Individual buildings No individual buildings Sub-image cropping

Case Study: Building and Urban Area Extraction

slide-80
SLIDE 80

Guided Graphics Sampling

Step 2 Step 1 Positive sub-images Urban areas No urban areas Building centroids

Positive samples (Individual buildings)

discard Individual buildings No individual buildings Sub-image cropping

Case Study: Building and Urban Area Extraction

 Maxima in the difference

  • f Gaussian (DoG) scale

space  DoG max at the center

  • f a building
slide-81
SLIDE 81

Guided Graphics Sampling

Step 2 Step 1 Positive sub-images Urban areas No urban areas Positive samples (urban areas) Building centroids Negative areas Negative samples (no buildings) discard Buffering Negative area creation Individual buildings No individual buildings Sub-image cropping Sub-image cropping Random locations Step 3

Case Study: Building and Urban Area Extraction

slide-82
SLIDE 82

Guided Graphics Sampling

Step 2 Step 1 Positive sub-images Urban areas No urban areas Positive samples (urban areas) Building centroids Negative areas Negative samples (no buildings) discard Buffering Negative area creation Individual buildings No individual buildings Sub-image cropping Sub-image cropping Random locations Step 3

Building centroids

Case Study: Building and Urban Area Extraction

slide-83
SLIDE 83

Guided Graphics Sampling

Step 2 Step 1 Positive sub-images Urban areas No urban areas Positive samples (urban areas) Building centroids Negative areas Negative samples (no buildings) discard Buffering Negative area creation Individual buildings No individual buildings Sub-image cropping Sub-image cropping Random locations Step 3

Building centroids Building buffers

Case Study: Building and Urban Area Extraction

slide-84
SLIDE 84

Guided Graphics Sampling

Step 2 Step 1 Positive sub-images Urban areas No urban areas Positive samples (urban areas) Building centroids Negative areas Negative samples (no buildings) discard Buffering Negative area creation Individual buildings No individual buildings Sub-image cropping Sub-image cropping Random locations Step 3

Building centroids Building buffers No buildings

Case Study: Building and Urban Area Extraction

slide-85
SLIDE 85

Guided Graphics Sampling

Step 2 Step 1 Positive sub-images Urban areas No urban areas Positive samples (urban areas) Building centroids Negative areas Negative samples (no buildings) discard Buffering Negative area creation Individual buildings No individual buildings Sub-image cropping Sub-image cropping Random locations Step 3

Building centroids Building buffers No buildings Negative samples No building

Case Study: Building and Urban Area Extraction

slide-86
SLIDE 86

Guided Graphics Sampling

Step 2 Step 1 Positive sub-images Urban areas No urban areas Positive samples (urban areas) Building centroids Negative areas Negative samples (no buildings) discard Buffering Negative area creation Individual buildings No individual buildings Sub-image cropping Sub-image cropping Random locations Step 3

Building centroids Building buffers No buildings Negative samples No building

 Sample of 10,000 graphics labels  Oversampling urban and single building to N=10,000

Case Study: Building and Urban Area Extraction

slide-87
SLIDE 87

Guided Graphics Sampling

Step 2 Step 1 Positive sub-images Urban areas No urban areas Positive samples (urban areas) Building centroids Negative areas Negative samples (no buildings) discard Buffering Negative area creation Individual buildings No individual buildings Sub-image cropping Sub-image cropping Random locations Step 3

Building centroids Building buffers No buildings Negative samples No building

 Sample of 10,000 graphics labels  Oversampling urban and single building to N=10,000

Case Study: Building and Urban Area Extraction

 Graphics samples as input data for convolutional neural network