S8941 SYNTHETIC LABEL DATA FOR TRAINING DEEP LEARNING ISR - - PowerPoint PPT Presentation

s8941 synthetic label data for training deep
SMART_READER_LITE
LIVE PREVIEW

S8941 SYNTHETIC LABEL DATA FOR TRAINING DEEP LEARNING ISR - - PowerPoint PPT Presentation

Place image here (13.33 x 3.5) S8941 SYNTHETIC LABEL DATA FOR TRAINING DEEP LEARNING ISR ALGORITHMS WILL RORRER, PROGRAM MANAGER NVIDIA GTC San Jose 29 March 2018 NON-EXPORT CONTROLLED THESE ITEM(S) / DATA HAVE BEEN REVIEWED IN


slide-1
SLIDE 1

NON-EXPORT CONTROLLED

THESE ITEM(S) / DATA HAVE BEEN REVIEWED IN ACCORDANCE WITH THE INTERNATIONAL TRAFFIC IN ARMS REGULATIONS (ITAR), 22 CFR PART 120.11, AND THE EXPORT ADMINISTRATION REGULATIONS (EAR), 15 CFR 734(3)(b)(3), AND MAY BE RELEASED WITHOUT EXPORT RESTRICTIONS.

HARRIS.COM | #HARRISCORP

Place image here (13.33” x 3.5”)

S8941 – SYNTHETIC LABEL DATA FOR TRAINING DEEP LEARNING ISR ALGORITHMS

WILL RORRER, PROGRAM MANAGER

NVIDIA GTC San Jose 29 March 2018

slide-2
SLIDE 2

| 2 S8941 – Synthetic Label Data for Training Deep Learning ISR Algorithms NON-Export Controlled Information

Agenda

Harris Corporation Introduction A Call to Action: The Urgency Behind the DoD’s Adoption of AI Review: Applications of Deep Learning at Harris Review: Harris’ Work to Scale Deep Learning Harris’ Approach for Handling the Label Data Burden Q&A

slide-3
SLIDE 3

| 3 S8941 – Synthetic Label Data for Training Deep Learning ISR Algorithms NON-Export Controlled Information

Harris Corporation introduction – segment overviews

Space and Intelligence Systems

Complete solutions encompassing advanced sensors and payloads, processing systems, and analytics for global situational awareness, space superiority missions, and Earth insights

Electronic Systems

Electronic warfare, avionics, robotics, advanced communications and maritime systems for the defense industry, as well as air traffic management solutions for the civil aviation industry

Communication Systems

Tactical and airborne radios, night vision technology, and defense and public safety networks

slide-4
SLIDE 4

| 4 S8941 – Synthetic Label Data for Training Deep Learning ISR Algorithms NON-Export Controlled Information

“The GEOINT discipline has grown beyond the limits of human interpretation and

  • explanation. The explosion of available data

diminishes the comparative advantage collection

  • provides. Instead, automated processing,

advancing tradecraft, human-machine collaboration, and the ability to anticipate behaviors will provide us a new advantage.” Robert Cardillo, Director of NGA “We’re going to find ourselves in the not too distant future swimming in sensors and drowning in data”

  • Lt. Gen. David A Deptula,

USAF Dep Chief of Staff for ISR

2010 "The skies will ‘darken’ with the hundreds of small satellites to be launched by U.S. companies and as procedures are developed to allow safe

  • peration of unmanned aerial vehicles in civil

airspace,"

Robert Cardillo, Director – NGA

2015

“So just how big is this rising tide? If we were to attempt to manually exploit the commercial satellite imagery we expect to have over the next 20 years, we would need eight million imagery analysts. Even now, every day in just one combat theater with a single sensor, we collect the data equivalent of three NFL seasons – every game. In high definition! Imagine a coach trying to understand the strategy of his opponents by watching every play made by every team in every game for three seasons – all in one single

  • day. Because three more seasons will be coming tomorrow. That’s what we ask
  • ur analysts to do – when we don’t augment them with automation. But with all this

data – and dramatic improvements in computing power – we have a phenomenal

  • pportunity to do and achieve even more.”

Robert Cardillo, Director – NGA

2017

A call to action: the urgency behind the DoD’s adoption of AI

slide-5
SLIDE 5

| 5 S8941 – Synthetic Label Data for Training Deep Learning ISR Algorithms NON-Export Controlled Information

Graph from https://www.dsiac.org

2017 ImageNet Challenge Object Classification Winner: WMW, Momenta.ai 2.25% Error Rate 14M training images 1,000 object categories

A call to action: the urgency behind the DoD’s adoption of AI

slide-6
SLIDE 6

| 6 S8941 – Synthetic Label Data for Training Deep Learning ISR Algorithms NON-Export Controlled Information

A call to action: the urgency behind the DoD’s adoption of AI

slide-7
SLIDE 7

| 7 S8941 – Synthetic Label Data for Training Deep Learning ISR Algorithms NON-Export Controlled Information

Harris Corporation Introduction A Call to Action: The Urgency Behind the DoD’s Adoption of AI Review: Applications of Deep Learning at Harris Review: Harris’ Work to Scale Deep Learning Harris’ Approach for Handling the Label Data Burden Q&A

Agenda

slide-8
SLIDE 8

| 8 S8941 – Synthetic Label Data for Training Deep Learning ISR Algorithms NON-Export Controlled Information

Harris Corp. has been internally investing in taking state-of-the-art deep learning technologies and applying them to remote sensing and geospatial intelligence customer problems

How long?

  • Harris has been working on deep

learning for over five years How much?

  • Multimillion dollar internal research and

development investment in the last three years

  • Additional commercialization investment

Investment approach

  • Research
  • Pilot Projects
  • Software tool development
  • Commercialization

Focus Areas:

Reducing the cost of training

  • Reduce dollar cost, human cost, and

computer cost of building new models Extensibility

  • Ability to quickly redeploy and repackage

tools to support new problem sets Support multiple types of data

  • New sensors and data fusion

Automation

  • Ability for processes to interface to tools,

removing human from the loop

Harris deep learning research and development

slide-9
SLIDE 9

| 9 S8941 – Synthetic Label Data for Training Deep Learning ISR Algorithms NON-Export Controlled Information

Harris’ deep learning R&D and applications

slide-10
SLIDE 10

| 10 S8941 – Synthetic Label Data for Training Deep Learning ISR Algorithms NON-Export Controlled Information

Harris’ deep learning R&D and applications

slide-11
SLIDE 11

| 11 S8941 – Synthetic Label Data for Training Deep Learning ISR Algorithms NON-Export Controlled Information

Problem: Clandestine airfields in South American countries used for illegal narcotic trafficking Goal: Detect new airfields and determine activity levels with high temporal resolution data (Planet)

Harris deep learning R&D motion imagery applications – high revisit rate still imagery

slide-12
SLIDE 12

| 12 S8941 – Synthetic Label Data for Training Deep Learning ISR Algorithms NON-Export Controlled Information

Sources:

  • Planet imagery
  • DigitalGlobe EGD imagery
  • Ecuadorean geoportal shapefile of known remote landing strips
  • Google Earth imagery
  • Wikimapia
  • OpenStreetMap

Harris deep learning R&D motion imagery applications – high revisit rate still imagery

slide-13
SLIDE 13

| 13 S8941 – Synthetic Label Data for Training Deep Learning ISR Algorithms NON-Export Controlled Information

Applications: Lessons Learned and Takeaways

Dramatic performance improvement p(Wet) AUC = 98.1%

High accuracy achievable with appropriate NN architecture and labeled data Perform ATR with multiple data types

MSI PAN SAR LIDAR

AI enabled by powerful GPUs (minutes instead of days)

slide-14
SLIDE 14

| 14 S8941 – Synthetic Label Data for Training Deep Learning ISR Algorithms NON-Export Controlled Information

Harris Corporation Introduction A Call to Action: The Urgency Behind the DoD’s Adoption of AI Review: Applications of Deep Learning at Harris Review: Harris’ Work to Scale Deep Learning Harris’ Approach for Handling the Label Data Burden Q&A

Agenda

slide-15
SLIDE 15

| 15 S8941 – Synthetic Label Data for Training Deep Learning ISR Algorithms NON-Export Controlled Information

The art of scaling machine learning

Machine Learning Problem Answers! PhD Training Data Labels Hardware

Curation

Data ingest Sensor knowledge

Lots of hardware

Understand problem

Present solution Retrain model Refinement

Notifications Geographic Metadata Orthorectification Performance monitoring

Classification Export Controls Scheduling

Mobile Access

Prioritization

slide-16
SLIDE 16

| 16 S8941 – Synthetic Label Data for Training Deep Learning ISR Algorithms NON-Export Controlled Information

Breaking it down

Accessibility Infrastructure Learning All Source / Multi-INT Source Information Automation

slide-17
SLIDE 17

| 17 S8941 – Synthetic Label Data for Training Deep Learning ISR Algorithms NON-Export Controlled Information

Rinse and repeat

Cognitive Ecosystem Higher Order Sense Making [ Automated Activity Based Intelligence ]

slide-18
SLIDE 18

| 18 S8941 – Synthetic Label Data for Training Deep Learning ISR Algorithms NON-Export Controlled Information

Harris’ work to scale deep learning for defense source information

slide-19
SLIDE 19

| 19 S8941 – Synthetic Label Data for Training Deep Learning ISR Algorithms NON-Export Controlled Information

Harris’ work to scale deep learning for defense all source/multi-INT

slide-20
SLIDE 20

| 20 S8941 – Synthetic Label Data for Training Deep Learning ISR Algorithms NON-Export Controlled Information

Harris’ work to scale deep learning for defense learning

slide-21
SLIDE 21

| 21 S8941 – Synthetic Label Data for Training Deep Learning ISR Algorithms NON-Export Controlled Information

Harris’ work to scale deep learning for defense infrastructure

slide-22
SLIDE 22

| 22 S8941 – Synthetic Label Data for Training Deep Learning ISR Algorithms NON-Export Controlled Information

Harris’ work to scale deep learning for defense accessibility and automation

slide-23
SLIDE 23

| 23 S8941 – Synthetic Label Data for Training Deep Learning ISR Algorithms NON-Export Controlled Information

Harris’s work to scale deep learning for defense automation – higher order sense making

LIDAR

Live Observations Tally: Planes Detected: 03 Cars Detected: 22 Trucks Detected: 10 Ships Detected: 02 Storage Tanks Detected: 08 Buildings Detected: 05 Crosswalks Detected: 12 Intersections Detected: 15 Etc…..

We will go from drowning in data… In the not too distant future… …to drowning in detections/features/observations. Deep learning/machine learning/artificial intelligence attention will shift from detecting objects and features to detecting actions, events, patterns of life.

slide-24
SLIDE 24

| 24 S8941 – Synthetic Label Data for Training Deep Learning ISR Algorithms NON-Export Controlled Information

Harris Corporation Introduction A Call to Action: The Urgency Behind the DoD’s Adoption of AI Review: Applications of Deep Learning at Harris Review: Harris’ Work to Scale Deep Learning Harris’ Approach for Handling the Label Data Burden Q&A

Agenda

slide-25
SLIDE 25

| 25 S8941 – Synthetic Label Data for Training Deep Learning ISR Algorithms NON-Export Controlled Information

Analytic pipeline and label data paradox

Model Governance Model Application Model Refinement Manage Observations Higher Order Sense Making

Increased Volume and Usage Traditional Hand- Constructed Algorithms / Analytics Basic Computer Vision Algorithms / Analytics Supervised Deep Learning Algorithms / Analytics Unsupervised Deep Learning Algorithms / Analytics Supervised deep learning based algorithms represent the state

  • f the art and are ready for widespread adoption IF the label

data burden can be overcome Label data paradox for hard to find targets:

  • Analyst needs an algorithm to help locate hard to find target
  • Algorithm needs many examples of the target to be trained

Expert Intensive & Mediocre Accuracy Expert Intensive & Some Accuracy Improvement Less Expert Intensive & Large Accuracy Improvement, BUT Label Data Hungry Technology Not Mature Goal is Zero Label Data Still Data Hungry

slide-26
SLIDE 26

| 26 S8941 – Synthetic Label Data for Training Deep Learning ISR Algorithms NON-Export Controlled Information

Traditional approaches for handling the label data burden

Manual Harvesting of Label Data ( Individual or Crowdsourced )

Positives Negatives

slide-27
SLIDE 27

| 27 S8941 – Synthetic Label Data for Training Deep Learning ISR Algorithms NON-Export Controlled Information

Traditional approaches for handling the label data burden

Group Random Chips by Semantic Similarities

CURATE

slide-28
SLIDE 28

| 28 S8941 – Synthetic Label Data for Training Deep Learning ISR Algorithms NON-Export Controlled Information

Traditional approaches for handling the label data burden

Public data sets

  • Natural Imagery:

‒ Common Objects in Context (COCO)

http://cocodataset.org/

‒ Pattern Analysis, Statistical Modelling and Computational Learning Visual Object Classes (PASCAL VOC)

http://host.robots.ox.ac.uk/pascal/VOC/index.html

‒ ImageNet

http://www.image-net.org/

  • Overhead Imagery:

‒ Cars Overhead with Context (COWC)

https://gdo152.llnl.gov/cowc/

‒ SpaceNet

https://wwwtc.wpengine.com/spacenet

‒ xView

http://xviewdataset.org/

slide-29
SLIDE 29

| 29 S8941 – Synthetic Label Data for Training Deep Learning ISR Algorithms NON-Export Controlled Information

Grouping of Random Chips

Pros

Pros and cons of traditional label data approaches

Cons

Public Datasets Targeted Manual Labeling

  • Minimal upfront work to begin

training on classes

  • Starting point for transfer

learning

  • Generate large number of

coarsely labeled chips quickly

  • Staring point for transfer

learning

  • Label by label human-level

accuracy

  • ‘Scalable’ with crowdsourcing
  • Starting point for transfer

learning

  • Limited to datatypes, classes,

and conditions included in the dataset

  • Requires significant manual

curation after grouping

  • Limited to classes and

conditions present in the data

  • Time consuming
  • Limited to classes and

conditions present in the data

Method

slide-30
SLIDE 30

| 30 S8941 – Synthetic Label Data for Training Deep Learning ISR Algorithms NON-Export Controlled Information

Answer: “Good” Labeled Data

To Define what a ‘Good’ label dataset is, first define how the desired algorithm is expected to be used An example: the ubiquitous ‘Airplane Finder’

  • If the algorithm is only expected to be applied

to a very narrow distribution of images to make detections, a relatively narrow distribution of labeled training data is needed

  • HOWEVER, if the algorithm is expected to be

applied to a very wide distribution of images to make detections, a robust distribution of labeled training data is needed

A = brittle, B = brittle, C = robust = valuable Algorithm robustness is largely driven by training label data robustness

What makes a “good” deep learning algorithm?

slide-31
SLIDE 31

| 31 S8941 – Synthetic Label Data for Training Deep Learning ISR Algorithms NON-Export Controlled Information

When does algorithm robustness matter?

…When you don’t know what type of data or collection conditions you need to find the object in. When the object is public, plentiful, and routine a brittle algorithm may suffice. If the object is newly discovered, scarce, elusive or deceptive a more robust algorithm is needed.

KCNA/REUTERS/NEWSWEEK

slide-32
SLIDE 32

| 32 S8941 – Synthetic Label Data for Training Deep Learning ISR Algorithms NON-Export Controlled Information

Variations to consider – collection geometry

Angles These identify the angle at which the sensor is imaging the ground, as well as the angular location of the sun with respect to the ground and

  • image. These features can be added without preprocessing. The following

angles are provided: Off-nadir Angle Angle in degrees (0-90∘) between the point on the ground directly below the sensor and the center of the image swath. Target Azimuth Angle in degrees (0-360∘) of clockwise rotation off north to the image swath’s major axis. Sun Azimuth Angle in degrees (0-360∘) of clockwise rotation off north to the sun. Sun Elevation Angle in degrees (0-90∘) of elevation, measured from the horizontal, to the sun.

LAND INFO WORLDWIDE MAPPING, LLC

slide-33
SLIDE 33

| 33 S8941 – Synthetic Label Data for Training Deep Learning ISR Algorithms NON-Export Controlled Information

High off nadir image examples

Off nadir angle: 51° Off nadir angle: 32.7° Off nadir angle: 34.5° Off nadir angle: 61° Off nadir angle: 61°

Increased deep learning algorithm robustness requires exposure to a wide range of collection conditions

slide-34
SLIDE 34

| 34 S8941 – Synthetic Label Data for Training Deep Learning ISR Algorithms NON-Export Controlled Information

Variations to consider – many different sensor models

The democratization of space

  • Many new sensors flying – offering

much more persistent coverage

  • However, this results in many different

sensor models each with their own characteristics

  • To make deep learning algorithms

robust, they will need exposure to these varieties of sensor models

Increased deep learning algorithm robustness requires exposure to or ability to quickly adapt to multiple sensor models

IN-Q-TEL / COSMIQWORKS

slide-35
SLIDE 35

| 35 S8941 – Synthetic Label Data for Training Deep Learning ISR Algorithms NON-Export Controlled Information

Example of real sensor and collection geometry variation

50 100 150 200 250 300 350

0.0 2.5 5.0 7.5 10.0 12.5 15.0 17.5 20.0 22.5 25.0 27.5 30.0 More

Off Nadir Angle

AOI Off Nadir Angle

slide-36
SLIDE 36

| 36 S8941 – Synthetic Label Data for Training Deep Learning ISR Algorithms NON-Export Controlled Information

Increasing deep learning algorithm robustness Variations in ‘what’ is being imaged

Target Geometry Target Materials Background Conditions

Steel Aluminum Fiberglass Camouflaged Glass Painted Etc Car Model B Car Model C Parked on Incline Articulating Conditions Orientation Etc Car Model A Pavement Concrete Grass Dirt Stand Alone / Groups Desert Jungle

slide-37
SLIDE 37

| 37 S8941 – Synthetic Label Data for Training Deep Learning ISR Algorithms NON-Export Controlled Information

Increasing deep learning algorithm robustness Variations in ‘how’ the target is being imaged

SmallSat Next

Sensor Model

Gov’t Sensor A Gov’t Sensor B Gov’t Sensor C

Off Nadir Angle

50° 40° 30° 20° 10° 0° 10° 20° 30° 40° 50°

Target Azimuth

0° 36° 72° 108° 144° 180° 216° 252° 288° 324° 360°

Sun Elevation

27.7° 39.6° 51.5° 63° 74.4°

Sun Azimuth

45° 90° 135° 180° 225° 270° 315° Noon Dusk Dawn June December Mar Sep

slide-38
SLIDE 38

| 38 S8941 – Synthetic Label Data for Training Deep Learning ISR Algorithms NON-Export Controlled Information

Why is so much label data needed?

Training Data Variation Variation in Data to be Analyzed

Target Variation Background Variation Collection Variation Sensor Variation Target Variation Background Variation Collection Variation Sensor Variation Target Variation Background Variation Collection Variation Sensor Variation Target Variation Background Variation Collection Variation Sensor Variation

Single / Very Few Labeled Data Larger Collection of Manual / Crowd Sourced Labeled Data Variation in Training Data = Variation in Data to be Analyzed

Target Variation Background Variation Collection Variation Sensor Variation Target Variation Background Variation Collection Variation Sensor Variation

Brittle Algorithm Performance Window Broader Algorithm Performance Window Robust Algorithm Performance Window

slide-39
SLIDE 39

| 39 S8941 – Synthetic Label Data for Training Deep Learning ISR Algorithms NON-Export Controlled Information

Prior to launch of a space based imaging systems, Harris generates imagery that simulates what the sensor will produce when in operations

A new approach to label data

Harris has decades long legacy providing high fidelity, physics-based, radiometrically correct remote sensing modelling and simulation services

slide-40
SLIDE 40

| 40 S8941 – Synthetic Label Data for Training Deep Learning ISR Algorithms NON-Export Controlled Information

DIRSIG

10 am, 7 degree look angle, Jan 1, Scene Azimuth 0 2 pm, 7 degree look angle, Jan 1, Scene Azimuth 225

Harris’ work to scale deep learning for defense source information – synthetic label data generation

  • 100% of training data synthesized using CAD models and Scene Simulator
  • The trained model is applied to real imagery
  • Successful detector produced for fighter jets in WV-2 Pan imagery
  • Limiting factors: (1) content of scene generator and (2) quality of simulation

6 CAD models used Objects placed in scene at various geometries Heat Map for fighter jets in IKONOS Pan Imagery

slide-41
SLIDE 41

| 41 S8941 – Synthetic Label Data for Training Deep Learning ISR Algorithms NON-Export Controlled Information

Automated synthetic labeled training data vision

Scene Modeling Collection Modeling

Automated Data Generation Workflow

Order of Battle

  • Air
  • Ground
  • Naval
  • Urban

Background Materials

  • Concrete
  • Asphalt
  • Crushed Stone
  • Dirt
  • Vegetation
  • Metal
  • Plastic
  • Glass
  • Sand

Target Classes

  • Planes
  • Vehicles
  • Ships
  • People
  • Facilities

Target Types

  • Commercial
  • Consumer
  • Military

Target Configurations

  • Open / Obscured
  • Orientation

Scenarios

  • Formations
  • Specific Routes
  • Dynamics

Atmosphere

  • Tropical
  • Desert
  • Clouds
  • Sun Conditions

Platform / Sensor Type

  • Array Size
  • Bandpass
  • Sampling
  • Scan Type

Platform Motion Scene Location Truth Generation

Sensor Modeling

Noise MTF

  • Optics
  • Detector

Exposure

  • Integration Time

Sensor Artifacts

  • Failed Detectors
  • Non-Uniformity

Ground Processing

  • DRA
  • Sharpening
  • Registration Effects
slide-42
SLIDE 42

| 42 S8941 – Synthetic Label Data for Training Deep Learning ISR Algorithms NON-Export Controlled Information

Pipeline for rapid build of new deep learning algorithms

1 – Specify a target of interest

  • Select from database of target models

2 – Generate synthetic labeled training data

  • Scene generation
  • Automatic labeling
  • Augmentation
  • Training data delivery

3 – Manage source and training data 4 – Train DL model 5 – Model governance 6 – Apply model to source imagery 7 – Refine model with feedback loop 8 – Manage Observations 9 – Interoperable for higher order sense making Automatically Generated Negatives Synthesized Positives Original Target Model Synthesized Satellite Imagery Augmentation MEGA Content Management Machine Learning / Training Algorithm Governance / Workflow Orchestration GIS Interface / Machine Learning Refinement / Observation Management

1 2 2 2 3 4 5 6 7 8 9

slide-43
SLIDE 43

| 43 S8941 – Synthetic Label Data for Training Deep Learning ISR Algorithms NON-Export Controlled Information

Pipeline for rapid build of new deep learning algorithms

Select Target

  • f Interest

Synthesize Training Data Manage Data Train DL Algorithm Model Governance Apply Model Refine Model Manage Observations Higher Order Sense Making

1 2 3 4 5 6 7 8 9

Hydra MEGA

  • Demonstration

starts with a CAD model of the target

  • f interest
  • For unclassified

demos an unclass target will be selected (transfer trucks)

  • For classified

demos a classified target will be selected

  • RIT DIRSIG
  • Harris LYNX
  • Scene generation
  • Object insertion
  • Augmentation
  • Output physicals

based synthetic training images

  • Label data from

movers

  • System that

ingests and manages all the training data in a method in which DL algorithms can access

  • Positives
  • Negatives
  • Hard Positives
  • Hard Negatives
  • MEGA services
  • n backend
  • GSF web

interface to execute training?

  • Training results

presented

  • Time to train

presented (highlight GPU impact)

  • Load newly

trained model into algorithm marketplace and registered with algorithm governance

  • Highlight multiple

algorithms registered, Harris made as well as 3rd party

  • Using

Hydra/DAGR imagery is passed to the model for detections to be made

  • Using DAGR

demo the ability to evaluate true/false positives, and true/false negatives

  • Understanding

information from movement

  • Observations

managed by Hydra / DAGR

  • Initial funding of

this demonstration will ensure the appropriate ‘hooks’ are in place for integration with higher order sense- making applications such as LUX

  • Activity pattern

recognition based

  • n movement alone
  • Correlation of PIA

info

  • Correlation of
  • ther INTs (SIGINT

LYNX DIRSIG

slide-44
SLIDE 44

| 44 S8941 – Synthetic Label Data for Training Deep Learning ISR Algorithms NON-Export Controlled Information

Test scenario for synthetic pipeline buildout

slide-45
SLIDE 45

| 45 S8941 – Synthetic Label Data for Training Deep Learning ISR Algorithms NON-Export Controlled Information

Example Multipliers

81 x 21 x 33 x 24 x 01 x 01 x 01 = 1,347,192 02 x 13 x 33 x 8 x 01 x 01 x 01 = 6,864 01 x 01 x 33 x 24 x 01 x 01 x 01 = 792

Variation multiplication

Types of Variation

  • 1 - Physical combinations (cab shape, trailer

shape, cab + trailer combination )

  • 2 - Material combinations (aluminum, steal,

glass, fiberglass, etc)

  • 3 - Background – placement of above

combinations in different locations in a scene

  • 4 - Collection Geometry (Altitude, TEA,

Azimuth look angle )

  • 5 - Lighting Conditions ( sun angle, morning,

noon, evening )

  • 6 - Atmospheric conditions (haze, etc)
  • 7 - Sensor Model (MTF, noise, etc per specific

sensor, WorldView, CorvusEye, etc)

Initial Test Pipeline Range

  • f Variations
  • 1 – Varied: 81 combinations
  • 2 – Varied: 21 combinations
  • 3 – Fixed: Single Background with

33 parking spots (infinite possible)

  • 4 – Varied: 1 Altitudes x 3 TEAs x 8

azimuth (GSD driven, 0*-90°, 0-360°)

  • 5 – Fixed (many possible)
  • 6 – Fixed (many possible)
  • 7 – Fixed: CorvusEye (many

possible)

slide-46
SLIDE 46

| 46 S8941 – Synthetic Label Data for Training Deep Learning ISR Algorithms NON-Export Controlled Information

  • Test model trained on synthetic training

data applied to synthetic scene

  • Targets identified as expected

Synthetic data trained model applied to synthetic data

slide-47
SLIDE 47

| 47 S8941 – Synthetic Label Data for Training Deep Learning ISR Algorithms NON-Export Controlled Information

Sensor model characterization

slide-48
SLIDE 48

| 48 S8941 – Synthetic Label Data for Training Deep Learning ISR Algorithms NON-Export Controlled Information

Comparison of synthetic vs real

slide-49
SLIDE 49

| 49 S8941 – Synthetic Label Data for Training Deep Learning ISR Algorithms NON-Export Controlled Information

Control algorithm for comparison to aide in honing synthetic

Algorithm Trained on Synthetic Data Algorithm Trained on Real Data

slide-50
SLIDE 50

| 50 S8941 – Synthetic Label Data for Training Deep Learning ISR Algorithms NON-Export Controlled Information

Conclusions/takeaway

Modeling and simulation pipeline provides ability to rapidly generate new training images covering a wide distribution of target geometries,

  • rientations, material composition, background, collection geometries,

lighting conditions, and sensor models Deep learning models trained on synthetic data can provide an initial starting point for locating real targets in real imagery Rapid generation of fully characterized target labels allows for full deep learning model characterization to understand which variations have most impact on model performance Next Steps:

  • Full characterization of label data variation
  • Performance Evaluation of Synthetic Data Trained Models to Real Data

Trained Models

slide-51
SLIDE 51

| 51 S8941 – Synthetic Label Data for Training Deep Learning ISR Algorithms NON-Export Controlled Information

Questions?

Look for upcoming blog posts about machine learning at Harris’ Blog website: http://www.harrisgeospatial.com/Company/PressRoom/Blogs.aspx

Trademarks are registered marks of their respective companies.

Will Rorrer, PMP

Machine Learning Business Development

wrorrer@harris.com

George Brown

Machine Learning Engineering Manager

gbrown12@harris.com

Ron Kneusel, PHD

Senior Data Scientist

rkneusel@harris.com

David Gorodetzky, PHD

Research Lead Machine Learning and Remote Sensing

dgorodet@harris.com