Hardware & Software Platform for Next Generation Industrial - - PowerPoint PPT Presentation

Hardware & Software Platform for Next Generation Industrial Drones

Chetak Kandaswamy Kai Yan Helmut Prendinger

Technical topics:

• Advanced controller

(No GPS, complexed obstacles)

• Long endurance and Self-Diagnosis
• Vision based sensing

What’s next in industry drones?

Market:

• Infrastructure inspection
• Agriculture
• Disaster Observation
• Search & Rescue

• Under-bridge inspection, No GPS.
• Using external camera to

automatically detect and maintain the position of copter.

• Small LiDARs on-board for

secondary collision avoidance.

• Wired power supply
• Developed by enRoute Co., Ltd.

In-use (Feb., 2016)

Infrastructure inspection

Long range observation

• Achieved 67 minutes hovering,

with a single 450Wh battery

• Aero-efficient frame made by

TORAYCA T-800S (The same on Boeing-787 Dreamliner)

• Self-Diagnosis battery pack,

warning ahead of failure. developed by Hitach Maxell, Ltd.

• Developed by enRoute Co., Ltd.

In-production (May., 2016)

Vision based sensing

• Depth sensing with a single Camera
• Surrounding sensing with four cameras for 360 degree collision avoidance.
• Enabled by Jetson TX1 (Implemented in CUDA)
• Developed by LabRomanec Inc., Developer’s kit available soon (www.labromance.com)

Hobbyist and scientific research

• Aerial video capturing
• Journalism event capture
• Cricket/Football/wedding
• Track property
• Dangerous place
• Extreme sports

Surveillance

• Aerial reconnaissance
• Track endangered

animals/poachers

• Track solar panels
• Agricultural farms
• Railway lines
• Defense against other

drones Rescue Missions

• High range, good

cameras

• Good samaritans taking

care of elderly

• Dengue epidemic

Delivery Drones

• Vaccines to remote

locations

• Courier in crowded area
• Pizzas/dry cleaning
• Delivery in dangerous

places

Drones as service

Deep Transfer Learning Lightweight Drones Security

Object Recognition: ImageNet

Pixel-wise label: PASCAL VOC

FCN-8

• Transform Fully connected layers into

Convolutional layers

• Instead of classes, get a heatmap at the output
• Learnable upsampling to bring output to initial size
• Refine the output using different layer’s predictions

○ Shallow layers : fine scale ○ Deep layers : coarse scale

An example : FCN-8s

Deep Transfer Learning (DTL)

• DTL emerged as a new paradigm in machine learning in which, a machine is trained using

deep models on a source problem, and then transfer learning to solve a target problem.

• DTL is an alternative to transfer learning with shallow architectures, in which one specifies a

model to several hidden levels of non-linear operations and then estimates the parameters via the likelihood principle. Why DTL? Utilizes the high-level features using Deep Models. Utilizes Transfer Learning method for limited labeled data problems. Overcomes traditional Transfer Learning methods negative feature transfer causing

• ptimization to fall into bad solution space.

DTL method 1: Layerwise Transfer Learning Application: Drug-discovery

Cancerous cells

• f Breast

Task : Classification of chemical mechanisms of action (MOA) by identifying substances that alter the phenotype of a cell which prevent tumor growth and metastasis. Classify : Host cell or Tumor cell

Challenge : Every day thousands of drugs are tested on millions of samples. Each sample has ~5000 cells leading to billion of cells to check. Capturing the images for analysis takes 6 months at a time. Costing 10,000 Euros for each trail. Result of DTL: Transference of weights of the source model obtained positive transference and we observe around 30% computation speed up and improvement in overall efficiency.

Examples of different MOA captured after compound incubation of Breast Cancer cells.

Unlabeled Images

Drone Data

Crowd sourcing Unlabeled Images Labeled Images

Other Data Feature search space

Aerial images: Google map Satellite Non-aerial images: Imagenet Deep transfer learning

Validating the model with the drone data

Implementing existing deep transfer learning methods on Caffee Fine-tuning Layerwise Source-Target-Source Ensemble Multi source

DTL Method 1 for Drones

DTL method 2: Multi-Source STS: Cross-sensor Biometrics Recognition

Example: In case of periocular images captured rom multiple devices may have different resolution, size, Illumination setting, etc. Practical problems of cross-sensor biometrics is that these data is collected from various devices and

• ften we need to train machine separately for

different machines. (Intro-compatibility issues)

Result of Multi-Source STS

Result : DTL performed ~10 % better than the Deep Learning model.

Unlabeled Images

Drone Data

Crowd sourcing Unlabeled Images Labeled Images

Other Data Feature search space

Aerial images: Google map Satellite Non-aerial images: Imagenet Deep transfer learning

Validating the model with the drone data

Implementing existing deep transfer learning methods on Caffee Fine-tuning Layerwise Source-Target-Source Ensemble Multi source

DTL Method 1 and 2 for Drones

Input data variations: (Multiple sources) Angles - 45 degree or 90 degree Altitude - High or Low Resolution - High or Low

Training methods

FCN-8 SegNet (New model)

Training methods

VGGNet GoogLeNet ResNet Inception-v4

Deep Learning ImageNet + Fine- tuning on drone dataset Source-Target- Source Multi-source Ensemble (Multi-source ensemble) (New method)

Object detection Segmentation

Initial members:

Prendinger Lab.

• Deep learning for Drones
• Jetson TX1 based flight controller
• Silver/Bronze cloud based mission controller
• Deep Drone Dataset (D3)

○ Online Annotation Tool

Check out for updates on http://www.deepdrone.net/