Simplif lifyin ing A AI for or C Com ommunic icatio ions, - - PowerPoint PPT Presentation

John D

• D. Fergu

guson info@deep eepwaved edigi gital.com

Simplif lifyin ing A AI for

• r C

Com

• mmunic

icatio ions, Radar, a , and W Wirele less Systems

Deep L Learn rning a and R Radio F Frequency (RF) ) Systems

2

Deep eep L Learning i g is Emer erging

• Intrusion
• n D

Detection

• n
• Threat

at c clas assi sificat ation

• Facia

ial r recognitio ition

• Imag

agery a anal alysis

• Tumor
• r D

Detection

• n
• Medical

al d data a a anal alysis

• Diag

agnosis

• Drug d

dis iscovery

• Pedestrian /

/ obst stac acle detection

• n
• Navig

igatio tion

• Street s

sign r read ading

• Speech r

recognitio ition

• Image c

cla lassif ificatio tion

• Speech r

recognitio ition

• Languag

age t translat ation

• Document /

/ d datab abas ase sear arching

Cyber Medicine Auton

• nom
• my

Inte ternet

Deep l p learni ning ng techno nology e enabl bled a ed and d accel eler erated ed b by GP GPU U pr processo essors s

• Has yet t

to impa pact de design a and nd appl pplicatio ions in wir ireless and r nd radio dio frequ quenc ncy s systems

Gamma ma

X-Ray ay UV UV Visib ible Infr frared Radio F Frequencies

1 p pm 10 p pm 10 n nm 400 n nm 700 n nm 1 mm 1 mm 100 k km

Radar ar Satellite lite Communicat ations Electronic W Warfare Tele lecommunic icatio tions Milita litary Communicat ations Navig igatio tion UAV Wi Wireless C Control Wireless N Networki king Internet o

• f Th

Things RF A Abla latio tion (Medical) l)

Radio F Freq equen ency T Technology i is Pervasive

Enabl bled b d by low-cost, h hig ighly capable le g gen eneral l pur urpose g se graphi hics processi essing u uni nits ( s (GPUs) s)

• Spectrum

um m moni nitoring ng ( (thr hreats)

• Intel

elligen ent s spec ectrum usage

• Electronic p

protection ( (anti-jam) m)

• Cognitive s

e system em c control

Spectru rum / / Ne Network rk Centric A Appl pplications

• Advanced m

modulati tion t techniques

• Adapti

tive wa waveforms

• Encrypti

tion a and s security ty

Devi evice / e / Basesta tation Centric A Appl pplications

• Voic
• ice /

/ imag age r recognit itio ion

• Multi

ti-sensor

• r f

fusion ion

• Decision m

making g and d data ta r reduction

User A r App Centric A Appl pplications

Where to

• Use

se Deep eep Lea earnin ing in RF RF Systems

3

Mo Modu dula late

1 0 1 1 0 0 1

User A r Apps Transm smit Frequ equenc ncy Conv nvert User A r Apps Recei eive e Frequ equenc ncy Conv nvert

1 0 1 1 0 0 1

Demodula ulate

Deep L Learn rning C Compari rison

• Multiple channels (RGB)
• x, y spatial dependence
• Temporal dependence (video)
• Single channel
• Frequency, phase, amplitude
• Temporal dependence
• Multiple channels
• Frequency, phase, amplitude
• Temporal dependence
• Complex data (I/Q)
• Large Bandwidths
• Human engineered

4

Image and Vi Video Audio a and L Language Systems a and S Sign gnals

Existing ng d deep l learni ning p poten entially a ada daptabl ble t to systems s and s nd signa nals

• Must

t contend wi with wi wideb eband s sign gnals and complex d data t types

Hardware for D Deep L Learning i in RF Systems

5

Training Infer eren ence

Pro ros Cons Pro ros Cons

CPU

• Supported by ML Frameworks
• Lower power consumption
• Slower than GPU
• Fewer software

architectures

• Adaptable architecture
• Software programmable
• Medium latency
• Low parallelism
• Limited real-time bandwidth
• Medium power requirements

GPU

• Supported by ML Frameworks
• Widely utilized
• Highly parallel / adaptable
• Good throughput vs power
• Overall power

consumption

• Requires highly parallel

algorithms

• Adaptable architecture
• High real-time bandwidth
• Software programmable
• Medium power requirements
• Not well integrated into RF
• Higher latency

FPGA

Not widely utilized, not well suited (yet)

• High power efficiency
• High real-time bandwidth
• Low latency
• Long development / upgrades
• Limited reprogrammability
• Requires special expertise

ASIC

Not widely utilized, not well suited

• Extremely power efficient
• High real-time bandwidth
• Highly reliable
• Low latency
• Extremely expensive
• Long development time
• No reprogrammability
• Requires special expertise

Critical al Performan ance P Parameters f for D Deep Lear arning i in RF Systems

6

Ada daptabilit ity / / Upg pgradabil bilit ity Deplo ployment Time me Lifecy cycle cle C Cost Real al Ti Time Band ndwidt idth Compute / e / Watt Latenc ncy CPU PU GPU PU FPG FPGA ASIC GPU PU s signa nal pr l processing ng can pr n provide w wide deba band d capa pability and nd software upg upgradabil ilit ity a at lower cost and de nd develo lopment t time

• Mus

ust conten end w with i inc ncrea eased ed l laten ency ( (~2 m 2 microsec second nd)

Outl tline

• Introduction to Deep Learning in RF
• Deepwave’s Technology
• Signal Detection and Classification
• Real-time Benchmarks on Embedded GPUs
• Summary

Why Has Deep Learn rning i in RF N Not B Been Ad Addressed

• AI requires large data sets
• Insufficient bandwidth to

send to remote data center

• No RF systems exist with

integrated AI computational processors

• Disjointed software
• Difficult to program and

understand

8

Bandwidth Li Limitati tions Limite ted C Compute te Resources Complica cated Softw tware

remote processing not possible at field site for RF and AI independently

Deepwave’s Software Defined R Radio

A Pl Platform f for a a Mul ultit itude of A Appl pplications ns

The P Platform

• rm

Complete Edge-compute AI Platform for RF

The S he Soft ftware

Simply build AI into wireless technology

3rd Party Software Artific icia ial I l Intell llig igence R Radio T Transceiv iver*

*Patent P Pendi ding ng AIR-T Hardware AI Frameworks Signal Processing Hardware Abstraction WAVELEARNER Software GPU Parallel Computing Deep / Machine Learning

Arti tifi fici cial Intelligence Radio T Transceiver (AIR-T) T)

• 2x2 MIMO T

Transcei eiver er

• Analog Devices 9371 chip
• Tunable from 300 MHz to 6 GHz
• 100 MHz bandwidth per channel
• Digital S

Signa nal / l / Deep L Learning ning Processors

• Xilinx Artix 7 FPGA
• NVIDIA Jetson TX2
• ARM Cortex-A57 (quad-core)
• Denver2 (dual core)
• Nvidia Pascal 256 Core GPU
• Shared GPU/CPU memory
• Conne

nectiv ivity

• 1 PPS / 10 MHz for GPS Synchronization
• External LO input
• HDMI, USB 2.0/3.0, SATA, Ethernet, SD Card,

GPIO

AI AIR-T Ha Hardware S e Spec pecifications

Mini I i ITX F X For

• rm F

Fac actor

13

Arti tifi fici cial Intelligence Radio T Transceiver (AIR-T) T)

Block D k Diag agram am

RF Transceiver Analog Devices 9371 NVIDIA Jetson GPU NVIDIA Parker (256 Core) CPU Arm A57 (4 Core) GPU/CPU Shared Memory 1 G GigE gE USB 3 B 3.0 GPI PIO HDMI MI SAT ATA

JESD SD PCIe Ie

Clock FPGA Xilinx Artix 7 GPI PIO REF EF TIME

Incorp

• rpor
• ration of
• f GPU

PU i in RF system a allows f for

• r wideband

pr processi essing o

• f signal da

data i in n software en environment

• Reduces

es dev evel elopmen ent t time e and c cost

2x2 M MIMO

Simplified d Programming

Deep eep Learning Dig igit ital l Signal al Proces essing

VHDL, Verilog

• r
• r

Cust stom S Software

• r
• r

Te TensorRT

14

FFT T Perfor

• rmance

e Testing

13

FPGA GA Sha hared ed M Mem emory cuFFT uFFT PCI CIe NVIDA DA Te Tegra TX2 X2 Compl plex int16 t 16 to complex f float32 32 Sha hared ed M Mem emory

Intel 7500U

Deploy A

• y Application
• n

TensorRT

Optimi mizer Runtime me

Optimize N e Neural N Net etwork

Infer eren ence e at the E Edge e with GR GR-Wavelearner

14

Train N Neural N Net etwork

GR GR-Wavel elea earner er Software

15

• Goal i

l is to he help lp the he o

• pe

pen s source c communit ity e easily ly de deplo loy deep l learning wi ng within s signal processing a g applications

• Wel

ell docu cumented R README w with th d dep ependency cy i installati tion instruction

• ns t

to g get et started ed q quick ckly

• Ubuntu 16.04 recommended, Windows 10 supported
• NVIDIA Docker Container 18.08*
• Signal

al c clas assifier er e exam ample p e provided ed:

• GNU Radio Flowgraph
• Python source code
• PLAN files that are executable on the AIR-T and Maxwell
• Signal data file example for testing
• Support f

for Te TensorRT 5.0

• Avai

ailab able a e at: d deep epwaved edigital al.com

• m/wavel

elear earner er

https://docs.nvidia.com/deeplearning/sdk/tensorrt-container-release-notes/rel_18.08.html

GN GNU R U Radio – Sof

• ftware D

Defin ined R Radio io ( (SDR) F ) Framework

• Popular o
• pen s

source s software d defined radio io ( (SDR) t ) toolkit it:

• RF Hardware optional
• Can run full software simulations
• Python A

API PI

• C++ under the hood
• Easily

ly c create D DSP a algorit ithms

• Custom user blocks
• Pri

rimarily ly us uses C CPU

• Advanced parallel instructions
• Recent development: RFNoC for FPGA

processing

• De

Deepwave i is integr grating G g GPU s support f for bot

• th D

DSP a and M ML

16

GR GR-Wavel elea earner er

• Out

ut o

• f t

tree (OOT) m modu dule le for GNU Ra Radi dio

• Allows u

user ers t to ea easily inc ncorporate de deep p learnin ning i into signa nal pr l processing

• C++ a

and nd Python A n API PI

• Ope

pen s n source G GPL PLv3 license

• Two bl

blocks c cur urrently ly:

• Inference – TensorRT wrapper fo

for GN GNU R Radio io

• Terminal S

Sink – Python module f for displa layin ing c classif ifie ier o

• utput

17

Outl tline

• Introduction to Deep Learning in RF
• Deepwave’s Technology
• Signal Detection and Classification
• Real-time Benchmarks on Embedded GPUs
• Summary

Mu Multi-transmit itter er Envir ironmental Sc l Scen enario io

19

AI AIR-T m moni nitors s congested spectrum um u using ng deep l p learni ning ng

Radar r Signal Detector r Model: Transmitted Signals

Radar W r Waveform rm Linea near P Pul ulse se

X X X X X X

No Non-Linea ear P Pul ulse

X X X

Phase C se Coded ed P Pul ulse se

X

Pul ulsed sed D Dopp ppler er

X X X

20

Techni hnique d que demons nstration n shown w n with n h nomina nal r rada dar s signa nals

• Met

ethod a applicable to e to c communicati tions, c cellular, a and o

• ther

er R RF protocols

Data taset Ove verview

• Goal: Develop a deep learning classifier

that detects signals below noise floor

• Requires training on noisy data with and

without interference

• Swept SNIR from -35 dB to 20 dB in 1 dB

increments

• 1000 training segments per SNIR
• 500 inference segments per SNIR
• Up to 3 interferers in each segment

21

Radar r Signal Detector r Model: Ex Example Classifier

22

Signal Feature Extraction Signal Classification Max Pool Flatten Signal al S Stream am Convolution Q I

• Q

I Q I Q I

Training Proc

• ces

ess and Progr

• gres

ess

• 1000 training segments per SNR
• 55 different SNR values
• Training on low SNR values increase

detection sensitivity

• 100% accuracy not expected due to

training at extremely low SNR values

• Softmax cross entropy
• Adam Optimizer

23

Deep ep Learning C Classi ssifier er T Training

Detecting and Classifying Low Power r Signals

24

Signal to Noise Ratio (dB)

Probably of Correct Classification

Neural n l networ

• rk

k sta tarti ting t g to c classify signal well ell b belo elow noi

• ise f

floor

• or

Detecting and Classifying Low Power r Signals

25

Signal to Noise Ratio (dB)

Probably of Correct Classification

Near 1 100% 0% classi ssification proba babi bility w with h SNR > > -10dB 0dB

Receiver r Operating Characteri ristic ( (ROC) C Curve

26

Signa nal-to- Noise R se Ratio (d (dB) Recei eiver er N Noise e Power er ( (milliwatts) s) Recei eived ed S Signal Power er ( (milliwatts) s) 20 1 100 10 1 10 1 1

• 10

1 0.1

• 20

1 0.01

• 30

1 0.001

Dec ecibel el ( (dB) R Refres esher er Prob

• bab

abili lity of Correct C Classif ific ication ion f for On One Radars rs

Receiver r Operating Characteri ristic ( (ROC) C Curve

27

Signa nal-to- Noise R se Ratio (d (dB) Recei eiver er N Noise e Power er ( (milliwatts) s) Recei eived ed S Signal Power er ( (milliwatts) s) 20 1 100 10 1 10 1 1

• 10

1 0.1

• 20

1 0.01

• 30

1 0.001

Dec ecibel el ( (dB) R Refres esher er

Why is PCC not zero?

Prob

• bab

abili lity of Correct C Classif ific ication ion f for All R ll Rad adars

Receiver r Operating Characteri ristic ( (ROC) C Curve

28

Conf nfus usion M n Matrix (-35 35 dB S SNR)

Surveillance Ground (LFM1) Ground (LFM2) MTI Airborne (Med PRF) Airborne (High PRF) Ground (Frank Code) Nautical (Short Range) Nautical (Long Range) Nautical (Long Range) Ground (NLFM1) Ground (NLFM2) Ground (NLFM3) Interference Nothing

Tr Truth

0. 0.20 20 0. 0.16 16 0. 0.12 12 0. 0.08 08 0. 0.04 04 0. 0.00 00

Predictio ion

DNN a appea ppears s to be r rand ndomly g guessi essing ng at l low S SNR w which w will crea eate u e unn nnec ecessa essary r requ quirem emen ents o s on downstrea eam p processi essing

Why is PCC not zero?

Prob

• bab

abili lity of Correct C Classif ific ication ion f for All R ll Rad adars

Meth thodology f for T Testi ting False Positi tive R Rate

29

False A se Alarms ( (Noise S se Sour urce O e Onl nly)

Total PFA = 0.41

Noise S se Sour urce Classi sifier er Someth thing o g or Nothi hing ng? CNN NN

Something

False A se Alarm (False se Pos

• sit

itiv ive)

Confu fusion Matrix a x and Signal to Noise Ratio

30

Sig ignificant fals alse ala alarm rate lim limit its alg algorithm’s ap applic icabili ility an and c creates n non

• n-zero

prob

• bab

abili lity o

• f cor
• rrect c

clas assif ific ication ion ( (PCC) a at low w SNR v values es

Deep eepwave Train inin ing M Meth thod t to

• Red

educe ce False A e Alarms

31

Proba bability o

• f Correc

ect C Classi ssification for V Various s Rada dars False A se Alarms ( (Noise O se Onl nly)

Total PFA = 10-4

• Method makes probability of false alarm

a training hyperparameter

• Example shows false alarm rate reduced

from 41% to 0.01%

Outl tline

• Introduction to Deep Learning in RF
• Deepwave’s Technology
• Signal Detection and Classification
• Real-time Benchmarks on Embedded GPUs
• Summary

Critical Perf rform rmance Parameters

• What makes a DNN model “good?”
• Hi

High gh S Sen ensiti tivity ty – detects low powered signals

• Low f

fals lse alar alarm r rate – minimize false positives

• High r

real t time e bandwi width th

• Low c

w computational r requirem emen ents

• Low l

w laten tency

• Most of these critical performance

parameters are adversarial

33

Max Pool Flatten

Signal al S Stream am

Convolution Q I

• Q

I Q I

Perf rform rmance Benchmark rking T Test S Setup

34

Def efine M e Model del Struc uctur ure Min Va Val Ma Max V x Val To Total CNN Stride 1 16 9 Number of Filters 4 256 7 Classifier Layer 1 Width 64 128 3 Classifier Layer 2 Width 32 64 3 Classifier Layer 3 Width 64 2 Batch Size 1 256 8 Total Model Combinations Tested 728 Model del T Tun uning V Variabl bles es Repea epeat for m mul ultiple m e model dels

Perf rform rmance Benchmark rking T Test S Setup

• 1000 training segments per SNR
• 55 different SNR values
• Softmax cross entropy
• Adam Optimizer
• Quadro GP100 GPU
• Create UFF File for each model

35

Tr Train M Model

Perf rform rmance Benchmark rking T Test S Setup

• Compute receiver operating

characteristic (ROC) curve for each model

• Define sensitivity to be where

median PCC = 50% for all signal types

36

Measu sure e Sensi nsitivi vity Sensi nsitivi vity

Perf rform rmance Benchmark rking T Test S Setup

• Crea

eate e TensorR

• rRT PL

PLAN f file for

• r

ea each pl platform t tes ested ed

• Load s

d signa nal d data into R RAM

• Stream u

unthrottled d data t to gr- wavel elea earne ner

• Measure d

data r rate a at t two locations:

• 1. Aggr

greg egate d e data r rate f e for e entire p e proces ess

• Nu

Number o

• f bytes

es p processed / / wall t time

• 2. Computati

tion d data r rate i e in work() f ) functi tion

• Num

umber o

• f bytes pr

process / compu putatio ion t n time

37

Measu sure R e Real Time T Throug ughput hput

Data Rate Benchmark rk for r AI AIR-T T (Te Tegra TX2) 2)

38

• Tested 9

91 differ erent CNN NN classifier er mode dels

• Maximu

mum r real-time i e infer eren ence e data r rate for 8 different b batch h sizes es

• Abl

ble to achi hieve 2 200 MSPS PS ( (real l samples) with A AIR IR-T

AI AIR-T

Data Rate Benchmark f for Desktop (Quadro P100)

39

• Tested 9

91 differ erent CNN NN classifier er mode dels

• Maximu

mum r real-time i e infer eren ence e data r rate for 8 different b batch h sizes es

• Using u

unif ified m memory will ll increase t throug ughpu hput

Desktop ( (GP100) 100)

Wall Time v

• vs. Compute T

Time f for r AI AIR-T

40

Wa Wall Time me Comput pute Time me

Real t l tim ime data rate lim limit ited by y GNU R Rad adio over erhea ead

Mode del Accuracy Benchmarks

41

Deepwave e Infer eren ence D e Display

42

Summa mmary

• Deep learning within signal processing is emerging
• Algorithms may be applied to signal’s data content or signal

itself

• High bandwidth requirements driving edge solutions
• Deepwave developed AIR-T
• Edge-compute inference engine with MIMO transceiver
• FPGA, CPU, GPU
• GR-Wavelearner
• Open source inference engine for signal processing
• Available now on our GitHub page
• Benchmarking analysis demonstrates AIR-T with GR-Wavelearner capable of signal

classification inference at 200 MSPS real-time data rates

• Improvements likely in future release

43

More info at www.deepwavedigital.com/sdr

info@deepwavedigital al.com