Machine Learning on Blue Waters Using TensorFlow with the Image - - PowerPoint PPT Presentation

▶
machine learning on blue waters using tensorflow with the
SMART_READER_LITE
LIVE PREVIEW

Machine Learning on Blue Waters Using TensorFlow with the Image - - PowerPoint PPT Presentation

Oct 19, 2022 305 likes •538 views

2/8/18 Machine Learning on Blue Waters Using TensorFlow with the Image Feature Detection Problem Or: How I Learned to Stop Worrying And Love AI Presented By: Dr. Aaron D. Saxton Todays Topics Blue Waters overview TensorFlow Basics

slide-1
SLIDE 1

Machine Learning on Blue Waters Using TensorFlow with the Image Feature Detection Problem

Or: How I Learned to Stop Worrying And Love AI Presented By: Dr. Aaron D. Saxton

2/8/18

slide-2
SLIDE 2

Todays Topics

  • Blue Waters overview
  • TensorFlow Basics
  • Statistics Review
  • Neural Networks
  • Convolutions
  • Convolutional Neural Networks
  • ImageNet
  • Blue Waters TensorFlow Process
  • Distributed Tensor Flow
  • TensorBoard

2

slide-3
SLIDE 3

Blue Waters Overview

  • Brief Summary
  • AMD Interlagos
  • NVIDIA Tesla
  • 22,636 XE Compute Nodes
  • 4,228 XK Compute Nodes
  • Cray Gemini Interconnect

3

slide-4
SLIDE 4

Blue Waters Overview

4

H T 3 H T 3

2 XE nodes

H T 3 H T 3

PCIe Gen2

2 XK nodes

slide-5
SLIDE 5

Blue Waters Overview

Sonexion: 26 usable PB

>1 TB/sec 100 GB/sec

Spectra Logic: 200 usable PB 400+ Gb/sec WAN

Scuba Subsystem: Storage Configuration for User Best Access

1.66 PB

10/40/100 Gb Ethernet Switch IB Switch External Servers

13.34 PFLOPS

slide-6
SLIDE 6

TensorFlow Basics

  • Python API
  • C++ under the hood
  • Mediator Design Pattern
  • Uses python context manager (with)
  • Workflow
  • Construct operations
  • Assign to name scope and or device
  • Enter Session context
  • run()
  • www.tensorflow.org/programmers_guide/low_level_intro

6

slide-7
SLIDE 7

TensorFlow Basics

7

Demo

slide-8
SLIDE 8

Statistics Review

  • Simple y = $ % & + ( regression
  • Least Squares to find m,b
  • With data set { &*, ,* } *./,..,1
  • Very special, often hard to measure ,*
  • Let the error be
  • 2 = ∑

[(,* − $ % &* + ( ]8

1 *./

  • Minimize 9 with respect to $ and (.
  • Simultaneously Solve
  • 2: $, ( = 0
  • 2<($, () = 0
  • Linear System
  • We will consider more general , = >(&)
  • 2: $, ( = 0 and 2< $, ( = 0 may not be linear

8

slide-9
SLIDE 9

Statistics Review

  • Regressions with parameterized sets of functions. e.g.
  • ! = #$% + '$ + ( (quadratic)
  • ! = ∑ #* $* (polynomial)
  • ! = +,-.(exponential)
  • ! =

/ /012(4567) (logistic)

  • After optimal parameters found,
  • Use function for inference
  • Have $, compute !

9

slide-10
SLIDE 10

Neural Networks

  • Activation functions
  • Softmax
  • !" #$, #&, … , #( =

+,- ∑ +,/

10

  • 7.5
  • 5
  • 2.5
2.5 5 7.5 10
  • 2.5
2.5
  • 2
  • 1.5
  • 1
  • 0.5
0.5 1 1.5 2 2.5 0.5 1 1.5
  • 2
  • 1.5
  • 1
  • 0.5
0.5 1 1.5 2 2.5 0.5 1 1.5

0 # = 0 # = 0 # = Logistic Arctan ReLU (Rectified Linear Unit)

slide-11
SLIDE 11

Neural Networks

11

  • Parameterized function
  • !" = % &'( + &(*
  • +, = -'. + -.!
  • /

,(*) = 2.(+)

  • -'3, -3, &'(, &(
  • Weights to be optimized

* ! + → 6

slide-12
SLIDE 12

Neural Networks

  • Finding Weights !"#, !#, %"&, %&
  • Back propagation
  • Nothing more than chain rule
  • Take partial derivative of error function '
  • This text is a good reference for nitty gritty details
  • The Elements of Statistical Learning, Second

Eddition, by Trevor Hastie, Robert Tibshirani, Jerome Friedman

  • Back propagation give errors (or loss)
  • Gradient Decent tells you how to update weights

12

slide-13
SLIDE 13

Convolutions

  • For two functions, ! " , $ "
  • (! ∗ $)(") = ∫

! + $ " − + -+

. /.

  • $ is the kernel to !
  • Above is a rolling average
  • http://setosa.io/ev/image-kernels/

13

slide-14
SLIDE 14

Convolutional Neural Networks

  • https://adeshpande3.github.io/adeshpande3.githu

b.io/The-9-Deep-Learning-Papers-You-Need-To- Know-About.html

  • Highlights
  • AlexNet
  • VGG Net
  • GoogLeNet (Inception)
  • Microsoft ResNet

14

slide-15
SLIDE 15

ImageNet

  • www.image-net.org
  • Large High Quality Dataset
  • 14,197,122 Images
  • 21841 synsets
  • Runs the Large Scale Visual Recognition

Challenge (ILSVRC)

  • Annotated
  • Bounding Boxes
  • synset
  • WordNet (http://wordnet.princeton.edu)

15

slide-16
SLIDE 16

ImageNet

  • Blue Waters hosts copy of ImageNet
  • Legal Term of Access
  • Create account on www.image-

net.org

  • Navigate to Term of Access
  • Accept Term of Access
  • Take screen shot or print to PDF

Term of Access with your name on it.

  • Email to saxton@illinois.edu
  • After I receive your Term of Access I

will give your Blue Waters user read permission to data

16

slide-17
SLIDE 17

ImageNet

17

Demo (Archive Tour)

slide-18
SLIDE 18

Blue Waters Tensorflow Process

  • github.com/asaxton/ncsa-bluewaters-tensorflow
  • Clone repo
  • cd ncsa-bluewaters-tensorflow/datasets/imagenet
  • qsub extract_data_from_archive.pbs
  • Wait for completion
  • qsub build_imagenet_data.pbs
  • Wait for completion
  • cd ncsa-bluewaters-tensorflow/run_scripts
  • qsub distributed_tf_launch.pbs
  • Result will be in the directory checkpoint_dir

18

slide-19
SLIDE 19

Blue Waters TensorFlow Process

19

Demo (Code Tour)

slide-20
SLIDE 20

Distributed TensorFlow

  • Resoures
  • www.tensorflow.org/deploy/distributed
  • www.oreilly.com/ideas/distributed-tensorflow

20

slide-21
SLIDE 21

Distributed TensorFlow

21

Demo (Code Tour)

slide-22
SLIDE 22

TensorBoard

  • Grab your checkpoint

22

Demo