SLIDE 1
Building and deploying deep learning models in medicine
Leon Chen co-founder, MD.ai
Source: https://xkcd.com
Building and deploying deep learning models in medicine Leon Chen - - PowerPoint PPT Presentation
Building and deploying deep learning models in medicine Leon Chen - - PowerPoint PPT Presentation
Building and deploying deep learning models in medicine Leon Chen co-founder, MD.ai Source: https://xkcd.com Leon Chen, MD George Shih, MD Ingredients Massive Data Massive GPU compute Sprinkle on some deep learning magic ??? Data
SLIDE 2
SLIDE 3
Leon Chen, MD George Shih, MD
SLIDE 4
Ingredients
Massive Data Massive GPU compute Sprinkle on some deep learning magic ???
SLIDE 5
Data considerations
- HIPAA and privacy concerns
- De-identification / anonymization
- Labels mined from EHR?
○ doctor’s notes, radiology/pathology reports, ICD9/10 codes, billing records, etc.
- Labels from scratch?
○ Impossible to crowdsource ○ Quality control (multiple annotators, inter-reader agreement statistics) ○ Expensive (e.g., 100s of $ per hour per doctor) ○ Coordination, logistics
algorithm engineering/tooling
SLIDE 6
SLIDE 7
SLIDE 8
SLIDE 9
Model summary
1) 3D convolutional ResNet for sex (M/F) prediction 2) 2D convolutional ResNet for nodule ROI generation 3) 3D convolutional ResNet for nodule ROI false positive reduction 4) 2D convolutional ResNet for generating bounding boxes from the nodule ROIs 5) 2D convolutional ResNet for predicting cancer probability on the bounding boxes 6) Xgboost meta-classifier for final cancer prediction using probability predictions from step 5 together with additional features generated during steps 1-5.
No external data used. We annotated the given data ourselves.
SLIDE 10
SLIDE 11
SLIDE 12
Clinical factors, including nodule characteristics on CT
ROC AUC: 0.90 - 0.96
Raw CT volume
ROC AUC: 0.89
Our model 1397 training 198 test (stage 1)
SLIDE 13
Deployment
Caffe2go style transfer (via facebook)
SLIDE 14
Deployment
SLIDE 15
SLIDE 16
model.save_weights('model.hdf5') with open('model.json', 'w') as f: f.write(model.to_json()) const model = new KerasJS.Model({ filepaths: { model: 'url/path/to/model.json', weights: 'url/path/to/model_weights.buf', metadata: 'url/path/to/model_metadata.json' }, gpu: true })
In Keras: In Keras.js:
SLIDE 17
try { await model.ready() const inputData = { 'input_1': new Float32Array(data) } const outputData = await model.predict(inputData) } catch (err) { // handle error }
SLIDE 18
Available Layers
- advanced activations: LeakyReLU, PReLU, ELU, ParametricSoftplus, ThresholdedReLU, SReLU
- convolutional: Convolution1D, Convolution2D, AtrousConvolution2D, SeparableConvolution2D, Deconvolution2D,
Convolution3D, UpSampling1D, UpSampling2D, UpSampling3D, ZeroPadding1D, ZeroPadding2D, ZeroPadding3D, Cropping1D, Cropping2D, Cropping3D
- core: Dense, Activation, Dropout, SpatialDropout2D, SpatialDropout3D, Flatten, Reshape, Permute, RepeatVector,
Merge, Highway
- embeddings: Embedding
- noise: GaussianNoise, GaussianDropout
- normalization: BatchNormalization
- pooling: MaxPooling1D, MaxPooling2D, MaxPooling3D, AveragePooling1D, AveragePooling2D, AveragePooling3D,
GlobalMaxPooling1D, GlobalAveragePooling1D, GlobalMaxPooling2D, GlobalAveragePooling2D
- recurrent: SimpleRNN, LSTM, GRU
- wrappers: Bidirectional, TimeDistributed
SLIDE 19
SLIDE 20
SLIDE 21
SLIDE 22
Deep learning in
GPU
Bind input tensors as textures Bind layer weights as textures (W, b) Bind uniforms (filter size, stride, etc.) Bind output textures Run WebGL fragment shader program Read texel values from output textures and map back to tensor
CPU
SLIDE 23
OffscreenCanvas, WebGL2, WebAssembly
- OffscreenCanvas - WebGL in worker threads
- WebGL2 - available as of Chrome 56 and Firefox 51
○ Direct texel lookup ○ Lots more texture formats, including integer textures ○ 3D textures ○ Non-power-of-2 texture support ○ No more loop restrictions in shaders ○ Uniform buffer objects - much faster binding and access
- WebAssembly
○ Binary, web-native compilation target from other languages, such as C/C++
SLIDE 24
SLIDE 25
SLIDE 26
How do models make it to widespread use?
Ease of use Clinically validated
SLIDE 27
distill.pub
SLIDE 28
GPUs
Preprocessing Annotation Modeling Training Deployment Validation
Cloud (e.g. docker, kubernetes) Edge (e.g. Keras.js) Data Ingestion Anonymization Preprocessing Continuous Create labels Architecture engineering (automated?)
SLIDE 29
Source: http://theconversation.com/how-the-internet-was-born-from-the-arpanet-to-the-internet-68072 Source: https://en.wikipedia.org/wiki/Scale-free_network
SLIDE 30