Building and deploying deep learning models in medicine Leon Chen - PowerPoint PPT Presentation

Building and deploying deep learning models in medicine Leon Chen co-founder, MD.ai

Source: https://xkcd.com

Leon Chen, MD George Shih, MD

Ingredients Massive Data Massive GPU compute Sprinkle on some deep learning magic ???

Data considerations ● HIPAA and privacy concerns ● De-identification / anonymization ● Labels mined from EHR? ○ doctor’s notes, radiology/pathology reports, ICD9/10 codes, billing records, etc. ● Labels from scratch? ○ Impossible to crowdsource ○ Quality control (multiple annotators, inter-reader agreement statistics) ○ Expensive (e.g., 100s of $ per hour per doctor) ○ Coordination, logistics algorithm engineering/tooling

Model summary 1) 3D convolutional ResNet for sex (M/F) prediction 2) 2D convolutional ResNet for nodule ROI generation 3) 3D convolutional ResNet for nodule ROI false positive reduction 4) 2D convolutional ResNet for generating bounding boxes from the nodule ROIs 5) 2D convolutional ResNet for predicting cancer probability on the bounding boxes 6) Xgboost meta-classifier for final cancer prediction using probability predictions from step 5 together with additional features generated during steps 1-5. No external data used. We annotated the given data ourselves.

Our model 1397 training 198 test (stage 1) Clinical factors, including nodule characteristics on Raw CT volume CT ROC AUC: 0.89 ROC AUC: 0.90 - 0.96

Deployment Caffe2go style transfer (via facebook)

Deployment

model.save_weights('model.hdf5') In Keras: with open('model.json', 'w') as f: f.write(model.to_json()) const model = new KerasJS.Model({ filepaths: { model: 'url/path/to/model.json', weights: 'url/path/to/model_weights.buf', In Keras.js: metadata: 'url/path/to/model_metadata.json' }, gpu: true })

try { await model.ready() const inputData = { 'input_1': new Float32Array(data) } const outputData = await model.predict(inputData) } catch (err) { // handle error }

Available Layers ● advanced activations : LeakyReLU, PReLU, ELU, ParametricSoftplus, ThresholdedReLU, SReLU ● convolutional : Convolution1D, Convolution2D, AtrousConvolution2D, SeparableConvolution2D, Deconvolution2D, Convolution3D, UpSampling1D, UpSampling2D, UpSampling3D, ZeroPadding1D, ZeroPadding2D, ZeroPadding3D, Cropping1D, Cropping2D, Cropping3D ● core : Dense, Activation, Dropout, SpatialDropout2D, SpatialDropout3D, Flatten, Reshape, Permute, RepeatVector, Merge, Highway ● embeddings : Embedding ● noise : GaussianNoise, GaussianDropout ● normalization : BatchNormalization ● pooling : MaxPooling1D, MaxPooling2D, MaxPooling3D, AveragePooling1D, AveragePooling2D, AveragePooling3D, GlobalMaxPooling1D, GlobalAveragePooling1D, GlobalMaxPooling2D, GlobalAveragePooling2D ● recurrent : SimpleRNN, LSTM, GRU ● wrappers : Bidirectional, TimeDistributed

Deep learning in Bind input tensors as textures Bind layer weights as textures (W, b) Bind uniforms (filter size, stride, etc.) Bind output textures Run WebGL fragment shader program Read texel values from output textures and map back to tensor CPU GPU

OffscreenCanvas, WebGL2, WebAssembly ● OffscreenCanvas - WebGL in worker threads ● WebGL2 - available as of Chrome 56 and Firefox 51 ○ Direct texel lookup ○ Lots more texture formats, including integer textures ○ 3D textures ○ Non-power-of-2 texture support ○ No more loop restrictions in shaders ○ Uniform buffer objects - much faster binding and access ● WebAssembly ○ Binary, web-native compilation target from other languages, such as C/C++

How do models make it to widespread use? Ease of use Clinically validated

distill.pub

Architecture Continuous GPUs engineering (automated?) Preprocessing Annotation Modeling Training Deployment Validation Data Ingestion Create labels Cloud (e.g. docker, kubernetes) Anonymization Edge (e.g. Keras.js) Preprocessing

Source: http://theconversation.com/how-the-internet-was-born-from-the-arpanet-to-the-internet-68072 Source: https://en.wikipedia.org/wiki/Scale-free_network

hello@md.ai @transcranial https://github.com/transcranial