ARTIFICIAL INTELLIGENCE FOR DIGITAL PATHOLOGY Kyunghyun Paeng, - - PowerPoint PPT Presentation

▶
artificial intelligence for digital pathology
SMART_READER_LITE
LIVE PREVIEW

ARTIFICIAL INTELLIGENCE FOR DIGITAL PATHOLOGY Kyunghyun Paeng, - - PowerPoint PPT Presentation

Aug 17, 2023 376 likes •648 views

ARTIFICIAL INTELLIGENCE FOR DIGITAL PATHOLOGY Kyunghyun Paeng, Co-founder and Research Scientist, Lunit Inc. 1. BACKGROUND: DIGITAL PATHOLOGY 2. APPLICATIONS BREAST CANCER AGENDA PROSTATE CANCER 3. DEMONSTRATIONS 4. CONCLUSION 2

slide-1
SLIDE 1

ARTIFICIAL INTELLIGENCE FOR DIGITAL PATHOLOGY

Kyunghyun Paeng, Co-founder and Research Scientist, Lunit Inc.

slide-2
SLIDE 2

2

AGENDA

  • 1. BACKGROUND: DIGITAL PATHOLOGY
  • 2. APPLICATIONS
  • BREAST CANCER
  • PROSTATE CANCER
  • 3. DEMONSTRATIONS
  • 4. CONCLUSION
slide-3
SLIDE 3

3

DIAGNOSTIC PROCEDURE

BACKGROUND: DIGITAL PATHOLOGY

Patient Detection

(X-ray, CT, MRI, ...)

Radiology Diagnosis

(biopsy, resection, ...)

Pathology Treatment Oncology

slide-4
SLIDE 4

4

LIMITATIONS OF CONVENTIONAL PATHOLOGY

BACKGROUND: DIGITAL PATHOLOGY

Diagnosis

(biopsy, resection, ...)

Pathology Slide Report

(-) Archiving (-) Workflow (-) Analysis

slide-5
SLIDE 5

5

RISE OF DIGITAL PATHOLOGY

BACKGROUND: DIGITAL PATHOLOGY

Diagnosis

(biopsy, resection, ...)

Pathology

(+) Archiving (+) Workflow (+) Analysis

Digital pathology

slide-6
SLIDE 6

6

WHY DO WE NEED AI IN DIGITAL PATHOLOGY ?

BACKGROUND: DIGITAL PATHOLOGY

25% disagreement among pathologists in breast biopsy report.

“Diagnostic Concordance Among Pathologists Interpreting Breast Biopsy Specimens.”, JAMA, 2015.

(+) Reproducibility (+) Accuracy (+) Workload reduction

slide-7
SLIDE 7

7

CHALLENGES IN AI FOR DIGITAL PATHOLOGY

BACKGROUND: DIGITAL PATHOLOGY

  • 1. Gigapixel images
  • 2. Quality variation

Grade 1 Grade 2 Grade 3

3! 4! 3? 4?

  • 3. Ambiguity in ground-truth definition

~ 100,000 pixels

slide-8
SLIDE 8

8

KEY APPLICATIONS:

#1. Tumor proliferation score prediction in breast resection specimen. #2. Gleason score prediction in prostate biopsy specimen.

slide-9
SLIDE 9

9

WHAT IS TUMOR PROLIFERATION SCORE ?

APPLICATION #1: BREAST CANCER

Breast resection specimen

Mitosis

Proliferation score

(in 10 consecutive HPFs)

Score 1: ~6 mitosis Score 2: 6~10 mitosis Score 3: 10~ mitosis

prognosis good bad

slide-10
SLIDE 10

10

TUMOR PROLIFERATION SCORE PREDICTION

Data statistics

APPLICATION #1: BREAST CANCER

656 ROIs from 73 slides

,

Mitosis #1 (x,y) Mitosis #N (x,y)

...

Auxiliary dataset Proliferation score

,

500 slides Training dataset Tumor Proliferation Assessment Challenge 2016

TUPAC16 | MICCAI Grand Challenge

Proliferation score

,

321 slides Test dataset

slide-11
SLIDE 11

11

TUMOR PROLIFERATION SCORE PREDICTION

System overview

APPLICATION #1: BREAST CANCER

Whole slide image Tissue region extraction Patch extraction at x40 ROI detection using cell density

...

Stain normalization Mitosis Detection Network

  • 1. The number of mitosis
  • 2. The number of cells

Feature vector based on statistical information Support Vector Machine

Proliferation score

Auxiliary set for mitosis detection

Phase 1: Handling whole slide images Phase 2: Mitosis detection Phase 3: Score prediction

slide-12
SLIDE 12

12

TUMOR PROLIFERATION SCORE PREDICTION

Phase 1: Handling whole slide images

APPLICATION #1: BREAST CANCER

Whole slide image Tissue region extraction Patch extraction at x40 ROI detection using cell density

...

Stain normalization

  • Resizing a whole slide image.
  • Finding a threshold.
  • Morphological operations.
  • Patch extraction with 10 HPFs size.
  • Cell detection in each patch.
  • 30 ROIs selection.
  • Stain normalization.
slide-13
SLIDE 13

13

TUMOR PROLIFERATION SCORE PREDICTION

Phase 2: Mitosis detection

APPLICATION #1: BREAST CANCER

Mitosis Detection Network Auxiliary set for mitosis detection

conv 1, 3x3, 16 16 8 128 x 128 resblock 2.1, 3x3, 64 resblock 2.3, 3x3, 64 resblock 1.1, 3x3, 32 resblock 1.3, 3x3, 32 resblock 3.1, 3x3, 128 resblock 3.3, 3x3, 128 mitosis normal Global pooling layer

  • Based on Residual Network (ResNet).
  • 9 residual blocks = 21 layers architecture.
  • 2 step training procedure.
  • Cropped global pooling layer.

Training step: , Inference step:

slide-14
SLIDE 14

14

TUMOR PROLIFERATION SCORE PREDICTION

Phase 3: Score prediction

APPLICATION #1: BREAST CANCER

  • 1. The number of mitosis
  • 2. The number of cells

Feature vector based on statistical information Support Vector Machine

Proliferation score

  • Converting each WSI to a 21-dim feature vector.
  • 10-fold cross validation from 500 training samples.
  • Feature selection based on cross validation results.
slide-15
SLIDE 15

15

TUMOR PROLIFERATION SCORE PREDICTION

Results

APPLICATION #1: BREAST CANCER

Tumor Proliferation Assessment Challenge 2016

TUPAC16 | MICCAI Grand Challenge

slide-16
SLIDE 16

16

WHAT IS GLEASON SCORE ?

APPLICATION #2: PROSTATE CANCER

Prostate biopsy specimen

Grade 1 Grade 2 Grade 3 Grade 4 Grade 5 Grade 1, 2 Grade 3 Grade 4 Grade 5 Core #1: 5+5 Core #2: 0 Core #3: 3+4 Core #4: 0

slide-17
SLIDE 17

17

GLEASON SCORE PREDICTION

Data statistics

APPLICATION #2: PROSTATE CANCER

{ Grade, Contours }

,

900 slides Training dataset { Grade, Contours }

,

50 slides Test dataset

  • The number of patients: 385
  • The number of slides: 1152
  • The number of cores: 4907
  • The number of normal cores: 2872
  • The number of cancer cores: 2035

Dataset from medical centers

slide-18
SLIDE 18

18

GLEASON SCORE PREDICTION

System overview

APPLICATION #2: PROSTATE CANCER

Normal Grade 3 Grade 5 Grade 4

Normal Grade 3 Grade 4 Grade 5 Gleason score classification network

1000 1100 1110 1111 Ranking loss with thermometer code Memory network-based refinement (25 neighbors)

Patch-based classification

...

Embedded memory vector Query vector Embedding Memory network Refined

  • utput
slide-19
SLIDE 19

Normal Grade 3 Grade 5 Grade 4

19

GLEASON SCORE PREDICTION

Patch-based classification

APPLICATION #2: PROSTATE CANCER

  • Normal patches from only fully normal slides. è +~5% gain
  • Ranking loss with thermometer code. è +2~3% gain

Key features for improving performance

  • ResNet 101 architecture.
  • 512x512 patch with 75% overlap.
  • Softmax loss with 4 class classification.

Baseline settings

~75%

Not a classification problem! Ordering problem!

1000 1100 1110 1111

Network decodes from the left-most bit to the right-most bit.

slide-20
SLIDE 20

20

GLEASON SCORE PREDICTION

Memory network-based refinement

APPLICATION #2: PROSTATE CANCER

Query vector (1x4dim) Memory vector (25x4dim)

...

Patch-level outputs (25 neighbors) Embedding

... ... ... ...

1x1024 25x1024 Innerproduct

...

Attention vector 25x1

...

Weighting 1D-CNN Softmax Refined output

+ ~5% gain

slide-21
SLIDE 21

21

GLEASON SCORE PREDICTION

Results

APPLICATION #2: PROSTATE CANCER

Patch-level performance Core-level performance

  • Baseline: 75%

+ Data cleansing: 80% + Ranking loss: 82.8% + Memnet refinement: 87.5%

  • Normal or cancer core?
  • AUC: 97.8%
  • Gleason score prediction?
  • Only 1st major: 83%
  • Both: 76.7%
slide-22
SLIDE 22

22

DEMO #1: BREAST CANCER

slide-23
SLIDE 23

23

DEMO #2: PROSTATE CANCER

slide-24
SLIDE 24

24

CONCLUSION

Artificial intelligence for digital pathology

Challenge #1. How to handle gigapixel images ? (i.e., whole slide images) ü Consider how to sample patches. (patch size, sampling step, ...) è with pathologists. ü Consider how to construct whole pipeline from gigapixel images to diagnosis. Challenge #2. How to handle quality variation between slides ? ü Design image processing modules carefully. ü Do cross-validation to avoid overfitting.

Lessons learned

Challenge #3. How to handle ambiguous ground-truth ? ü Design task-specific loss. ü Sanitize training dataset as much as possible you can. ü Don’t be satisfied with patch-based results.

slide-25
SLIDE 25

THANK YOU

TEAM MEMBERS