Radiology Reporting with Life: Three Years' Experience using - - PowerPoint PPT Presentation

▶
radiology reporting with life three years experience
SMART_READER_LITE
LIVE PREVIEW

Radiology Reporting with Life: Three Years' Experience using - - PowerPoint PPT Presentation

Sep 23, 2023 462 likes •753 views

Radiology Reporting with Life: Three Years' Experience using Hyperlinked Interactive Multimedia Reporting Adoption and Value: Objective Evaluation via Click Through Rates Authors: Folio LR, Cohen G, Machado LB Radiology and Imaging Sciences,

slide-1
SLIDE 1

#SIIM18

@LesFolio

Radiology Reporting with Life: Three Years' Experience using Hyperlinked Interactive Multimedia Reporting

Adoption and Value: Objective Evaluation via Click Through Rates

Presenting author: Les Folio, DO, MPH, MSc, MAS Col (ret) USAF

Director, Clinical Image Processing Service and Lead Radiologist for CT, NIH CC Adjunct Clinical Professor of Radiology, George Washington University Hospital

Authors: Folio LR, Cohen G, Machado LB Radiology and Imaging Sciences, Clinical Center National Institutes of Health

slide-2
SLIDE 2

#SIIM18

@LesFolio

t

Disclosures, Disclaimers, Conflicts of Interest

Presenter manages a research agreement with Carestream Health

(the NIH Clinical Center PACS shown in this presentation)

Presenter has government issued diagnostic imaging patents

(unrelated; no royalties)

Presenter receives book author royalties

(Springer) This research was supported [in part] by the NIH Clinical Center Intramural Research Program The content is the responsibility of the presenter and does not necessarily represent the official views of the National Institutes of Health

s

slide-3
SLIDE 3

#SIIM18

@LesFolio

t

Objectives

  • Graphically depict adoption rates of hyperlinks in interactive reporting
  • over time since we implemented in Feb 2015
  • among radiology subspecialties and modalities supporting efficiency
  • Analyze clinicians/radiologists interaction with EMR, PACS, reports
  • and report hyperlinks evidenced by click through analysis by referring medical subspecialties
  • Objectively assess report value by exploring multimedia report interaction
  • by click through behaviors of referring clinicians and radiologists

Work in progress, AI: link to annotation supervised deep learning

  • how interactive reports help mitigate the deep learning missing “link”
slide-4
SLIDE 4

#SIIM18

@LesFolio

t

Background

  • Radiology reports have not dramatically changed in over 100 years
  • Since Roentgen’s discovery (1895)
  • Radiologist and oncologists surveyed on reporting preferences
  • verified oncologists and radiologists prefer hyperlinks
  • Descriptions in our reports are hyperlinked to measurements
  • to the annotation, providing unique opportunities:
  • to objectively analyze referring physicians Click Through Rate (CTR)
  • provides valuable labeling for Artificial Intelligence (AI)/ deep learning

* Folio L. Quantitative Radiology Reporting and Tumor Metrics: Survey of Oncologists and Radiologists. AJR. Oct 2015.

slide-5
SLIDE 5

#SIIM18

@LesFolio

Template-based reports used 100 years ago…

Russell D. Carman 1875-1926

Head of Roentgenology at the Mayo Clinic

Example of Dr. Carman’s report (1913), Mayo Clinic

Background (cont.)

Source: “Radiology: an illustrated history” by Ronald L. Eisenberg

slide-6
SLIDE 6

#SIIM18

@LesFolio

t

FINDINGS: Chest CT: Lungs, pleurae: lung nodules for example right upper lobe (1.9 cm x 1.5 cm) (series 4, image 81) Unchanged

Example Hyperlink Insertion while Dictating

Series 4 Image 81

  • Minimizes crosscheck
  • Metadata automatically includes: x,y,z

location, who measured, when, relation and designation, name, lesion type

  • “Active annotation” is either most recent

measured, clicked or “b” shortcut

Record

slide-7
SLIDE 7

#SIIM18

@LesFolio

t

slide-8
SLIDE 8

#SIIM18

@LesFolio

t

slide-9
SLIDE 9

#SIIM18

@LesFolio

t

slide-10
SLIDE 10

#SIIM18

@LesFolio

t

FINDINGS: Chest CT: Lungs, pleurae: Unchanged lung nodules for example right upper lobe (0.8 cm x 0.4 cm) (series 4, image 84) Mediastinum, heart, great vessels: Unchanged mediastinal adenopathy for example subcarinal (2.5 cm x 1.4 cm) (series 2, image 27) and right hilar adenopathy for example (5.1 cm x 2.4 cm) (series 2, image 32) and (2.1 cm x 1.4 cm) (series 2, image 25) Abdomen CT: Lymph nodes, abdominopelvic vascular: unremarkable Liver, spleen, biliary, gallbladder, pancreas: unremarkable GU Kidneys, ureters, adrenal glands: unremarkable GI Small and large bowel, mesentery, peritoneum: unremarkable Pelvic CT: Central pelvis, sidewalls: Unchanged anterior pelvic wall mass. Osseous structures, spine, body wall, soft tissues: unremarkable IMPRESSION:

  • 1. Unchanged lung nodules
  • 2. Stable mediastinal and hilar adenopathy/masses
  • 3. Unchanged anterior pelvic wall masses
  • 4. No evidence of new soft tissue mass

Multimedia Enhanced Radiology Reports

slide-11
SLIDE 11

#SIIM18

@LesFolio

t

Radiology Report Impression: “Stable metastatic lesions” Partial Response = Hope Stable Disease = “therapy is not reducing my cancer”

This number is what “counts” Patients can get conflicting messages in Patient Portal

Mu Mult ltim imedia ia a and Hyperlin links F s Facili ilitate Repor

  • rtin

ing f for

  • r Clin

linic ical Tria rials ls

RECIST 1.1

Evaluation of target lesions Complete Response (CR) Disappearance of target lesions (LN<1cm) Partial Response (PR) ≥30% decrease from baseline sum of target lesions size Progressive Disease (PD) ≥20% increase from baseline or best response* + absolute increase ≥ 5mm

  • n target or Non target lesions / New

lesions Stable Disease (SD) Neither CR or PD

slide-12
SLIDE 12

#SIIM18

@LesFolio

t

slide-13
SLIDE 13

#SIIM18

@LesFolio

t

1 Bookmarks = any image annotation

Not all bookmarks are hyperlinked Bookmark capability preceded links

2 Hyperlinks = hyperlinked text-to-annotation

Requires a bookmark

3 We had bookmark capability before hyperlinks

Hyperlinks implemented Feb 2015 3

Creation of Bookmarks1 & Report Hyperlinks2 (by Modality)

July 2014 – Dec 2017

slide-14
SLIDE 14

#SIIM18

@LesFolio

t

Creation of Bookmarks & Report Hyperlinks (Modality & Subspecialty)

July 2016 – Dec 2017

slide-15
SLIDE 15

#SIIM18

@LesFolio

t

Adoption by Radiologists

  • Radiologists immediately implemented interactive reporting
  • Shortly after capability was implemented
  • Expected subspecialty and modality variation
  • Body radiologists adopted more than neuro radiologists
  • Also greater adoption for CT and PET-CT than MRI
  • NIH adoption was greater than other centers; perhaps because
  • f NIH emphasis on oncology and clinical trials
slide-16
SLIDE 16

#SIIM18

@LesFolio

t

Click Through Analysis

  • Objective evaluation value of report hyperlinks to referring docs
  • Also to radiologists; we show body radiologist CTR
  • CTR is an indirect, yet objective measure of report value
  • A clinician must open a report to click; usually indicates reading them
  • We target PACS & EMR capability refreshers to infrequent users
  • To those clicking on report hyperlinks least often
  • Surgeons infrequent
slide-17
SLIDE 17

#SIIM18

@LesFolio

t

Body Radiologists click throughs

  • f bookmarks and hyperlinks

* Total CT and MR studies with reports signed by Body radiologists ** As above but where the study contains bookmarks or the report contains hyperlinks *** Total CT and MR Body studies with bookmarks or hyperlinks where these were clicked at least once by a radiologist 1116 822 1403

200 400 600 800 1000 1200 1400 1600

Total created* Total created with bookmarks or hyperlinks** Total read and bookmark or hyperlink clicked***

# of studies Access from PACS to report bookmarks and hyperlinks by BODY (CT+MR) Radiologists Dec 2017

More studies are accessed than created Body radiologists find value in bookmarks they create for follow up purposes

slide-18
SLIDE 18

#SIIM18

@LesFolio

t

Physician Clicks from EMR to Thin Client Viewer Showing Images/Report

Report ”Click Through” analysis confirms value of imaging exams and reports

Institute

About 80% of click throughs from our EMR to our PACS are from four NIH institutes. Example action: Can help target EMR/ PACS training

slide-19
SLIDE 19

#SIIM18

@LesFolio

t

Discussion

  • Revolutionary technologic implementation (hyperlinks) often challenging
  • widespread adoption support improved efficiency
  • Applying advanced hyperlink analytic tools to radiology reports
  • can be objective evidence of report value
  • Study limitations include a small sample and unique capability
  • based on our experience of interactive reporting
  • we are confident our sample is representative of our hyperlink use
  • annotations can be distracting (initial complaints)
  • training provided to show Ctrl+G hides graphics; also “show only mine”
slide-20
SLIDE 20

#SIIM18

@LesFolio

t

Optimal Radiologist Annotations?

Intersection over union (IoU) and paired t-tests: Linear, two-diameter, and oval were 0.29±0.23, 0.70±0.22, and 0.73±0.15 respectively.

slide-21
SLIDE 21

#SIIM18

@LesFolio

t

Supervised Deep Learning Model of Annotated Lung CT images Interconnected with Hyperlinks in Interactive Radiology Reports

Do M, Folio L, Machado L. APECED Deep Learning. SCBTMR 2018.

slide-22
SLIDE 22

#SIIM18

@LesFolio

t

DeepLesion: 32,735 measured lesions/ bounding boxes (soon to be publically available)

slide-23
SLIDE 23

#SIIM18

@LesFolio

t

Real estate mantra (Location3) twist: Location, Labeling, Learning

slide-24
SLIDE 24

#SIIM18

@LesFolio

t

Conclusions

  • We demonstrated adoption of bookmark and hyperlink use
  • subdivided by radiology subspecialty; supporting efficiency
  • body radiologists had the highest hyperlink usage
  • We objectively assessed report value by referring clinicians & institutes
  • by analyzing CTR of hyperlinks within our EMR, reports
  • Report text lined directly to annotations help fill missing DL link

Manuscript Submitted to JDI May 2018.

slide-25
SLIDE 25

#SIIM18

@LesFolio

t

Thank you

Les.Folio@nih.com

slide-26
SLIDE 26

#SIIM18

@LesFolio

t

Data in an a Cancer Trial: An NCI Example

  • How much data is there?
  • 10 data points/ lesion
  • Other related data = 17
  • Assume 3 lesions per CT
  • Hence 47 data points per CT exam
  • A trial with 50 patients w 5 CT each:
  • Results in 11,750 data points
  • Currently all handwritten
  • Then typed, then retyped (at least

twice)

1 2 3 4 5 6 7, 8 9 10

1 2 3

1 2

12

slide-27
SLIDE 27

#SIIM18

@LesFolio

t

Digital Data Management

  • “ENABLE”
  • Exportable Notation And Bookmark List Engine
  • Following Bookmark List exportation from PACS:
  • ENABLE exports verified tumor measurements
  • and associated meta data
  • Goals:
  • Eliminate handwriting
  • Minimize transcription errors
  • Increase efficiency
  • Fully digital transfers
  • N. Goyal, L Machado, L. Folio. ENABLE (Exportable Notation And Bookmark List Engine): An Interface to Manage

Tumor Measurement Data from PACS to Cancer Databases. JDI. January 2017

RESIST (RECIST)

slide-28
SLIDE 28

#SIIM18

@LesFolio

t

ENABLE-ing Data Management

  • Three outputs include digital RECIST sheets, data models and XL
  • Publically available on Github
  • https://github.com/Ngoyal95/ENABLE

* Machado L, Folio L. Managing tumor measurements and data in PACS. Feb 2017. JDI