● Walk away with a better understanding of the terms: ○ Artificial intelligence (AI) ○ Machine learning (ML) Objectives ○ Natural language processing (NLP) ● Feel more prepared to make informed decisions around the use cases of AI, ML, and NLP in your practice
● Terms & Definitions Agenda ● Guiding Principles ● Machine Learning @ Flatiron ● Takeaways ● Q&A
● Terms & Definitions Agenda ● Guiding Principles ● Machine Learning @ Flatiron ● Takeaways ● Q&A
Definition of Terms Artificial Intelligence (AI) Natural Machine Language Learning Processing (ML) (NLP)
Definition of Terms Artificial Intelligence (AI): AI the theory and development of computer systems able to perform tasks that normally require human intelligence , such ML NLP as visual perception, speech recognition, decision-making, and translation between languages.
Deep Blue
Definition of Terms Machine Learning (ML): AI the field of study that uses statistical techniques to give computers the ability to learn without being explicitly programmed. ML NLP
Machine Learning: Recognizing Faces
Machine Learning: Recognizing Faces
Machine Learning: Recognizing Faces Sharang Phadke ? Sharang Phadke ? Sharang Phadke Sharang Phadke ? Sharang Phadke ?
Sharang Phadke ~10 years ago
Definition of Terms Natural Language Processing (NLP): AI a collection of computational techniques that enable computers to analyze, understand, derive meaning from and make use of ML NLP human language.
NLP: Identifying Sentence Structures Visit note He does not want to pursue chemotherapy
Definition of Terms Artificial Intelligence (AI) Natural Machine Language Learning Processing (ML) (NLP)
● Terms & Definitions Agenda ● Guiding Principles ● Machine Learning @ Flatiron ● Takeaways ● Q&A
Guiding Principles for Machine Learning at Flatiron Health ML is math, ML is a tool, ML will empower humans, not magic. not a product. not replace them. ML is built on a series of It can enable new features, ML can make humans mathematical formulas, and but ML is not a product we faster and more accurate, understanding the errors these can use alone but it can’t do everything. formulas make is important
ML is math, not magic
ML is math, not magic
ML is math, not magic
ML is a tool, not a product User interface Prediction bar Keyboard
ML will empower humans, not replace them Suggestions, opportunities for human input
ML will empower humans, not replace them ML is great at Humans are great at ● Sifting through lots and ● Synthesizing information lots of examples ● Applying domain-specific ● Recognizing tiny knowledge patterns in data ● Adapting to new information Humans will always be necessary for ● Generating data for the model to learn from ● Evaluating the performance of ML models
Consider ML on a case by case basis ML performs well today when ... ● The question is a simple problem statement ● The question involves pattern recognition ● There are a lot of examples available to learn from
Consider ML on a case by case basis ML is a challenging fit when ... ● The problem requires synthesis and reasoning ● There is a large and complex knowledge base to learn ● There are many exceptions to the rule ● For those exceptions, there are few examples to learn from
Inform your decision making Identify the use case: Make ML work for you: Understand the math: What question are we asking How will this use case of ML What kinds of examples the ML model to answer for help with my day-to-day will be fed into the system us? work? to help it learn patterns? How will ML be How will the system deal part of a larger product for with cases where the ML my practice? prediction is wrong?
● Terms & Definitions Agenda ● Guiding Principles ● Machine Learning @ Flatiron ● Takeaways ● Q&A
Today, roughly 4% of all adult cancer patients enroll in clinical trials.
Screen patients using the visit list
Screen patients using the visit list
A patient snapshot is shown in the sidebar
A patient snapshot is shown in the sidebar
Metastatic status is usually only in visit notes
Why did we choose to use ML for this application? Recognizing documented evidence of metastatic status in a patient’s chart ML performs well today when ... ● The question is a simple problem statement ● The question involves pattern recognition ● There are a lot of examples available to learn from
Magic? Math! 0 ≤ Score ≤ 1 Likely not Likely metastatic metastatic
How is the model trained? Step 2: Record how frequently Step 1: Extract snippets of each snippet is in text from patient metastatic patients’ document documents vs. non-metastatic patients. Text Snippets Text Snippets Patient is early stage Patient is early stage Bone mets present Bone mets present Patient Recurrence Recurrence Docs Patient is a sports fan Patient is a sports fan Local disease Local disease
How does the model make inferences? Step 1: Extract snippets of Step 2: Use the recorded text from patient information about the documents importance of each snippet to make an inference Bone mets present Patient A’s Highly Likely Metastatic Recurrence Documents Patient is a sports fan
How do we measure success? > 90% accuracy when an inference is surfaced
How does the model handle instances when it’s not correct? Is clear when it doesn’t know - surfaces Leaves room for human input the inference as “Unknown”
Inform your decision making Understand the math: Identify the use case: Make ML work for you: Narrows down the list of The OncoTrials feature uses Determines “Is there patients to review for trials examples of patients we documented evidence that know are metastatic or not the patient is metastatic?” The model has an “Unknown” category when Enhances the overall patient uncertain, and allows room matching workflow in for human input OncoTrials
● Terms & Definitions Agenda ● Guiding Principles ● Machine Learning @ Flatiron Takeaways ● ● Q&A
Takeaways Artificial Intelligence (AI) Ask the key questions Understand the math Natural Machine Language Learning Processing Identify the use case (ML) (NLP) Make ML work for you
Questions
We want to hear from you! Questions, comments, product feedback? Come visit us at the New Features booth
Recommend
More recommend
