vis tools and case studies vis tools and case studies
play

Vis tools and case studies Vis tools and case studies Thomas - PowerPoint PPT Presentation

Jun 21, 2023 •119 likes •1.4k views

9/3/2018 Visualization of machine learning algorithms Vis tools and case studies Vis tools and case studies Thomas Torsney-Weir VDA research group, University of Vienna http://localhost:8080/slides.html#/ 1/127 9/3/2018 Visualization of

  1. 9/3/2018 Visualization of machine learning algorithms Why is this hard? Why is this hard? Machine learning Machine learning Visualization Visualization Fast algorithms Multi-dimensional spaces Sufficient data Comparing complex data Automatic learning Showing uncertainty http://localhost:8080/slides.html#/ 46/127

  2. 9/3/2018 Visualization of machine learning algorithms Vis helping ML Vis helping ML http://localhost:8080/slides.html#/ 47/127

  3. 9/3/2018 Visualization of machine learning algorithms Vis helping ML Vis helping ML How do they work together? Building models Validating models Understanding models http://localhost:8080/slides.html#/ 48/127

  4. 9/3/2018 Visualization of machine learning algorithms Building models Building models http://localhost:8080/slides.html#/ 49/127

  5. 9/3/2018 Visualization of machine learning algorithms Building models Building models Meta parameters Model selection http://localhost:8080/slides.html#/ 50/127

  6. 9/3/2018 Visualization of machine learning algorithms What are meta parameters? What are meta parameters? Meta parameters control how learning takes place Learning rate Number and size of network layers Slack variables Stopping conditions http://localhost:8080/slides.html#/ 51/127

  7. 9/3/2018 Visualization of machine learning algorithms Why study meta-parameters? Why study meta-parameters? http://localhost:8080/slides.html#/ 52/127

  8. 9/3/2018 Visualization of machine learning algorithms Why study meta-parameters? Why study meta-parameters? http://localhost:8080/slides.html#/ 53/127

  9. 9/3/2018 Visualization of machine learning algorithms Manual method Manual method 1.0 0.5 0.0 0.5 1.0 http://localhost:8080/slides.html#/ 54/127

  10. 9/3/2018 Visualization of machine learning algorithms Manual method Manual method 1.0 0.5 0.0 0.5 1.0 http://localhost:8080/slides.html#/ 55/127

  11. 9/3/2018 Visualization of machine learning algorithms How to study them? How to study them? run a bunch of models and examine outputs paramorama design galleries http://localhost:8080/slides.html#/ 56/127

  12. 9/3/2018 Visualization of machine learning algorithms Paramorama Paramorama Automated sampling Manual sampling Pretorius, A. Johannes, Mark-Anthony P. Bray, Anne E. Carpenter, and Roy A. Ruddle. “Visualization of parameter space for image analysis,” 2011. http://localhost:8080/slides.html#/ 57/127

  13. 9/3/2018 Visualization of machine learning algorithms Paramorama Paramorama Pretorius, A. Johannes, Mark-Anthony P. Bray, Anne E. Carpenter, and Roy A. Ruddle. “Visualization of parameter space for image analysis,” 2011. http://localhost:8080/slides.html#/ 58/127

  14. 9/3/2018 Visualization of machine learning algorithms Paramorama Paramorama Pretorius, A. Johannes, Mark-Anthony P. Bray, Anne E. Carpenter, and Roy A. Ruddle. “Visualization of parameter space for image analysis,” 2011. http://localhost:8080/slides.html#/ 59/127

  15. 9/3/2018 Visualization of machine learning algorithms Design galleries Design galleries Marks, Joe, Brad Andalman, Paul A. Beardsley, William Freeman, Sarah Gibson, Jessica Hodgins, Thomas Kang, et al. “Design Galleries: A general approach to setting parameters for computer graphics and animation,” 1997. http://localhost:8080/slides.html#/ 60/127

  16. 9/3/2018 Visualization of machine learning algorithms http://localhost:8080/slides.html#/ 61/127

  17. 9/3/2018 Visualization of machine learning algorithms How to study them? How to study them? use a more principled approach http://localhost:8080/slides.html#/ 62/127

  18. 9/3/2018 Visualization of machine learning algorithms Objective measures Objective measures Ground truth - - Dice: 0.85 ... Image Error: 0.25 http://localhost:8080/slides.html#/ 63/127

  19. 9/3/2018 Visualization of machine learning algorithms Visual parameter space exploration Visual parameter space exploration conceptual pipeline Michael Sedlmair, Christoph Heinzl, Stefan Bruckner, Harald Piringer, and Torsten Möller "Visual parameter space analysis: A conceptual framework" IEEE Transactions on Visualization and Computer Graphics. 20(12) 2014. http://localhost:8080/slides.html#/ 64/127

  20. 9/3/2018 Visualization of machine learning algorithms Tuner Tuner - - Torsney-Weir, Thomas, Ahmed Saad, Torsten Möller, Britta Weber, Hans-Christian Hege, Jean-Marc Verbavatz, and Steven Bergner. “Tuner: Principled parameter finding for image segmentation algorithms using visual response surface exploration,” 2011. http://localhost:8080/slides.html#/ 65/127

  21. 9/3/2018 Visualization of machine learning algorithms http://localhost:8080/slides.html#/ 66/127

  22. 9/3/2018 Visualization of machine learning algorithms Tuner Tuner Torsney-Weir, Thomas, Ahmed Saad, Torsten Möller, Britta Weber, Hans-Christian Hege, Jean-Marc Verbavatz, and Steven Bergner. “Tuner: Principled parameter finding for image segmentation algorithms using visual response surface exploration,” 2011. http://localhost:8080/slides.html#/ 67/127

  23. 9/3/2018 Visualization of machine learning algorithms http://localhost:8080/slides.html#/ 68/127

  24. 9/3/2018 Visualization of machine learning algorithms Tuner Tuner Torsney-Weir, Thomas, Ahmed Saad, Torsten Möller, Britta Weber, Hans-Christian Hege, Jean-Marc Verbavatz, and Steven Bergner. “Tuner: Principled parameter finding for image segmentation algorithms using visual response surface exploration,” 2011. http://localhost:8080/slides.html#/ 69/127

  25. 9/3/2018 Visualization of machine learning algorithms http://localhost:8080/slides.html#/ 70/127

  26. 9/3/2018 Visualization of machine learning algorithms Tuner Tuner Torsney-Weir, Thomas, Ahmed Saad, Torsten Möller, Britta Weber, Hans-Christian Hege, Jean-Marc Verbavatz, and Steven Bergner. “Tuner: Principled parameter finding for image segmentation algorithms using visual response surface exploration,” 2011. http://localhost:8080/slides.html#/ 71/127

  27. 9/3/2018 Visualization of machine learning algorithms Building models Building models Meta parameters can have a large influence on performance Almost all ML algorithms require tuning Manual tuning is time consuming and error prone http://localhost:8080/slides.html#/ 72/127

  28. 9/3/2018 Visualization of machine learning algorithms Validating and verifying models Validating and verifying models http://localhost:8080/slides.html#/ 73/127

  29. 9/3/2018 Visualization of machine learning algorithms What do we mean? What do we mean? How do we know our models are working? model selection Committee on Mathematical Foundations of Verification, Validation, and Uncertainty Quantification; Board on Mathematical Sciences and Their Applications, Division on Engineering and Physical Sciences, National Research Council. Assessing the reliability of complex models: Mathematical and statistical foundations of verification, validation, and uncertainty quantification , 2012. http://www.nap.edu/openbook.php?record_id=13395 . http://localhost:8080/slides.html#/ 74/127

  30. 9/3/2018 Visualization of machine learning algorithms http://localhost:8080/slides.html#/ 75/127

  31. 9/3/2018 Visualization of machine learning algorithms Validating and verifying models Validating and verifying models Summary statistics are not always enough Balancing multiple objectives is difficult Certain training points might be very important http://localhost:8080/slides.html#/ 76/127

  32. 9/3/2018 Visualization of machine learning algorithms Examples Examples HyperMoVal - local inspection Sliceplorer - global inspection Tuner - error inspection http://localhost:8080/slides.html#/ 77/127

  33. 9/3/2018 Visualization of machine learning algorithms HyperMoVal HyperMoVal Piringer, Harald, Wolfgang Berger, and Jurgen Krasser. “HyperMoVal: Interactive visual validation of regression models for real-time simulation,” 2010. http://localhost:8080/slides.html#/ 78/127

  34. 9/3/2018 Visualization of machine learning algorithms Sliceplorer views Sliceplorer views Single layer NN (26 nodes) Dual layer NN (5 and 3 nodes) SVM (polynomial kernel) SVM (RBF kernel) Torsney-Weir, Thomas, Michael Sedlmair, and Torsten Möller. “Sliceplorer,” 2017. http://localhost:8080/slides.html#/ 79/127

  35. 9/3/2018 Visualization of machine learning algorithms Tuner error views Tuner error views Examining multi-dimensional functions error view shows where model is unsure can visually verify the model Prediction Prediction Optimization Optimization Error view Error view http://localhost:8080/slides.html#/ 80/127

  36. 9/3/2018 Visualization of machine learning algorithms Validating and verifying models Validating and verifying models Understand fit for individual samples Visual inspection to understand extrapolation Uncertainty can help to understand quality of prediction http://localhost:8080/slides.html#/ 81/127

  37. 9/3/2018 Visualization of machine learning algorithms Understanding models Understanding models http://localhost:8080/slides.html#/ 82/127

  38. 9/3/2018 Visualization of machine learning algorithms Who needs this? Who needs this? models are complex the business world likes spreadsheets because they can walk through the calculations http://localhost:8080/slides.html#/ 83/127

  39. 9/3/2018 Visualization of machine learning algorithms Simple vs complex models Simple vs complex models Simple Simple Complex Complex few factors multi-layer neural network small integer factors Gaussian process model low-depth trees non-linear many decisions http://localhost:8080/slides.html#/ 84/127

  40. 9/3/2018 Visualization of machine learning algorithms What does complexity buy us? What does complexity buy us? Global vs local models Deep-learning networks can deal with feature selection Can deal with edge cases http://localhost:8080/slides.html#/ 85/127

  41. 9/3/2018 Visualization of machine learning algorithms Understanding models Understanding models Just an answer is not enough (show your work) Humans have trouble understanding complex models Interactivity can bring people into the model http://localhost:8080/slides.html#/ 86/127

  42. 9/3/2018 Visualization of machine learning algorithms Methods Methods interaction walkthroughs simpler models ala LIME (Ribeiro et al. 2016) direct inspection http://localhost:8080/slides.html#/ 87/127

  43. 9/3/2018 Visualization of machine learning algorithms Examples Examples regression: Muhlbacher and Piringer clustering: Dis-function text processing: TagRefinery smaller models: Explanation explorer http://localhost:8080/slides.html#/ 88/127

  44. 9/3/2018 Visualization of machine learning algorithms Mühlbacher and Piringer Mühlbacher and Piringer Directly interact with the model building process Mühlbacher, Thomas, and Harald Piringer. “A partition-based framework for building and validating regression models,” 2013. Best Paper Award. http://localhost:8080/slides.html#/ 89/127

  45. 9/3/2018 Visualization of machine learning algorithms http://localhost:8080/slides.html#/ 90/127

  46. 9/3/2018 Visualization of machine learning algorithms Dis-function Dis-function Build a distance function interactively Brown, Eli T, Jingjing Liu, Carla E Brodley, and Remco Chang. “Dis-Function: Learning Distance Functions Interactively,” 2012. http://localhost:8080/slides.html#/ 91/127

  47. 9/3/2018 Visualization of machine learning algorithms http://localhost:8080/slides.html#/ 92/127

  48. 9/3/2018 Visualization of machine learning algorithms TagRefinery TagRefinery Tutorial/walkthrough system Text processing pipeline Kralj, Christoph, Mohsen Kamalzadeh, and Torsten Möller. “TagRefinery: A visual tool for tag wrangling,” 2017. http://localhost:8080/slides.html#/ 93/127

  49. 9/3/2018 Visualization of machine learning algorithms TagRefinery TagRefinery Kralj, Christoph, Mohsen Kamalzadeh, and Torsten Möller. “TagRefinery: A visual tool for tag wrangling,” 2017. http://localhost:8080/slides.html#/ 94/127

  50. 9/3/2018 Visualization of machine learning algorithms TagRefinery TagRefinery Kralj, Christoph, Mohsen Kamalzadeh, and Torsten Möller. “TagRefinery: A visual tool for tag wrangling,” 2017. http://localhost:8080/slides.html#/ 95/127

  51. 9/3/2018 Visualization of machine learning algorithms LIME method LIME method Ribeiro, Marco Tulio, Sameer Singh, and Carlos Guestrin. “‘Why should I trust you?’: Explaining the predictions of any classifier,” 2016. http://localhost:8080/slides.html#/ 96/127

  52. 9/3/2018 Visualization of machine learning algorithms http://localhost:8080/slides.html#/ 97/127

  53. 9/3/2018 Visualization of machine learning algorithms Explanation explorer Explanation explorer Krause, Josua, Aritra Dasgupta, Jordan Swartz, Yindalon Aphinyanaphongs, and Enrico Bertini. “A workflow for visual diagnostics of binary classifiers using instance-level explanations,” 2017. http://localhost:8080/slides.html#/ 98/127

  54. 9/3/2018 Visualization of machine learning algorithms http://localhost:8080/slides.html#/ 99/127

  55. 9/3/2018 Visualization of machine learning algorithms Direct inspection Direct inspection e.g. hidden states in a neural network http://localhost:8080/slides.html#/ 100/127

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

M&V 2.0: Modern Measurement Residential Case Studies Agenda Topics for discussion 1 What

M&V 2.0: Modern Measurement Residential Case Studies Agenda Topics for discussion 1 What

M&V 2.0: Modern Measurement Residential Case Studies Agenda Topics for discussion 1 What is M&V 2.0? 2 How does M&V 2.0 work? 3 FAQs 4 Case studies 2 Nomenclature Analytic tools and services that provide automated,

267 views • 22 slides
Successful Public/Private Hotel Case Studies & Tools in a Post-RDA Era January 27, 2015 by:

Successful Public/Private Hotel Case Studies & Tools in a Post-RDA Era January 27, 2015 by:

Presentation to Americas Lodging Investment Summit (ALIS) Successful Public/Private Hotel Case Studies & Tools in a Post-RDA Era January 27, 2015 by: Kosmont Companies 865 S. Figueroa Street, Suite 3500, Los Angeles, CA 90017

497 views • 20 slides
Discussions Tools in Collaborate Ultra Topics Case Studies Controversy Transfer

Discussions Tools in Collaborate Ultra Topics Case Studies Controversy Transfer

Discussions Tools in Collaborate Ultra Topics Case Studies Controversy Transfer Summary Learn and Share Practice for Larger Assessments

156 views • 11 slides
Case studies and case selection Gary Goertz Kroc Institute for International Peace Studies

Case studies and case selection Gary Goertz Kroc Institute for International Peace Studies

Case studies and case selection Gary Goertz Kroc Institute for International Peace Studies University of Notre Dame ggoertz@nd.edu Spring 2018 One uses case studies based on Z to answer the question: How generalizable is the case study?

1.23k views • 29 slides
Innovator Case Studies: Innovator Case Studies: Oncology Networks Oncology Networks J. Ike

Innovator Case Studies: Innovator Case Studies: Oncology Networks Oncology Networks J. Ike

Innovator Case Studies: Innovator Case Studies: Oncology Networks Oncology Networks J. Ike Nicoll Alan Armer, Ph.D. Chief Executive Officer Vice President, Research & Cancer Clinics of Excellence Development Catholic Health Initiatives

1.17k views • 12 slides
Case Studies - Case Studies - Eduroam in Slovenia Eduroam in Slovenia Rok Pape ARNES -

Case Studies - Case Studies - Eduroam in Slovenia Eduroam in Slovenia Rok Pape ARNES -

1 Akademska in raziskovalna mrea Slovenije Case Studies - Case Studies - Eduroam in Slovenia Eduroam in Slovenia Rok Pape ARNES - Academic and research network of Slovenia aaa-podpora@arnes.si Eurocamp, Ljubljana, April 2006 2 Start

1.25k views • 18 slides
Siner- -GI case studies: Objectives and GI case studies: Objectives and Siner methodology of

Siner- -GI case studies: Objectives and GI case studies: Objectives and Siner methodology of

FOOD AND AGRICULTURE ORGANIZATION OF THE UNITED NATIONS - SINER-GI Sharing views on Quality Products Linked to Geographical Origin; How they can contribute to rural development? 31 January -1 February 2008 Siner- -GI case studies:

940 views • 10 slides
Innovator Case Studies: Innovator Case Studies: Physician Practices Physician Practices Barbara

Innovator Case Studies: Innovator Case Studies: Physician Practices Physician Practices Barbara

Innovator Case Studies: Innovator Case Studies: Physician Practices Physician Practices Barbara L. McAneny, M.D. Lee S. Schwartzberg, M.D., FACP Chief Executive Officer, New Mexico Medical Director, The West Clinic Oncology Hematology

1.22k views • 32 slides
Innovator Case Studies: Innovator Case Studies: Community Hospitals Community Hospitals Paul R.

Innovator Case Studies: Innovator Case Studies: Community Hospitals Community Hospitals Paul R.

Innovator Case Studies: Innovator Case Studies: Community Hospitals Community Hospitals Paul R. ODea Angela Blackburn Executive Director Vice President, Clinical Operations, Mary Cedars-Sinai Outpatient Cancer Center Crowley Cancer

1.14k views • 20 slides
Case Studies Sasikumar M Overview Set of internal case studies Marathi Tutor SQL

Case Studies Sasikumar M Overview Set of internal case studies Marathi Tutor SQL

Case Studies Sasikumar M Overview Set of internal case studies Marathi Tutor SQL tutor Simultaneous equation tutor Marathi tutor (Abhijit Joshi) Attempt to follow first language learning of children Key points:

1.37k views • 31 slides
Principles of case control studies Part I Nature of the design of case control studies

Principles of case control studies Part I Nature of the design of case control studies

Principles of case control studies Part I Nature of the design of case control studies Odds ratio as the measurement of association Many slides in this presentation are from the World Health Organization and ization and Many slides in

1.33k views • 17 slides
Case Studies in Financial Aid Federal M ethodology Institutional M ethodology EFC Case

Case Studies in Financial Aid Federal M ethodology Institutional M ethodology EFC Case

Case Studies in Financial Aid Federal M ethodology Institutional M ethodology EFC Case Studies Needs Analysis : Formula by which an applicants ability to pay for educational expenses is evaluated and calculated. Principals of

1.18k views • 29 slides
RSC II Municipal Permitting Case Studies RSC II Municipal Permitting Case Studies February 25,

RSC II Municipal Permitting Case Studies RSC II Municipal Permitting Case Studies February 25,

Helping Massachusetts Municipalities Create a Cleaner Energy Future COMMONWEALTH OF MASSACHUSETTS Charles Baker , Governor Matthew Beaton, Secretary Judith Judson, Commissioner RSC II Municipal Permitting Case Studies RSC II Municipal

1.48k views • 43 slides
Neuropharmacy Masterclass Case Studies Jan 19 Case 1 29 yr old female RRMS diagnosed

Neuropharmacy Masterclass Case Studies Jan 19 Case 1 29 yr old female RRMS diagnosed

Neuropharmacy Masterclass Case Studies Jan 19 Case 1 29 yr old female RRMS diagnosed 10yrs ago Rebif started 2008 Jan 2016 - experiencing injection fatigue What are her options ? dimethyl fumarate or teriflunomide Case 1

507 views • 16 slides
EP EPIDEMIOL IDEMIOLOGIC OGICAL AL ST STUDIES: UDIES: tools tools to s to sha hape nutr

EP EPIDEMIOL IDEMIOLOGIC OGICAL AL ST STUDIES: UDIES: tools tools to s to sha hape nutr

For non-commercial use only PPP PPP IN INSIGH SIGHTS TS FR FROM OM PRECLINIC PRECLINICAL AND AL AND EP EPIDEMIOL IDEMIOLOGIC OGICAL AL ST STUDIES: UDIES: tools tools to s to sha hape nutr pe nutrit itional ional inter

612 views • 20 slides
Other types of studies Case Studies an in depth study of a particular situation rather than

Other types of studies Case Studies an in depth study of a particular situation rather than

Other types of studies Case Studies an in depth study of a particular situation rather than a sweeping statistical survey. a study of how something works in a very specific situation hard to generalize results generalized

575 views • 28 slides
Improving Undergraduate PL Curriculum Kathleen Fisher AT&T Labs Research 1 ACM SIGPLAN

Improving Undergraduate PL Curriculum Kathleen Fisher AT&T Labs Research 1 ACM SIGPLAN

Improving Undergraduate PL Curriculum Kathleen Fisher AT&T Labs Research 1 ACM SIGPLAN Workshop on Programming Language Curriculum (PLC) Motivation: Initiate discourse on the role of programming languages in the undergraduate

374 views • 26 slides
Algorithms for Graphical Games Luis E. Ortiz MIT CSAIL February 1, 2005 DIMACS Workshop on

Algorithms for Graphical Games Luis E. Ortiz MIT CSAIL February 1, 2005 DIMACS Workshop on

Algorithms for Graphical Games Luis E. Ortiz MIT CSAIL February 1, 2005 DIMACS Workshop on Bounded Rationality Joint work with Sham Kakade, Michael Kearns, John Langford, Michael Littman and Robert Schapire In this talk... Large population

592 views • 45 slides
Way Presented by Tamera Parsons, CPHQ July 24, 2017 1 Housekeeping Slides were sent this

Way Presented by Tamera Parsons, CPHQ July 24, 2017 1 Housekeeping Slides were sent this

Engaging the Frontline in Process Improvement, and Having Fun Along the Way Presented by Tamera Parsons, CPHQ July 24, 2017 1 Housekeeping Slides were sent this morning Webinar is being recorded Please use the telephone

503 views • 27 slides
APAC & ATAC July Meeting July 20/July 21, 2020 Texas Education Agency | Governance &

APAC & ATAC July Meeting July 20/July 21, 2020 Texas Education Agency | Governance &

APAC & ATAC July Meeting July 20/July 21, 2020 Texas Education Agency | Governance & Accountability | Performance Reporting Please mute your mic. Thank you! Zoom Meeting Norms Mute your microphone when necessary. o Zoom has a Mute

519 views • 28 slides
Introduction to HPSG Class 2: Constituent Order Variation & Complex Predicates Stefan M

Introduction to HPSG Class 2: Constituent Order Variation & Complex Predicates Stefan M

Constituent Order Variation & Complex Predicates Introduction to HPSG Class 2: Constituent Order Variation & Complex Predicates Stefan M uller Ivan A. Sag Theoretical Linguistics/Computational Linguistics Linguistics & CSLI

1.98k views • 186 slides
TTMG 5001 Principles of Management for Engineers Session 2: Sep 21 Fall 2011 Michael Weiss

TTMG 5001 Principles of Management for Engineers Session 2: Sep 21 Fall 2011 Michael Weiss

TTMG 5001 Principles of Management for Engineers Session 2: Sep 21 Fall 2011 Michael Weiss www.carleton.ca/tim www.carleton.ca/tim/tim.pdf weiss@sce.carleton.ca Session 2 objectives Upon completion of the session, you will know about

334 views • 31 slides
CSE3009: (Software Architecture and Design) Yann-Gal

CSE3009: (Software Architecture and Design) Yann-Gal

CSE3009: (Software Architecture and Design) Yann-Gal Guhneuc Patterns in Practice This work is licensed under a Creative Commons Attribution-NonCommercial- ShareAlike 3.0 Unported License Patterns

2.3k views • 162 slides
Review EEE118: Electronic Devices and Circuits Considered the four modes of operation of a BJT.

Review EEE118: Electronic Devices and Circuits Considered the four modes of operation of a BJT.

b b b EEE118: Lecture 12 Review EEE118: Electronic Devices and Circuits Considered the four modes of operation of a BJT. Looked at examples of the input, output and transfer Lecture XII characteristics of a BJT. Developed a large signal

260 views • 4 slides

More recommend