1 automating machine learning and deep learning workflows
play

1 Automating Machine Learning and Deep Learning Workflows 2 - PowerPoint PPT Presentation

Jan 07, 2023 •546 likes •794 views

1 Automating Machine Learning and Deep Learning Workflows 2 Information Name: Mourad Mourafiq Author of an open source platform: Polyaxon twitter: @mmourafiq GitHub: mouradmourafiq 3 What is Polyaxon Solves the

  1. � 1

  2. Automating Machine Learning and Deep Learning Workflows � 2

  3. Information • Name: Mourad Mourafiq • Author of an open source platform: Polyaxon • twitter: @mmourafiq • GitHub: mouradmourafiq � 3

  4. What is Polyaxon • Solves the machine learning life cycle • Can be deployed on premise or on any cloud platform • Is open source • Works with any library or framework • Can be used by single users or large organizations • Provides compliance, auditing, and security � 4

  5. Why you need a tool to manage your ML operations? • Software development is mature • Why not use the same tools? • What is the difference between software development and ML development? • What is the difference between software deployment and ML deployment? � 5

  6. Difference between software development and ML development • Development objectives • Vetting and quality assurance • Development stack � 6

  7. Difference between software deployment and ML deployment • ML deployment needs a Feedback Loop • Iteration and refinement • People involved in the deployment cycle � 7

  8. What should a ML platform answer • Should be flexible to support open source initiatives • Provides different deployment options • Ideally open source • Works with any library or framework • Scales with users • Provides compliance, auditing, and security � 8

  9. ML development lifecycle • Data access • Data exploration and Feature engineering • Experimentation: iteration, packaging, reusability, reproducibility. • Scaling: Scheduling, orchestration and optimization • Tracking: code, data, params, artifacts, metrics • Insights, reporting, and knowledge distribution • Model management: packaging, deployment, and distribution • Compliance, auditing, and access management. • Automation, events, and workflows • User experience � 9

  10. • Data access � 10

  11. • Data exploration & Feature engineering � 11

  12. • Experimentation • Different environments: local, remote, cluster • Portability and reusability • Reproducibility � 12

  13. • Experimentation: Different environments � 13

  14. • Experimentation: Packaging • polyaxon run -f polyxonfile.yaml • polyaxon run -f polyxonfile.yaml —local � 14

  15. • Scheduling & Orchestration � 15

  16. • Hyperparams tuning & distributed training � 16

  17. • Experiments tracking � 17

  18. • Experiments tracking � 18

  19. • Insights, reporting, and knowledge distribution � 19

  20. • Model Management � 20

  21. • Compliance & Governance • Manage model development and deployment • Rigorous and auditable workflows � 21

  22. • Automation & Events • Simple yet effective specification to create workflows and automation • Integration with other pipelining tools, e.g. airflow • Events and triggers based on data, code, metrics, … � 22

  23. mourad@polyaxon.com twitter: @mmourafiq GitHub: mouradmourafiq https://polyaxon.com twitter: @polyaxonai GitHub: polyaxon � 23

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

How Developers Iterate on Machine Learning Workflows -- A Survey of the Applied Machine Learning

How Developers Iterate on Machine Learning Workflows -- A Survey of the Applied Machine Learning

How Developers Iterate on Machine Learning Workflows -- A Survey of the Applied Machine Learning Literature Doris Xin 1 , Litian Ma 1 , Shuchen Song 1 , Rong Ma 2 , Aditya Parameswaran 1 1 University of Illinois at Urbana-Champaign 2 Peking

337 views • 20 slides
Deep Learning: Intro Juhan Nam Review of Traditional Machine Learning The traditional machine

Deep Learning: Intro Juhan Nam Review of Traditional Machine Learning The traditional machine

GCT634/AI613: Musical Applications of Machine Learning (Fall 2020) Deep Learning: Intro Juhan Nam Review of Traditional Machine Learning The traditional machine learning pipeline Temporal Frame-level Unsupervised Classifier Summary

613 views • 27 slides
Basics of Machine Learning and Deep Learning (Part I) Machine Learning Tom Mitchell: An

Basics of Machine Learning and Deep Learning (Part I) Machine Learning Tom Mitchell: An

CSci 8980 Basics of Machine Learning and Deep Learning (Part I) Machine Learning Tom Mitchell: An algorithm that is able to learn from data Learning? A computer program is said to learn from experience E with respects to some

416 views • 37 slides
Effectiveness of Deep Learning Vs. Machine Learning in a Health Care Use Case RxToDx A Data

Effectiveness of Deep Learning Vs. Machine Learning in a Health Care Use Case RxToDx A Data

Effectiveness of Deep Learning Vs. Machine Learning in a Health Care Use Case RxToDx A Data Science Machine Learning/Deep Learning Show Case Dima Rekesh/Julie Zhu/Ravi Rajagopalan Optum Technology November, 2017 Analytics Machine

483 views • 32 slides
Bringing the human back in the loop Joaquin Vanschoren, TU/e (new) data Meta-learning Automate

Bringing the human back in the loop Joaquin Vanschoren, TU/e (new) data Meta-learning Automate

Bringing the human back in the loop Joaquin Vanschoren, TU/e (new) data Meta-learning Automate optimization data analysis improved initial workflows workflows Online collaboration Automating machine learning: a human-robot symbiosis

663 views • 19 slides
Deep Learning (CNNs) Deep Learning Readings: Matt Gormley Murphy 28 Bishop

Deep Learning (CNNs) Deep Learning Readings: Matt Gormley Murphy 28 Bishop

10-601 Introduction to Machine Learning Machine Learning Department School of Computer Science Carnegie Mellon University Deep Learning (CNNs) Deep Learning Readings: Matt Gormley Murphy 28 Bishop

985 views • 86 slides
Machine Learning 11 AI Slides (6e) c Lin Zuoquan@PKU 1998-2020 11 1 11 Machine Learning

Machine Learning 11 AI Slides (6e) c Lin Zuoquan@PKU 1998-2020 11 1 11 Machine Learning

Machine Learning 11 AI Slides (6e) c Lin Zuoquan@PKU 1998-2020 11 1 11 Machine Learning 11.1 Learning agents 11.2 Inductive learning 11.3 Deep learning 11.4 Statistical learning 11.5 Reinforcement learning 11.6 Transfer learning

1.7k views • 159 slides
Deep Learning and Hardware: Matching the Demands from the Machine Learning Community Ekapol

Deep Learning and Hardware: Matching the Demands from the Machine Learning Community Ekapol

Deep Learning and Hardware: Matching the Demands from the Machine Learning Community Ekapol Chuangsuwanich Department of Computer Engineering, Chulalongkorn University Deep learning Artificial Neural Networks rebranded Deeper models Bigger

808 views • 44 slides
Interpretable & Transparent Deep Learning Fraunhofer HHI, Machine Learning Group Wojciech

Interpretable & Transparent Deep Learning Fraunhofer HHI, Machine Learning Group Wojciech

Fraunhofer Image Processing Heinrich Hertz Institute Interpretable & Transparent Deep Learning Fraunhofer HHI, Machine Learning Group Wojciech Samek Northern Lights Deep Learning Workshop (NLDL19) Troms, Norway 8th January 2019

1.15k views • 51 slides
Machine Learning Basics Prof. Kuan-Ting Lai 2020/4/4 Machine Learning Francois Chollet , Deep

Machine Learning Basics Prof. Kuan-Ting Lai 2020/4/4 Machine Learning Francois Chollet , Deep

Machine Learning Basics Prof. Kuan-Ting Lai 2020/4/4 Machine Learning Francois Chollet , Deep Learning with Python, Manning, 2017 Machine Learning Flow ( ) ( ) ( ) Data Evaluation Training

783 views • 62 slides
Softmax Classifier + SGD Todays Class Intro to Machine Learning What is Machine Learning?

Softmax Classifier + SGD Todays Class Intro to Machine Learning What is Machine Learning?

CS6501: Deep Learning for Visual Recognition Softmax Classifier + SGD Todays Class Intro to Machine Learning What is Machine Learning? Supervised Learning: Classification with K-nearest neighbors Unsupervised Learning: Clustering with

1.36k views • 91 slides
Automating Cloud Deployment for Deep Learning Inference of Real-time Online Services Yang Li

Automating Cloud Deployment for Deep Learning Inference of Real-time Online Services Yang Li

Automating Cloud Deployment for Deep Learning Inference of Real-time Online Services Yang Li Zhenhua Han Quanlu Zhang Zhenhua Li Haisheng Tan DNN-driven Real-time Services Speech Recognition Image Classification Neural Machine Translation

500 views • 16 slides
Improving Reproducible Deep Learning Workflows with DeepDIVA M. Alberti 1 * , V. Pondenkandath 1*

Improving Reproducible Deep Learning Workflows with DeepDIVA M. Alberti 1 * , V. Pondenkandath 1*

Improving Reproducible Deep Learning Workflows with DeepDIVA M. Alberti 1 * , V. Pondenkandath 1* , L. Vgtlin 1 , M. Wrsch 12 , R. Ingold 1 , M. Liwicki 13 *Equal contribution 1 DIVA Group, University of Fribourg, Switzerland 2 IIT, FHNW

396 views • 20 slides
Distributed Deep Learning with Horovod Alex Sergeev, Machine Learning Platform, Uber Engineering

Distributed Deep Learning with Horovod Alex Sergeev, Machine Learning Platform, Uber Engineering

Distributed Deep Learning with Horovod Alex Sergeev, Machine Learning Platform, Uber Engineering @alsrgv Deep Learning Continues to improve accuracy long after older algorithms reach data saturation State of the art for vision,

866 views • 41 slides
Deep Learning Russ Salakhutdinov Associate Professor Machine Learning

Deep Learning Russ Salakhutdinov Associate Professor Machine Learning

Deep Learning Russ Salakhutdinov Associate Professor Machine Learning Department Carnegie Mellon University Canadian Institute of Advanced Research Mining for Structure Massive increase in

659 views • 49 slides
Machine Learning 10-701 Tom M. Mitchell Machine Learning Department Carnegie Mellon University

Machine Learning 10-701 Tom M. Mitchell Machine Learning Department Carnegie Mellon University

Machine Learning 10-701 Tom M. Mitchell Machine Learning Department Carnegie Mellon University March 31, 2011 Today: Readings: Learning representations III Deep Belief Networks ICA CCA Neuroscience example Latent

408 views • 11 slides
HOW TO SET UP A COMMUNITY-BASED ENVIRONMENTAL HEALTH SENSING PROJECT Dawn Nafus @dawnnafus

HOW TO SET UP A COMMUNITY-BASED ENVIRONMENTAL HEALTH SENSING PROJECT Dawn Nafus @dawnnafus

HOW TO SET UP A COMMUNITY-BASED ENVIRONMENTAL HEALTH SENSING PROJECT Dawn Nafus @dawnnafus Dawn.nafus@intel.com WHY COMMUNITY? WHY ENVIRONMENT? OUR SETUP (sort of) Fenceline.org ESDR (and a bunch of other things we tried and got no

484 views • 9 slides
Privacy-Preserving Computation with Trusted Computing via Scramble-then-Compute Hung Dang, Anh

Privacy-Preserving Computation with Trusted Computing via Scramble-then-Compute Hung Dang, Anh

Privacy-Preserving Computation with Trusted Computing via Scramble-then-Compute Hung Dang, Anh Dinh, Ee-Chien Chang, Beng Chin Ooi School of Computing National University of Singapore PETS 2017 Privacy-Preserving Computation with Trusted

477 views • 34 slides
Off-line Data Vali lidation for Water Network Modeling Studies M. Quiones* G, C. Verde** and

Off-line Data Vali lidation for Water Network Modeling Studies M. Quiones* G, C. Verde** and

Off-line Data Vali lidation for Water Network Modeling Studies M. Quiones* G, C. Verde** and L. Torres** *Universidad Tecnolgica de La Habana J.A. Echeverra **II-Universidad Nacional Autnoma de Mxico Nov 2019 1 Content 1.

609 views • 13 slides
approaches Dave Reynolds, Epimorphics Ltd @der42 Validation requirements based on experiences

approaches Dave Reynolds, Epimorphics Ltd @der42 Validation requirements based on experiences

Validation: requirements and approaches Dave Reynolds, Epimorphics Ltd @der42 Validation requirements based on experiences with data.gov.uk Linked Data Most current Linked Data in data.gov.uk is: described using a range of vocabularies

466 views • 7 slides
CS145: INTRODUCTION TO DATA MINING Text Data: Topic Model Instructor: Yizhou Sun

CS145: INTRODUCTION TO DATA MINING Text Data: Topic Model Instructor: Yizhou Sun

CS145: INTRODUCTION TO DATA MINING Text Data: Topic Model Instructor: Yizhou Sun yzsun@cs.ucla.edu December 4, 2017 Methods to be Learnt Vector Data Set Data Sequence Data Text Data Logistic Regression; Nave Bayes for Text

418 views • 30 slides
Take your Wor ord tem emplates s to o the he ne next xt le level! l! Doc ocuments

Take your Wor ord tem emplates s to o the he ne next xt le level! l! Doc ocuments

Take your Wor ord tem emplates s to o the he ne next xt le level! l! Doc ocuments tsCorePack for or Dynamics 365 365 DocumentsCorePack Unified document solutionfor Dynamics 365 Gener Ge erate Pro Process ss Auto utomate

356 views • 6 slides
Chapter 9 Cardinality Estimation How Many Rows Does a Query Yield? Cardinality Estimation

Chapter 9 Cardinality Estimation How Many Rows Does a Query Yield? Cardinality Estimation

Cardinality Estimation Torsten Grust Chapter 9 Cardinality Estimation How Many Rows Does a Query Yield? Cardinality Estimation Database Profiles Assumptions Architecture and Implementation of Database Systems Estimating Operator Summer

675 views • 34 slides
(R)DBMSs today DMBS Approx. market share (Microsoft Access) Database Systems Oracle

(R)DBMSs today DMBS Approx. market share (Microsoft Access) Database Systems Oracle

3/3/2014 (R)DBMSs today DMBS Approx. market share (Microsoft Access) Database Systems Oracle 40-50% IBM DB2 20-30% Microsoft SQL Server ~15% RDBMSs Sybase 3% The NoSQL movement () MySQL 1% PostgreSQL 0.5% 1 2 Oracle

346 views • 8 slides

More recommend