Taking R Mainstream in Production Systems Misha Lisovich - PowerPoint PPT Presentation

Mar 24, 2024 •380 likes •578 views

Taking R Mainstream in Production Systems Misha Lisovich misha@honestbuildings.com The Question Q: Should I Use R in production? A: Yes! (In a couple of years) The Process 1. Productize - Compelling data products - Innovation pipeline 2.

Taking R Mainstream in Production Systems Misha Lisovich misha@honestbuildings.com
The Question Q: Should I Use R in production? A: Yes! (In a couple of years)
The Process 1. Productize - Compelling data products - Innovation pipeline 2. Ruggedize - Toolchain: Rstudio, Devtools, Github, Travis CI, Docker - Strong testing - Production-ready Architecture 3. Assimilate - Command line tools - Make it into HTTP APIs - Make it into Docker containers
Step 1: Productize Internal Products: - Ad-hoc Analyses - Internal Dashboards - Automated reports - Rapid Prototyping External Products: - End-user data products - Backend services
1. Dashboards Data & Job Monitoring Business Intelligence Internal Tools
2. Automated Reports = .Rmd -> html
3. Rapid Prototyping
4. Backend Services Batch Data Processing (ETL) R APIs
5. End-user Products
Step 2: Ruggedize 1. Create reproducible architecture 2. Set up strong testing & CI 3. Separate Production and Dev 4. Set up monitoring & reporting
Case Study: HB Architecture - Rstudio - Containerized Architecture - Continuous Integration - Multiple Environments - Notifications/Monitoring
Data Architecture Containers Docker Compose Web elasticsearch: Shiny image: elasticsearch Server Shiny shiny-server: Elastic rAPI Server image: shiny Elastic ports: - "443:443" links: ETL + = SQL - elasticsearch SQL S3 etl: ETL image:etl data volumes: - .:/data ETL Rstudio rAPI ETL etl-data: Data Server image: etl-data
Environments Production Staging staging-www.dataproduct.com www.dataproduct.com staging-internal-dashboards.com internal-dashboards.com Shiny Shiny Elastic Elastic Server Server SQL S3 SQL S3 data data ETL ETL volume volume
Continuous Integration commit Github Travis CI latest-stable tag pull latest-stable pull latest-stable Success! Staging Production
Docker Registry/Rolling Back Changes Deployed to Prod Save Versioned Image data Docker ETL volume Registry Danger! Need to Rollback! Load Older Image data Docker ETL volume Registry
Step 3: Assimilate! ( i.e. , be kind to your devs)
Assimilate (contd) - HTTP APIs - OpenCPU, rapier - Docker containers - Rocker - Command line tools - Rscript, littler, docopt
Thank you! misha@honestbuildings.com

Download Presentation

Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Moving HPC into the Mainstream Addressing the challenges of Mainstream HPC WHOS ON THE PANEL

Moving HPC into the Mainstream Addressing the challenges of Mainstream HPC WHOS ON THE PANEL Jon Collins / The Register Nick Curcuru / MasterCard Onur Celebioglu / Dell EMC Bryan Betts / Freeform Dynamics AGENDA HPC coming into the mainstream

478 views • 14 slides

THE BANK- -STATUS DECISION STATUS DECISION THE BANK OF THE FOREIGN BORN OF THE FOREIGN BORN

THE BANK- -STATUS DECISION STATUS DECISION THE BANK OF THE FOREIGN BORN OF THE FOREIGN BORN Taking That First Step Into The Taking That First Step Into The Financial Mainstream Financial Mainstream Sherrie L.W. Rhine Federal Reserve Bank

391 views • 20 slides

Taking Taking Taking Taking Aspiration and Aspiration and Aspiration and Aspiration and

Michael B Arthur Taking Taking Taking Taking Aspiration and Aspiration and Aspiration and Aspiration and Possibility into Possibility into Possibility into Possibility into Emeritus Professor, Suffolk University, Boston, USA Career

921 views • 30 slides

Strategic options Decide not to proceed with the mainstream projects 1. Defer a decision on

7/13/2010 Strategic options Decide not to proceed with the mainstream projects 1. Defer a decision on whether or not to proceed and in what form and 2. circumstances Proceed with mainstream development on a gradual phased basis 3. Proceed

351 views • 4 slides

The Journey to Mainstream Rosemary Cunningham Thomas, CEO Episurf Medical AB Annual General

The Journey to Mainstream Rosemary Cunningham Thomas, CEO Episurf Medical AB Annual General Meeting 2016 May 24 th 2016 Episurf Medicals goals and objectives are to mainstream patient-specific treatment options as a new standard within the

518 views • 19 slides

WELCOME! TAKING COOPERATION FORWARD 1 TAKING COOPERATION FORWARD 2 THE CO-ORDINATION TEAM

WELCOME! TAKING COOPERATION FORWARD 1 TAKING COOPERATION FORWARD 2 THE CO-ORDINATION TEAM Moderator: Screen and Support and Polls: Questions: Saydrina Ana-Maria Marlene Govender Baston Damerau TAKING COOPERATION FORWARD 3 Webinar 3

996 views • 66 slides

Audience journeys into non-mainstream film Bridgette Wessels Peter Merrington Matthew Hanchard

Audience journeys into non-mainstream film Bridgette Wessels Peter Merrington Matthew Hanchard Outline of talk Project background Research findings - what are the patterns of different journeys into non-mainstream film from the audience

711 views • 31 slides

Mainstream (Section 811) Vouchers Notice of Funding Availability (NOFA) April 18, 2018

Mainstream (Section 811) Vouchers Notice of Funding Availability (NOFA) April 18, 2018 Available Funding Omnibus Appropriations Act, 2017 made ~$13 million available for new Mainstream housing choice vouchers Omnibus Appropriations Act,

448 views • 17 slides

Why Geropalliative Medicine Must Become Mainstream for All Specialties Daniel R. Hoefer, MD CMO

Why Geropalliative Medicine Must Become Mainstream for All Specialties Daniel R. Hoefer, MD CMO Outpatient Palliative Care Sharp HospiceCare Disclosure I have no relevant financial disclosures Objectives Describe the changing paradigm of

1.01k views • 55 slides

California Cadet Corps Curriculum on Study Skills Taking Tests Test Taking Agenda B1.

California Cadet Corps Curriculum on Study Skills Taking Tests Test Taking Agenda B1. Test Preparation Tips B2. Strategies for Improving Test Performance B3. Short Answer Tests B4. Taking Essay Tests B5. Multiple Choice Test Taking

1.25k views • 53 slides

Financial Aid 101 Taking your Education to the Next Level! 2 Welcome We will be taking

Financial Aid 101 Taking your Education to the Next Level! 2 Welcome We will be taking questions using the Q&A feature at the end of our presentation. Click the ? symbol to see the feature box on the right of your screen.

952 views • 49 slides

5/15/2019 Disclosure Why Palliative Medicine Must Become Mainstream for All Specialties I have

5/15/2019 Disclosure Why Palliative Medicine Must Become Mainstream for All Specialties I have no relevant financial disclosures Daniel R. Hoefer, MD CMO Outpatient Palliative Care Sharp HospiceCare Objectives Describe the changing

298 views • 11 slides

Child & Family Agency, Ireland The Developm ent and Mainstream ing Program m e for

Child & Family Agency, Ireland The Developm ent and Mainstream ing Program m e for Prevention, Partnership and Fam ily Support: Research and Evaluation Study Dissem ination Event, 16th February 20 17, Croke Park, Dublin. Key Points:

538 views • 10 slides

Supported Employment As mainstream as possible As specialist as needed How supported

Supported Employment As mainstream as possible As specialist as needed How supported employment works Place, Train, Maintain Everyone can work (if they want to!) Individual support Employers are our customers! Get

101 views • 6 slides

Hertfordshires approach to Top -up Funding High Needs Funding (HNF) in Mainstream Schools

SEND Funding in Mainstream Schools and Settings Hertfordshires approach to Top -up Funding High Needs Funding (HNF) in Mainstream Schools and Settings www.hertfordshire.gov.uk www.hertfordshire.gov.uk Content The changes The

250 views • 24 slides

MIKE.A THE DJ Born in 1999, Mike.A is a Dj Producer Now Mike mainly produces Mainstream, from

MIKE.A THE DJ Born in 1999, Mike.A is a Dj Producer Now Mike mainly produces Mainstream, from Montreal, Qc, Canada, with an Bigroom, and Progressive House. One of enormous repertoire of music consisting of his many singles was Paradise

274 views • 5 slides

Open Source At The Top Of The Rack John W. Linville LinuxCon Europe 13 October 2014 John W.

Introduction Switches Today Open Opportunities New Horizons Problems Plan Of Action Conclusion Open Source At The Top Of The Rack John W. Linville LinuxCon Europe 13 October 2014 John W. Linville Tux On Top Introduction Switches Today

734 views • 28 slides

Promenade Park Garden Play Area New Play Area Layout wynne-williams associates Promenade Park

Promenade Park Garden Play Area New Play Area Layout wynne-williams associates Promenade Park Garden Play Area Vision for a New Play Area wynne-williams associates wynne-williams associates Promenade Park Garden Play Area LOCALLY EQUIPPED

323 views • 4 slides

Lab #5 4-Cylinder Single Overhead Cam Engine Dissection Equipment: 4-Cylinder Mazda 16 Valve

Engr 3 Mission College Faculty: Kate Disney TA: Andrew Dina Lab #5 4-Cylinder Single Overhead Cam Engine Dissection Equipment: 4-Cylinder Mazda 16 Valve SOHC 92 (Manual Transmission) Ratchet with 2 and 12 extensions 17 mm socket

456 views • 20 slides

Usability and Accessibility CS 4720 Web & Mobile

Usability and Accessibility CS 4720 Web & Mobile Systems CS 4720 What makes a good interface? Don't say anything but SIS. What

735 views • 27 slides

docker for data science Data science academy - HelloFresh Max Halford June 2017 0 why use

docker for data science Data science academy - HelloFresh Max Halford June 2017 0 why use docker? setup issues A lot of the time... Data science teams have a single server Project X needs Python 3 and Project Y needs Python 2

267 views • 22 slides

Setup docker rm $(docker ps -aq) docker network rm my_net Demo - Install and activate yum -y

Setup docker rm $(docker ps -aq) docker network rm my_net Demo - Install and activate yum -y install docker s ystemctl start docker systemctl enable docker docker images - Launch pre-canned container: hello-world (busybox is another good

480 views • 4 slides

SmartBatt Smart and Safe Integration of Batteries in Electric Vehicles An EU funded project

SmartBatt Smart and Safe Integration of Batteries in Electric Vehicles An EU funded project SmartBatt Smart and Safe integration of batteries into electric vehicles SmartBatt EU funded under 7 th Framework Programme Objectives: -

582 views • 40 slides

Large Subalgebras and the Structure of Crossed Products, Lecture 3: Large Subalgebras and the

Rocky Mountain Mathematics Consortium Summer School University of Wyoming, Laramie Large Subalgebras and the Structure of Crossed Products, Lecture 3: Large Subalgebras and the Radius 15 June 2015 of Comparison Lecture 1 (1 June 2015):

278 views • 7 slides