TVM & THE APACHE SOFTWARE FOUNDATION MARKUS WEIMER MEMBER, - PowerPoint PPT Presentation

TVM & 
 THE APACHE SOFTWARE FOUNDATION MARKUS WEIMER MEMBER, APACHE SOFTWARE FOUNDATION ARCHITECT, MICROSOFT ML PLATFORM

TVM & 
 THE APACHE SOFTWARE FOUNDATION Why I am here MARKUS WEIMER MEMBER, APACHE SOFTWARE FOUNDATION ARCHITECT, MICROSOFT ML PLATFORM

TVM & 
 THE APACHE SOFTWARE FOUNDATION Why I am here MARKUS WEIMER MEMBER, APACHE SOFTWARE FOUNDATION ARCHITECT, MICROSOFT ML PLATFORM Pays me to be here

MARKUS WHO? � Newcomer to TVM ☺ � ML researcher and developer, worked on � CofiRank � Data Parallel SGD � Apache REEF � ML.NET � Member of the Apache Software Foundation � Inaugural PMC Chair for REEF � Mentor for mxnet, HiveMall, Nemo Here to help the TVM community get into The Apache Software Foundation

Intro about the ASF The Apache Way Community choices Next steps AGENDA

Intro about the ASF The Apache Way Community choices Next steps AGENDA

Open. Innovation. Community. The Apache Software Foundation The Apache projects are defined We consider ourselves provides support for the Apache by collaborative consensus based not simply a group of projects Community of open-source processes, an open, pragmatic sharing a server, but rather a software projects, which provide software license and a desire to community of developers and software products for the public create high quality software that users. good. leads the way in its field. Source: https://www.apache.org/

HISTORY: APACHE HTTPD 199? 1995 1999 2016 Admins First release Non-profit, Most widely trading of httpd membership deployed patches for based webserver for NCSA corporation 19 years [1] webserver Protects contributor and intellectual property [1] http://news.netcraft.com/archives/2016/04/21/april-2016-web-server-survey.html

WHAT IS THE ASF? -- A NON-PROFIT COOPERATION Officers Appoint Board (PMC PMC Chairs) Project Project Project Project Project Contributors Committers Elect Members Source for the numbers: http://www.infoworld.com/article/3079813/open-source-tools/the-apache-foundations-incredible-rise.html

WHAT IS THE ASF? -- A NON-PROFIT COOPERATION 100s Officers Appoint 100s Board (PMC PMC Chairs) Project Project Project Project 1000s Project Contributors Committers Elect Members Source for the numbers: http://www.infoworld.com/article/3079813/open-source-tools/the-apache-foundations-incredible-rise.html

PROJECT LIFECYCLE • Curriculum to learn responsibilities/procedures necessary to protect foundation: 
 Diverse community, accept new contributors, develop “in the open” Incubatio • Searching for a purpose and a role in the ecosystem. n • More users/legacy/visibility • Massive ego; “conquered the web”, etc. Mature • Propose new abstraction/interface, rather than deep changes to core function • Project retirement stage • Code still available, may be “complete”, but community has moved on Attic

Intro about the ASF The Apache Way Community choices Next steps AGENDA

Intro about the ASF The Apache Way Community choices Next steps AGENDA

CORE PRINCIPLES � Collaborative software development � Respectful, honest, technical-based interaction � Consistently high quality software � Commercial-friendly standard license � Faithful implementation of standards � Security as a mandatory feature

CORE PRINCIPLES � Collaborative software development � Respectful, honest, technical-based interaction � Consistently high quality software � Commercial-friendly standard license � Faithful implementation of standards � Security as a mandatory feature

CORE PRINCIPLES � Collaborative software development Community over code � Respectful, honest, technical-based interaction � Consistently high quality software � Commercial-friendly standard license � Faithful implementation of standards � Security as a mandatory feature

MERITOCRACY AND ROLES

PMC Developer Member Member • Use the • @apache.org • Board Liaison software • Tactical • VP @ ASF • Give answers • Project • ASF owners • Ask questions decision • Add code, stewardship making graphics, testing, … User Committer PMC Chair MERITOCRACY AND ROLES

Don’t be a sycophant Be easy to Disagree respectfully work with Focus on the technical details AUTHORITY IN APACHE Implied, Roles are responsibilities; authority necessary to perform occasionall function y explicit e.g., releases are lock-free; a struggling RM can (should) be preempted aversion to Leads rotate, authority partitioned to subdomains “leaders”

Explicit consensus Implicit consensus Veto powers • Releases, new • Code changes, • Every committer committers / documentation, … has veto powers PMCs, rules against code • Given by absence changes, … changes based on of veto technical reasons • [DISCUSS], • Pro tip: never [VOTE], [VOTE] [RESULT] email argue whether the threads reasons are technical CONSENSUS DRIVEN DEVELOPMENT

VOTING � +1: I want this to happen � Binding: The vote counts � +0: Meh � E.g. PMC votes on release (majority vote) � Non-binding � -0: Ewh, really? � E.g. User votes in a release � -1: No way. Veto (where possible) Voting should be relatively rare. The goal is that the project proceeds by consensus , and its members negotiate outcomes with one another. The votes (+1,-1, …) are sometimes used in discussions to express opinions outside of a [VOTE] thread.

As public as possible: Easy to search and archive user@ for usage questions E-Mails follow RFC 3676 dev@ for development discussion Plain text, Markdown where needed Quoting with `>` private@ for personal and legal matters COMMUNICATIONS

“If it didn’t happen on the mailing list, it didn’t happen”

LESSON: THE APACHE PROCESS LEADS TO BETTER SOFTWARE

LESSON: THE APACHE PROCESS LEADS TO BETTER SOFTWARE “If it isn’t on the mailing list, it didn’t happen”

LESSON: THE APACHE PROCESS LEADS TO BETTER SOFTWARE “If it isn’t on the mailing list, it didn’t happen” � The bad: � Higher latency per issue: Discussing things over email takes longer � This can be trained and gets get better over time.

LESSON: THE APACHE PROCESS LEADS TO BETTER SOFTWARE “If it isn’t on the mailing list, it didn’t happen” � The bad: � Higher latency per issue: Discussing things over email takes longer � This can be trained and gets get better over time. � The good: � Full visibility, no need for meetings to “bring everybody on the same page” � Documents the decision process, not just the outcome � Every developer (even the shy ones) have equal influence.

LESSON: THE APACHE PROCESS LEADS TO BETTER SOFTWARE “If it isn’t on the mailing list, it didn’t happen” � The bad: � Higher latency per issue: Discussing things over email takes longer � This can be trained and gets get better over time. � The good: � Full visibility, no need for meetings to “bring everybody on the same page” � Documents the decision process, not just the outcome � Every developer (even the shy ones) have equal influence. � The phenomenal: � Quality goes up (code as well as discussion becomes part of your CV) � Throughput goes up (less blockage, uncertainty)

Intro about the ASF The Apache Way Community choices Next steps AGENDA

Intro about the ASF The Apache Way Community choices Next steps AGENDA

PROCESS HISTORY � As communities form, they make Lucene Hadoop their own rules • Started out of Lucene � In practice, people copy / merge the Spark • Community rules from past projects overlap with Hadoop � REEF copied from Spark and Hadoop, which in turn spawned from Lucene REEF • Looks towards Spark and Hadoop

CTR VS. RTC

CTR VS. RTC Commit then review (CTR) � Every committer does the code changes they see fit. � Other committers review and undo changes. � Stabilization happens during release. � Messy commit log

CTR VS. RTC Commit then review (CTR) Review then commit (RTC) � Every committer does the code changes they � Every change is reviewed by another committer. see fit. � Every change is merged by another committer. � Other committers review and undo changes. � Always releasable software � Stabilization happens during release. � Extremely clean commit log � Messy commit log � Used by every Big Data project in Apache