SLIDE 1
VERY LARGE DEVELOPMENT
HOW TO RUN CODE REVIEW FOR 1000+ OPEN SOURCE DEVELOPERS
Joe Gordon
ABOUT ME
Developer @ Full time upstream OpenStack Developer contact information jog0 on freenode github.com/jogo launchpad.net/~jogo
VERY LARGE DEVELOPMENT HOW TO RUN CODE REVIEW FOR 1000+ OPEN - - PowerPoint PPT Presentation
VERY LARGE DEVELOPMENT HOW TO RUN CODE REVIEW FOR 1000+ OPEN - - PowerPoint PPT Presentation
VERY LARGE DEVELOPMENT HOW TO RUN CODE REVIEW FOR 1000+ OPEN SOURCE DEVELOPERS Joe Gordon ABOUT ME Developer @ Full time upstream OpenStack Developer contact information jog0 on freenode github.com/jogo launchpad.net/~jogo VERY LARGE
SLIDE 2
SLIDE 3
VERY LARGE DEVELOPMENT
AS DEFINED BY OHLOH BY BLACK DUCK “ Very large, active development team Over the past twelve months, 1241 developers contributed new code to
- OpenStack. This is one of the largest open-
source teams in the world, and is in the top 2% of all project teams on Ohloh. For this measurement, Ohloh considers only recent changes to the code. Over the entire history of the project, 1661 developers have contributed.”
SLIDE 4
WHAT IS ?
SLIDE 5
WHAT IS ?
LARGEST OPEN SOURCE PROJECT?
(THAT I KNOW OF) is 881,970 lines of python code and OpenStack is 968,893!
SLIDE 6
APIS
SLIDE 7
APIS
SLIDE 8
ARCHITECTURE
SLIDE 9
WHO IS OPENSTACK
SLIDE 10
SLIDE 11
SLIDE 12
DEPLOYMENT SCALE
THE DATA CENTER
(Rackspace data center)
SLIDE 13
DEVELOPMENT SCALE
SLIDE 14
BY THE NUMBERS
SLIDE 15
GRAPHS
In January 2011 61 Contributors 71,181 lines of code
SLIDE 16
PROS AND CONS OF USING
Pros fast to develop approachable language 'fun' Cons type checking
- ther static analysis
concurrency
SLIDE 17
DEVELOPMENT PROCESS
SLIDE 18
LIFE OF A GITHUB PATCH TODAY
- 1. Fork repo
- 2. Write code, test code and push to your github repo
- 3. Submit a pull request
- 4. travis-ci tests patch
- 5. Patch is reviewed
- 6. Patch is amended to address any negative comments
- 7. Code is merged
- 8. travis-ci runs on trunk
SLIDE 19
WHERE WE STARTED
LAUNCHPAD (BZR AND ALL)
(3 YEARS AGO)
SLIDE 20
REVIEW PROCESS EVOLUTION
SLIDE 21
GERRIT
SLIDE 22
GATE PEP8 + UNIT TESTS
SLIDE 23
GATE PEP8 + PY26, PY27, INTEGRATION TESTS
SLIDE 24
SLIDE 25
GATE + CHECK
SLIDE 26
CURRENT
SLIDE 27
LIFE OF AN OPENSTACK PATCH TODAY
- 1. Code is written and tested locally
- 2. Submitted for code review
- 3. Code is automatically tested on submission
- 4. Code is peer reviewed
- 5. Patch is amended to address any negative peer reviews
- 6. Code is approved (or rejected)
- 7. Code is re-tested as it will be merged
- 8. Code is merged
SLIDE 28
TODAY
SLIDE 29
PRINCIPLES
Never break trunk Master branch is always green Developers are never blocked on broken trunk Transparency Automate everything Egalitarian Be Strict. Reduce burden on reviewers
SLIDE 30
TOOLS
Work Flow Testing Integration Testing Communication
SLIDE 31
WORK FLOW
Gerrit git-review Jenkins
SLIDE 32
TESTING
tox testrepository Automated code quality checks: flake8+pep8+pyflakes+hacking
SLIDE 33
TESTING
AUTOMATED CODE QUALITY CHECKS
E1** indentation E2** white space E5** line length F401 module imported but unused F821 undefined name name H103 correct apache 2 license H30* import rules H301 docstring should not start with a space
SLIDE 34
INTEGRATION TESTING
devstack zuul recheck elastic-recheck
SLIDE 35
INTEGRATION TESTING
ELASTIC-RECHECK
Measure integration tests in percent failure, not black and white Hard to debug transient failures - 10s of MB of logs Trivial to classify failures once have elastic search query
SLIDE 36
COMMUNICATION
lauchpad etherpad pastebin irc mailing list
SLIDE 37
SUGGESTIONS FOR OTHER TOOLS?
SLIDE 38
OTHER WAYS WE MAKE DEVELOPMENT SCALE
SHRINK PROJECT SCOPE
OpenStack started with just two projects - Nova and Swift. Now the same functionality is spread out over 6 projects. Nova, Swift, Glance, Keystone, Cinder, Neutron
SLIDE 39
THANK YOU
QUESTIONS?
Statistics from ohlo.net Powered by reveal.js