Tools ls for m maintai aini ning an o ng an open so n sour - - PowerPoint PPT Presentation

Tools ls for m maintai aini ning an o ng an open so n sour urce P Pyt ython p n project

Ben N en Nuttall ll @b @ben en_nut nuttall ll

@ben_nuttall

Be Ben N Nut uttall ll

• So

Software engin e engineer er at B BBC BC N News L Labs bs

• Former

erly ly a at R Rasp spbe berry Pi Pi Founda undatio ion

• Crea

eator o

• f g

gpio piozer zero, p piwh whee eels a and p nd pyjokes

• Openso

pensour urce.com c corres espo ponden ent

• twit

witter er.com/ben_nu ben_nuttall

• git

github.com/bennut bennuttall ll

@ben_nuttall

Be Ben N Nut uttall ll

• So

Software engin e engineer er at B BBC BC N News L Labs bs

• Former

erly ly a at R Rasp spbe berry Pi Pi Founda undatio ion

• Crea

eator o

• f g

gpio piozer zero, p piwh whee eels a and p nd pyjokes

• Openso

pensour urce.com c corres espo ponden ent

• twit

witter er.com/ben_nu ben_nuttall

• git

github.com/bennut bennuttall ll

@ben_nuttall

Wha What this tal alk co covers

 Organisi

ising a g a Pyt ython n module ule

 Dist

istribu ibuting s ing software

 Using

ng gi git/GitHu Hub

 Virtual en

l envir ironmen ents

 Tes

esting & ng & a aut utomated ed tes estin ing

 Documen

entatio ion

 Licensin

ing g so software

@ben_nuttall

Wha What this tal alk i is no s not

• A thorough

gh follo llow-alo long ng tutorial o l on how to use ea e each o

• f the ~

e ~50 tools ls men entio ioned ned

• Me

e tell elling ing you wh whic ich tools ls to us use

• Me

e tell elling ing you that you n need ed to k know all o l of thes ese e tools ls in inside- ide-out ut in in o

• rder

der to be c be consider idered ed a a pr prope per Pyt Python pr programmer er

@ben_nuttall

gp gpiozero

• Pyt

ython li n library pr providing s iding sim imple ple API PI for ph physic ical c l comput utin ing wit with R Rasp spber berry P Pi

• Ea

Eases es the l e learni ning c ng curve e for young ung peo people, ple, b beginn eginner ers a s and ed nd educ ucators

• Nic

ice Pyt e Pythonic ic API PI wit with a advanc nced ed toolin ling for e expe xperienc ienced p ed programmer ers

• gpio

gpiozer ero.r .rea eadt dthedo edocs.io .io

• git

github.com/gp gpio iozer zero/gp gpio iozero

@ben_nuttall

piwh wheels

• Tooling

ling for a aut utomatin ing bu buil ildin ing g wheels o els of ever erythin ing on Py n PyPI PI

• piwh

piwheels. els.org – g – pip- pip-compa patib ible repo eposit sitory h hosting ing Arm wh whee eels

• Natively

ely c compil iled ed Arm wh wheels eels bu built ilt o

• n

Pi 3 i 3 hardw dware

• Repo

epository h hosted o d on 1 × n 1 × Pi se Pi serves es 1 1 millio llion do n downlo nloads ds per er m month

• piwh

piwheels. els.org

• git

github.com/piwh piwheels els/piwh piwheels eels

@ben_nuttall

Dave Jone nes

• Profes

essio sional p l programmer er, a amateur eur de dentis ist

• Resp

esponsible ible for im implem lemen enting m ing my c crazy zy i idea eas

• I wr

write e the e fjr fjrst st 9 90%, %, h he e wr writ ites es the ne e next 9 90%

• Co-author o
• f gpio

gpiozer ero a and piwh piwheels eels (als lso aut uthor o

• f pic

picamer era, c colo lorzer ero, pic picraft, sens sense- e- em emu, u, la lars, st structa, compoundpi, pi, pis pisen ense, e, pibo pibootctl, l, .. ...) .)

• Introdu

duced ed m me e to m most st o

• f the

e tools in in this is talk

@ben_nuttall

Writing ng a Pyt Pytho hon mo n modul ule

. └── project.py

@ben_nuttall

GitHub ub -

• p

personal al

@ben_nuttall

GitHub ub -

• o
• rgan

ganisation

@ben_nuttall

GitHub ub -

• o
• rgan

ganisation

@ben_nuttall

GitHub ub -

• co

collab laborators

@ben_nuttall

GitHub ub -

• b

branch anches

@ben_nuttall

GitHub ub -

• r

rele leas ases

@ben_nuttall

GitHub ub -

• i

issue ssues

@ben_nuttall

GitHub ub -

• p

pull ull r reque uests

@ben_nuttall

GitHub ub -

• p

project ct b boar ards

@ben_nuttall

Distribut uting so ng softwar are – h – how? w?

Package ma managers:

• Linu

nux – apt, rpm, m, yu yum

• Language package managers – pip, npm, gem
• Linu

nux portable – sna nap, fm fmatpak, AppIma mage

• Mac – home

mebrew Download from site tes:

• GitHub, GitL

tLab, Sourceforge

• Personal websites
• curl

@ben_nuttall

Distribut uting so ng softwar are – – wh why?

• Ea

Ease o e of a access ss

• Ex

Expe pectations

• Trust

st a and c confjd fjdenc ence

• St

Stabil ilit ity

@ben_nuttall

Lice censi nsing ng

• It’s im

impo portant to c choose a e a li licence e for a a pr projec ect

• Think

ink a about wh what would a ld ann nnoy you

• It’s im

impo portant to sp spec ecif ify whic ich lic licen ence

• It’s im

impo portant to in inclu lude de the lic e licenc ence e wit with the s e source c e code a de and d nd distribu ibutio ions

• Refer

er to choosea ealic licen ense. e.com

@ben_nuttall

Pack ackagi aging a P ng a Pyt ython m n module ule

. ├── project │ ├── __init__.py │ └── project.py ├── README.rst └── setup.py

@ben_nuttall

Pack ackagi aging a P ng a Pyt ython m n module ule – se – setup up.p .py

import os from setuptools import setup, find_packages def read(fname): return open(os.path.join(os.path.dirname(__file__), fname)).read() setup( name="project", version="0.1.0", author="Ben Nuttall", description="Really cool project", license="MIT", keywords=["sample", "project"], url="https://github.com/bennuttall/project", packages=find_packages(), long_description=read('README.rst'), )

@ben_nuttall

Pub Publi lishing a Py a Python m n module ule o

• n PyP

n PyPI

• Regis

egister a an a n account o

• n p

pypi. ypi.org

• Put your

ur a accoun unt d det etails ils in ~ n ~/.p .pypir ypirc:

ben@magicman:~ $ cat .pypirc [pypi] username: bennuttall password: correcthorsebatterystaple

• Ins

nstall ll Twine: wine:

• pip install twine

@ben_nuttall

Pub Publi lishing a Py a Python m n module ule o

• n PyP

n PyPI

@ben_nuttall

__ini nit__.p .py ch choice ces: s: g gpiozero

User er:

from gpiozero import LED

__ini _init__ __.p .py: y:

from .input_devices import LED, ...

set setup up.py

__version__ = '1.5.1' setup( version=__version__ ... )

@ben_nuttall

__ini nit__.p .py ch choice ces: s: p piwh wheels ls

User er:

from piwheels.master.pypi import PiWheelsTransport

__ini _init__ __.p .py: y:

__version__ = '0.17'

set setup up.py

import piwheels as app setup( version=app.__version__, ... )

@ben_nuttall

Entry p points

set etup.py: entry_points = { 'console_scripts': [ 'project = project.cli:main', ], } setup( ... entry_points=entry_points, ... )

@ben_nuttall

Virtual ual e environm nments

• Vir

irtual e environment for a a P Pytho hon p proje

• ject
• You
• u c

create e the e e envir iron

• nmen

ent, pip i install in l into it it

• Is

Isola

• lated from
• m you
• ur system Python

hon a and s system em p packages es

• Buil

ild you

• ur proje
• ject i

insid ide e it it, with c h cha hange ges " "in installe lled" in in real l time mkvirtualenv -p /usr/bin/python3 project pip install -e . pip install ipython deactivate workon project

@ben_nuttall

Mak akefj fjles

all: @echo "make install - Install on local system" @echo "make develop - Install symlinks for development" install: pip install . develop: pip install -e .

@ben_nuttall

Testing ng

• Writ

ite e tes ests to valida lidate wh what your c code de is s is sup uppo posed sed to do do

• Keep

eep your o

• ld

ld tes ests s to m make su e sure no e nothin ing g br brea eaks in s in fut uture

• For m

maxim imum um e efg fgect, wr write e tes ests b before you u writ ite e code de!

• Tes

esting c ng can be n be per performed ed qu quic ickly kly lo locall lly

• Tes

esting c ng can be n be a aut utomated – ed – e. e.g. . Travis a after er pu push sh

• Be

Be pr pragm gmatic ic! Test st edg edge c e cases, es, d don’t be be e exhaus ustiv ive

@ben_nuttall

Testing ng - asse ssert

from project import add assert add(2, 2) == 4

@ben_nuttall

Testing ng - pyt ytest

from project import add def test_add(): assert add(2, 2) == 4

@ben_nuttall

Testing ng - la layout ut

. ├── Makefile ├── project │ ├── __init__.py │ └── project.py ├── setup.py └── tests └── test_add.py

@ben_nuttall

pyt ytest

@ben_nuttall

Testing ng - pyt ytest

from project import add import pytest assert add(2, 2) == 4 with pytest.raises(TypeError): add("foo", "bar")

@ben_nuttall

Testing ng – m – mock ck

>>> from unittest.mock import Mock >>> m = Mock(msg=Mock(return_value="hello")) >>> m <Mock id='140656083514272'> >>> m.msg() 'hello'

@ben_nuttall

Testing ng – m – mock ck patch ch

def test_timeofday_value(mock_factory): with TimeOfDay(time(7), time(8), utc=False) as tod: assert repr(tod).startswith('<gpiozero.TimeOfDay object') assert tod.start_time == time(7) assert tod.end_time == time(8) assert not tod.utc with patch('gpiozero.internal_devices.datetime') as dt: dt.now.return_value = datetime(2018, 1, 1, 6, 59, 0) assert not tod.is_active dt.now.return_value = datetime(2018, 1, 1, 7, 0, 0) assert tod.is_active dt.now.return_value = datetime(2018, 1, 2, 8, 0, 0) assert tod.is_active dt.now.return_value = datetime(2018, 1, 2, 8, 1, 0) assert not tod.is_active

@ben_nuttall

Tox

• Run tes

ests in in mult ltiple Py le Python n versio sions s ns sim imult ultaneo eous usly ly

• Ubun

untu u user sers – – lo look for " "Dea eads dsnakes PP s PPA"

• tox.ini

.ini: [tox] envlist = {py27,py35,py36,py37,py38}

@ben_nuttall

Coverage age.p .py

• Mea

easurin ing c code c e cover erage o e of Pyt Python pr progr grams

• Moni

nitors s your pr progr gram, no noting ng wh whic ich p parts o

• f the c

e code de have e been e been exec ecuted ed

• Ana

naly lyses es the e so sour urce to iden identify c code e that c could uld have been e been exec ecuted ed bu but was n not

• Typic

ypicall lly u used ed to ga gauge ge the e e efg fgectiv ivenes ess o

• f test

ests

• Sh

Shows wh whic ich pa parts o s of your ur c code a de are b e bein eing touched b ed by your ur test ests, a and wh whic ich a are e not

@ben_nuttall

Coverage age

@ben_nuttall

Testing ng – – Travis C s CI

@ben_nuttall

GitHub ub – – Travis C CI & I & co codeco cov i integr gration

@ben_nuttall

Mak akefj fjles

all: @echo "make install - Install on local system" @echo "make develop - Install symlinks for development" @echo "make test - Run tests" install: pip install . develop: pip install -e . test: coverage run --rcfile coverage.cfg -m pytest -v tests coverage report --rcfile coverage.cfg

@ben_nuttall

Docume cumentation

• Tut

utoria ials

• Ho

How-to gu guides ides

• Ex

Expla plana natio ion

• Refer

erenc ence

• https:/

/documentation.divio.com/

@ben_nuttall

Docume cumentation - n - Gi GitHub ub

@ben_nuttall

Docume cumentation - n - Mar arkdown wn

# Ti Titl tle Some me t text xt ## H Hea eader er 2

• Li

List st it item

• [l

[lin ink]( ](htt ttp: p:// //foo

• o.c

.com/ m/)

@ben_nuttall

Docume cumentation - n - mkdocs cs

• Markdo

down- wn-ba based do ed documen entatio ion b builder uilder

• Ex

Expo ports s to s static ic H HTML

• Ea

Easy to wr writ ite, e, ea easy to dep deplo loy

• Can h

n host a anywh wher ere – e – e.g. G . Git itHub Hub pa pages o es or self self-host sted ed

@ben_nuttall

Docume cumentation – R n – ReST ( (ReSt Struct uctur ured Text)

===== Title ===== Some text Header 2 ======== * List item * :doc:`api_input`

@ben_nuttall

Docume cumentation - n - sphinx

• Re

ReST

• Extracts d

doc

• cs from d

m doc

• cstrin

ings gs

• Can e

emb mbed ad addit ition ional al b bespok

• ke d

docs cs

• Multiple o
• utputs:
• HTML
• PDF
• Epub
• La

Langu guage ge d doc

• cs l

lin inkin ing ( g (e.g. gp

• g. gpioz
• zero ca

can l lin ink to Python hon d doc

• cs usin

sing Re g ReST)

• Cross

ss-proj

• ject l

lin inkin ing ( g (e.g. gp

• g. gpio

iozero c

• can l

lin ink to

• pic

icame mera u usin sing Re g ReST)

@ben_nuttall

Docume cumentation - n - sphinx

Regular Classes =============== The following classes are intended for general use with the devices they

• represent. All classes in this section are concrete (not abstract).

LED

• .. autoclass:: LED(pin, \*, active_high=True, initial_value=False, pin_factory=None)

:members: on, off, toggle, blink, pin, is_lit, value PWMLED

• .. autoclass:: PWMLED(pin, \*, active_high=True, initial_value=0, frequency=100, pin_factory=None)

:members: on, off, toggle, blink, pulse, pin, is_lit, value

@ben_nuttall

Docume cumentation - n - sphinx

@ben_nuttall

Docume cumentation - n - Read adTheDocs cs

• Mult

ltip iple ver ersio sions ns ( (v1. 1.0, v1. 1.1, v1.2...) 1.2...)

• Br

Branc nches es

• Mult

lti- i-us user er m managem emen ent

• Ea

Easy to in integr egrate e wit with G GitHu Hub b to aut utomate bu e buildin ilding

@ben_nuttall

Docume cumentation - n - Gr Grap aphviz

@ben_nuttall

Docume cumentation - n - Gr Grap aphviz

digraph { graph [rankdir=RL]; node [shape=rect, style=filled, color="#2980b9", fontname=Sans, fontcolor="#ffffff", fontsize=10]; edge [arrowhead=normal, style=solid]; Button -> LED; }

@ben_nuttall

Docume cumentation - n - Gr Grap aphviz

digraph { graph [rankdir=RL]; edge [arrowhead=normal, style=solid]; /* Devices */ node [shape=rect, style=filled, color="#2980b9", fontname=Sans, fontcolor="#ffffff", fontsize=10]; led [label="Garden light"] light [label="Light sensor"] motion [label="Motion sensor"] /* functions */ node [shape=oval, style=filled, color="#9ec6e0", fontcolor="#ffffff"]; booleanized all_values all_values -> led; booleanized -> all_values; motion -> all_values; light -> booleanized; }

@ben_nuttall

Docume cumentation - n - Gr Grap aphviz

@ben_nuttall

Pr Project ct s struct uctur ure

. ├── coverage.cfg ├── docs/ ├── .github/ ├── .gitignore ├── LICENSE ├── Makefile ├── project/ ├── setup.py ├── tests/ ├── tox.ini └── .travis.yml

@ben_nuttall

Wha What this tal alk co covered

 Organisi

ising a g a Pyt ython n module – ule – m modul dule st e structure, e, s set etup.p .py, M Makefjl fjles es

 Dist

istribu ibuting s ing software – e – PyP PyPI, pip pip

 Using

ng gi git/GitHu Hub – b – r repo eposi sitories, ies, us user ers & o & orgs gs, colla llabo borators, iss ssues ues, PR PRs, projec ect boards, s, in integ egratio ions

 Virtual en

l envir ironmen ents – – vir irtualen lenvwr wrapp pper er

 Tes

esting & ng & a aut utomated ed tes estin ing – – asser ert, p pyt ytes est, m mock, c coverage, ge, tox, Travis C is CI

 Documen

entatio ion – n – markdo down, wn, R ReS eST, m mkdo docs, s, sp sphinx, inx, gr graphviz iz

 Licensin

ing g so software – e – c choosea ealic licen ense. e.org

@ben_nuttall

Tooli ling Tue uesd sday

• My tooling blo

ing blog: g: https: s:/ /toolin ling. g.bennut bennuttall. ll.com/

• Ins

nspir pired b ed by L Les es P Pounder er: https: s:/ /bigl. igl.es/

• New p

posts e s ever ery Tuesd esday

• New p

posts e s ever ery other er Tues uesda day

• New p

posts e s ever ery now a and then en, so somet etimes o s on a a Tue uesd sday

Tools ls for m maintai aini ning an o ng an open so n sour urce P Pyt ython p n project

Ben N en Nuttall ll @b @ben en_nut nuttall ll