Bootstrapping Django with ease OSCON 2012 www.concentricsky.com - - PowerPoint PPT Presentation

Bootstrapping Django with ease

Introductions

Mike Biglan

CTO

Wiggins

Architect @coffindragger

Josh Bothun

Senior Developer

Areas of Concentric Sky

Mobile Enterprise Web/Django

(Some) Technologies We Use

• Chef
• Jenkins
• AWS
• Sphinx
• Memcached
• South
• git
• Backbone.js
• Jinja2
• Celery
• DebugToolbar
• jQuery / jQueryUI
• Compass / Sass
• Mongo & MySQL

Released CSky Creations

• djenesis – Easy Django Bootstrapping
• breakdown - Lightweight jinja2 template

prototyping server

• django-cachemodel – Automatic caching

for django models

CSky Creations in the Works

• skycloud – Tool for AWS auto-scaling
• django-skycms – CMS as a library
• django-sky-visitor – Invitations, email

confirmation, etc

• django-client-admin – Flexible admin for

client

CSky Teams

STAFF

50+

10 Django Team 5 Design Team 9 PM Team 3 System Admins

Team Interfacing

• Each project is a mix from each team
• Members might switch amongst projects
• Interfaces between teams
• Clean separation of handoffs and

responsibilities

• Best-practices shared within teams

“Bootstrapping”

• Pulling one up by one’s bootstraps
• Computers do this much better than humans
• Django – starting a project, new or existing

Bootstrapping with Ease

Making booting (and rebooting) a django project as easy as possible

• Ease is a continually moving target
• Shouldn’t sacrifice other goals for ease (e.g.

security, modularity, be PHP)

Moving Targets

Moving targets create new

• problems. And new solutions.
• CGI – Solving static HTML
• JavaScript – Solving non-interactivity
• CSS – Solving 100+ HTML pages
• Puppet/Chef – Solving 100+ servers

Problem/Solution Lifecycle

• 1. Problem grows
• 2. Many create their own solution; some

polish; some release

• 3. Community flocks to one (or a few) solutions
• 4. Solution(s) gets anointed and others die off

Goals of Easy Bootstrapping

• 1. More people using Django
• 2. More people using Django better

More People Using Django

• As few commands as possible to try

Django out

• And to try someone else’s project out.
• Must be highly likely to just work
• All especially critical for beginners
• Django’s runserver & sqlite...great!
• It just works...

Making Mistakes should be Hard

• Checklists are great
• The best checklists

are invisible

• To automate lists of

commands, must have consistency

Consistency Makes things Easier

• Ease of replication across servers
• Ease of sharing
• Both templates and existing projects
• Easier to build upon

Consistently Standardized

• Standardization: a tool to get to

consistency

• This can be in an organization. Or the full

community

• Oh yeah, and...

Standard had Better be Right

Fragmentation

• Poor standards lead to fragmentation
• Hard to build things on top of

fragmented environments

• Imagine building for single browser. Not

Mosaic.

Android Fragmentation

Flexibility vs Standardization

• Standardization can be rigid; can kill flexibility
• Especially as the target moves
• E.g. contrib.auth
• Flexibility allows preference, innovation
• Best of both worlds is standardization with

structured options of flexibility

• Django does this well

Django

• Django is a set of higher-order building blocks
• If blocks are too large, sacrifices what can be

made (i.e. not flexibile)

• If blocks are too small, becomes inefficient

and unmanageable

• If block types are wrong, not loosely

coupled (i.e. non-orthogonal)

3 Stages of Project Structure Grief

• Denial
• Bargaining
• Acceptance

Denial

• Stuff required libraries in a ‘lib’ directory
• Stick django there too
• Name everything MyProjectSomething

I’ll just take the easy way out...

Denial

• project starts to get bloated
• version management becomes a headache
• upgrade paths become tangled

I’ll just take the easy way out...

Why this doesn’t work

Bargaining

• 1. Have the team argue about which way to do things
• 2. Assign someone to write a document about it
• 3. ???
• 4. Ignore it when you start a new project

We’ll just adopt some standard practices...

Bargaining

• Projects become inconsistent
• It is not in source control
• No way to manage changes across projects
• There is no history of changes or discussion

We’ll just adopt some standard practices...

Why this doesn’t work

Acceptance

• requirements.txt, version management
• also solves the ‘git submodule’ problem
• virtualenv is great, always use it

Embrace the community

Acceptance

• dependency management
• concurrent project management
• sand-boxing
• solves the “chicken-n-egg” problem
• replication

No, really, use virtualenv

What’s in a template?

• For a django project, there are two styles:
• 1. toplevel directory is python code
• 2. toplevel directory is a website project

Toplevel is python code

• Pros
• More pythonic; Everything

is python code

• It just works
• Cons
• Everything is python code
• Where do non public files

go?

Django’s default template ./manage.py ./my_project/__init__.py ./my_project/settings.py ./my_project/urls.py ./my_project/wsgi.py

Default Django Project

./manage.py ./my_project/__init__.py ./my_project/settings.py ./my_project/urls.py ./my_project/wsgi.py ./my_project/my_app/__init__.py ./my_project/my_app/models.py

• How about ./my_project/

my_app/?

• Name then becomes

‘my_project.my_app’

• What about reusability?
• What if I want to rename

my_project?

• Where do apps go?

Default Django Project

• How about ./my_app/?
• Better for reusability, but...
• Now I have to add ./ to the

PYTHON_PATH

• Where do apps go?

./manage.py ./my_project/__init__.py ./my_project/settings.py ./my_project/urls.py ./my_project/wsgi.py ./my_app/__init__.py ./my_app/models.py

Default Django Project

• Was it ‘myapp1’ or ‘myapp2’

that contains the main urls and settings...

• What was the name

again?

./manage.py ./myapp1/models.py ./myapp1/urls.py ./myapp2/models.py ./myapp2/urls.py ./myapp3/models.py ./myapp3/urls.py

Toplevel is website project

• Pros
• Closer to real world

environments

• Consistency across

projects

• Cons
• ??? None I guess!

http://github.com/concentricsky/djenesis-template.git

• only have to add ./apps to PYTHON_PATH
• everything else is ignored by WSGI server

Python code is sandboxed Toplevel is website project

• I always know where to look first: ./apps/

mainsite/urls.py

• More important than it may seem at first blush

Consistent Entry Point Name Toplevel is website project

• Loaded after settings.py as overrides.
• Easily set up a local development environment.
• Production credentials are at no risk.
• Easily deploy devel, staging, production stacks.

settings_local Toplevel is website project

• Library dependencies and versioning is simple.
• New developers can start working quicker.
• Simple virtualenv integration

I have requirements.txt Toplevel is website project

Okay, but now what?

• 1. Install and use virtualenv
• 2. Design and use a template

How hard can it be?

Taking a closer look

$ mkvirtualenv myproject && workon myproject

OR

$ virtualenv ~/myproject/env && source ~/myproject/env/ bin/activate

THEN

Our approach

Djenesis: our solution to the bootstrapping problem

djenesis

• What is it?
• A tool that facilitates the process of

creating new and setting up existing Django projects.

• Available on github & PyPi (apache license)
• github.com/concentricsky/djenesis

djenesis - Benefits

• Simplifies routine workflows for:
• Starting a new project
• Getting an existing project up and running with ease
• Encourages developers to use a standard project

structure and best python practices (virtualenv)

• Allows external project templates
• Supports git, svn, hg URLs
• Supports Django 1.4 project templates

djenesis - History

• 1. Single zip file of a stub project
• 2. Script that wrapped stub
• 3. Decouples templates from script
• 4. Can use external templates, git URL
• 5. Now: supports Django 1.4 templates

djenesis - External Templates

• Inflating a project from a git/svn/hg source

means you can (and should) version control your project templates

• Collaborate with your internal teams on an

ideal project structure

djenesis Stack

djenesis - Setup

• Requires virtualenv
• Install virtualenv through pip or easy_install
• Install djenesis through pip or easy_install

djenesis – New Project

Before

$ virtualenv ~/myproject/env $ source ~/myproject/env/bin/activate $ pip install django $ git clone github.com/my/template.git /tmp/tmpl $ django-admin.py startproject --template=/tmp/tmpl ~/ myproject/code $ rm -rf /tmp/tmpl

With djenesis

$ djenesis myproject git+https://github.com/my/template.git $ source ~/myproject/env/bin/activate

djenesis – Existing Project

Before

$ virtualenv ~/myproject/env $ source ~/myproject/env/bin/activate $ git clone github.com/my/project.git ~/myproject/code $ pip install -r ~/myproject/code/requirements.txt

With djenesis

$ djenesis myproject -i git+https://github.com/my/project.git $ source ~/myproject/env/bin/activate

djenesis - Configuration

• Use a default template

DJENESIS_DEFAULT_TEMPLATE=/path/to/template

• Always use virtualenvwrapper (if installed)

DJENESIS_VIRTUALENVWRAPPER=1

• Set these in your .bashrc

djenesis – Next?

• 1. startapp template in project template
• 2. setup.py Support
• 3. djenesis post-create hooks for common tasks

(copy local settings, syncdb, import fixtures)

• 4. Activate virtualenv when djenesis is called

Bootstrapping Django with ease