Developing your own Swift middleware OpenStack Summit Atlanta, May - - PowerPoint PPT Presentation

Developing your own Swift middleware OpenStack Summit Atlanta, May 2014

About me

• Christian Schwede
• Developer @ eNovance
• Mostly working on Swift, testing and automation
• Started using Swift in 2012

Agenda

• Using middlewares to extend Swift functionality
• Introduction to wsgi, middlewares and paste.deploy
• Developing our own middleware for Swift Proxy
• Testing & Packaging
• References

Writing Swift middlewares?

Swift middlewares?

• Most of Swift features are implemented as a middleware

○ logging, tempurl, dlo, ratelimit, tempauth, quotas, ...

• Flexibility to extend existing functionality
• No need to fork or modify existing code
• Several additional middlewares outside of Swift

wsgi and middlewares?

wsgi

• simple interface between web servers & web applications
• defined in PEP0333 and PEP3333
• application

○ callable object (with a __call__ method)

• server

○ invokes the callable once for each client request

Client Server App

application objects

• must accept two positional arguments

○ environ: Python dictionary ○ start_response: callable ■ status = „200 OK“ ■ headers = [(„header_name“, „header_value“), ]

environ

• REQUEST_METHOD
• PATH_INFO
• QUERY_STRING
• HTTP_HEADERNAME
• wsgi.input

from wsgiref.simple_server import make_server def myapp(environ, start_response): body = [] body.append("PATH_INFO: '%s'\n" % environ.get('PATH_INFO')) headers = [('Content-Type', 'text/plain')] start_response('200 OK', headers) return body srv = make_server('localhost', 8000, myapp) srv.serve_forever()

Sample server

middleware

• can act as a server for an application and vice versa
• run multiple applications side-by-side
• authentication
• rerouting a request
• content processing

Client Server App Middleware

class SummitMiddleware(object): def __init__(self, app, *args, **kwargs): self.app = app def __call__(self, env, start_response): response = self.app(env, start_response) if env.get('PATH_INFO') == '/echo': length = int(env.get('CONTENT_LENGTH') or 0) return env.get('wsgi.input').read(length) return response srv = make_server(‘localhost’, 8000, SummitMiddleware(myapp))

Sample middleware

Testing, packaging & deploying

class FakeApp(object): def __call__(self, env, start_response): start_response('200 OK', []) return "" class TestSummitMiddleware(unittest.TestCase): def test_simple_request(self): environ = {'REQUEST_METHOD': 'PUT'} req = Request.blank('/echo', environ, body="Hello World") mw = SummitMiddleware(FakeApp()) resp = req.get_response(mw) self.assertEqual("Hello World", resp.body)

Testing a middleware

paste.deploy

• load WSGI applications and servers from URI
• uses INI-style configuration files
• separates config from code
• Paste Script can serve applications from config files
• widely used in Openstack

[app:sample] use = egg:sample#app [filter:middleware] use = egg:sample#middleware suffix = /echoresponse [pipeline:main] pipeline = middleware sample [server:main] use = egg:Paste#http port = 8000

config.ini for paste.deploy

setup(name='sample', packages=['sample', ], zip_safe=False, entry_points={ 'paste.app_factory': ['app=sample.app:app_factory'], 'paste.filter_factory': ['middleware = sample.middleware: filter_factory'] })

setup.py

Writing a Swift middleware

Preview middleware

• PUT

○ create a small preview image and store it separate

• GET

○ return preview if QUERY_STRING contains “preview”

• DELETE

○ also delete preview if exists

Useful helpers

• swift.common.utils.split_path

path = /v1/AUTH_account/test/img.jpg ver, acc, cont, obj = split_path(path)

• swift.common.wsgi.make_subrequest

○ add middleware after authentication middleware

• swift.common.swob.wsgify

○ decorator

@wsgify def __call__(self, request): # request.params # request.path_info # request.method # request.environ # request.body return self.app

wsgify?

@wsgify def __call__(self, req): try: (ver, acc, con, obj) = split_path(req.path_info, 4, 4, True) except ValueError: return self.app preview_path= '/%s/%s/%s_%s/%s' % (ver, acc, con, self.suffix, obj) if req.method == 'GET' and request.params.has_key('preview'): req.path_info = preview_path

Return preview

Extract preview

if req.method == 'PUT': preview = create_preview(request.body) if preview: sub = wsgi.make_subrequest( request.environ, path=preview_path, body=preview) sub.get_response(self.app)

Delete preview

if req.method == 'DELETE': sub = wsgi.make_subrequest(req.environ, path=preview_path) sub.get_response(self.app) return self.app

References

3rd party middlewares

• swauth - github.com/gholt/swauth
• swift3 - github.com/stackforge/swift3
• CDMI - github.com/osaddon/cdmi
• Swift informant - github.com/pandemicsyn/swift-informant
• Swift Origin Server - github.com/dpgoetz/sos
• Ceilometer - ceilometer/objectstore/swift_middleware.py

• github.com/enovance/swift-middleware-sample
• docs.openstack.org/developer/swift/

○ middleware.html ○ development_middleware.html ○ development_auth.html ○ associated_projects.html

• legacy.python.org/dev/peps/pep-3333

Something to read

THANK YOU!

christian@enovance.com | @cschwede_de | OpenStack Juno Summit | May 2014, Atlanta