AsyncIO in Production War Stories Micha Wysokiski 17.09.2018 1 | - - PowerPoint PPT Presentation

▶
asyncio in production war stories
SMART_READER_LITE
LIVE PREVIEW

AsyncIO in Production War Stories Micha Wysokiski 17.09.2018 1 | - - PowerPoint PPT Presentation

Feb 05, 2024 35 likes •295 views

AsyncIO in Production War Stories Micha Wysokiski 17.09.2018 1 | AsyncIO in production - War Stories | Micha Wysokiski | PyWaw #78 Akamai is a content delivery network (CDN) and cloud services provider has deployed the most

slide-1
SLIDE 1 | AsyncIO in production - War Stories | Michał Wysokiński | PyWaw #78 1

AsyncIO in Production War Stories

Michał Wysokiński

17.09.2018

slide-2
SLIDE 2 | AsyncIO in production - War Stories | Michał Wysokiński | PyWaw #78 2

Akamai

  • is a content delivery network (CDN)

and cloud services provider

  • has deployed the most pervasive, highly-

distributed CDN with more than 250,000 servers in over 130 countries and within more than 4,000 networks around the world, which are responsible for serving between 10 and 30% of all web traffic

  • protects against web attacks such as

SQL injection, XSS and RFI

  • has 16 offices in EMEA (Europe, Middle

East and Africa)

  • and 1 office in Poland (Kraków)
slide-3
SLIDE 3 | AsyncIO in production - War Stories | Michał Wysokiński | PyWaw #78 3

How does it work

slide-4
SLIDE 4 | AsyncIO in production - War Stories | Michał Wysokiński | PyWaw #78 4

A bit of history

  • September 22nd 2012

asyncore: included batteries don’t fit – submitted to Python ideas

  • December 12th 2012

PEP 3156

  • March 16th 2014 [3.4] (provisional)

AsyncIO module release with Python 3.4 (as provisional API) @asyncio.coroutine/yield from (introduced in 3.3)

  • September 13th 2015 [3.5]

async/await keywords introduced with Python 3.5 asynchronous iteration, asynchronous context managers

  • December 23rd 2016 [3.6] (stable)

asynchronous generators, asynchronous comprehensions

slide-5
SLIDE 5 | AsyncIO in production - War Stories | Michał Wysokiński | PyWaw #78 5

How asyncio code looks like

for exec_uuid in tasks: task_data = await self.task_queries.get_data(exec_uuid) asyncio.ensure_future(self._run_task(exec_uuid)) exec_statuses = await asyncio.gather( *[self._run_task(ip, task_data) for ip in ips] )

slide-6
SLIDE 6 | AsyncIO in production - War Stories | Michał Wysokiński | PyWaw #78 6

How asyncio code looks like – with timeouts

await asyncio.wait_for( ioloop.run_in_executor( self.processing_pool, lambda: plugin.run( ip, **exec_data.args ) ), timeout=getattr(plugin, 'timeout', self.default_plugin_timeout) ) async with timeout(1.5): # async-timeout await inner()

slide-7
SLIDE 7 | AsyncIO in production - War Stories | Michał Wysokiński | PyWaw #78 7

Reasons for its existence

  • It’s useful for handling independent tasks (similar to threads*)
  • Everyone started doing it in their own way
  • Modifying CPython was giving hope of better performance
slide-8
SLIDE 8 | AsyncIO in production - War Stories | Michał Wysokiński | PyWaw #78 8

A story of mixing AsyncIO and threads

slide-9
SLIDE 9 | AsyncIO in production - War Stories | Michał Wysokiński | PyWaw #78 9

We started with a bit of a…

slide-10
SLIDE 10 | AsyncIO in production - War Stories | Michał Wysokiński | PyWaw #78 10

Yyy, what just happened?

Task was destroyed but it is pending! task: <Task pending create() done at run.py:5 wait_for=<Future pending cb=[Task._wakeup()]>> ERROR:asyncio:Task exception future: <Task finished coro=<SSHConnection._run_task() done, exception=CancelledError()> concurrent.futures._base.CancelledError was never retrieved

slide-11
SLIDE 11 | AsyncIO in production - War Stories | Michał Wysokiński | PyWaw #78 11

Dependencies nightmare

  • Tornado (ioloop is a wrapper for asyncio loop)
  • Momoko (async wrapper for psycopg2)
  • uvloop (wrapper for libuv, replacement for asyncio loop)
  • async_test (to get rid of the standard library low level testing code)
slide-12
SLIDE 12 | AsyncIO in production - War Stories | Michał Wysokiński | PyWaw #78 12

Dependencies nightmare

  • Tornado (ioloop is a wrapper for asyncio loop)
  • Momoko (async wrapper for psycopg2) – PRETTY DEAD
  • uvloop (wrapper for libuv, replacement for asyncio loop)

basically rewrites asyncio loop which sometimes causes unexpected results

  • async_test (to get rid of the standard library low level testing code)

– single developer, not super stable (resource allocation), not compatible with uvloop

slide-13
SLIDE 13 | AsyncIO in production - War Stories | Michał Wysokiński | PyWaw #78 13

@asyncio.coroutine/yield from* -> async/await

slide-14
SLIDE 14 | AsyncIO in production - War Stories | Michał Wysokiński | PyWaw #78 14

@tornado.gen.coroutine/yield -> async/await

slide-15
SLIDE 15 | AsyncIO in production - War Stories | Michał Wysokiński | PyWaw #78 15

A story of an asynchronous http client

slide-16
SLIDE 16 | AsyncIO in production - War Stories | Michał Wysokiński | PyWaw #78 16

The good, the bad and the ugly

class A: def get_data(self): with ###.TCPConn(ssl_context, limit=1) as tcp_conn: async with ###.HttpClient(tcp_conn) as http_client: async with http_client.get(self.streamer_url.encoded) as response: content_iterator = response.content.__aiter__() # no aiter() while not self.stop_streaming: async with async_timeout.timeout(60): try: data = await content_iterator.__anext__() # no anext() except StopAsyncIteration: logger.debug(‘Stop aiteration for stream: %s', str(self)) break

slide-17
SLIDE 17 | AsyncIO in production - War Stories | Michał Wysokiński | PyWaw #78 17
slide-18
SLIDE 18 | AsyncIO in production - War Stories | Michał Wysokiński | PyWaw #78 18

A story of an ElasticSearch client

slide-19
SLIDE 19 | AsyncIO in production - War Stories | Michał Wysokiński | PyWaw #78 19

Fixing async bomb

import aiojobs scheduler = await aiojobs.create_scheduler( limit=MAX_TASKS_SIZE, pending_limit=WAITING_DOCS_BUFFER_SIZE, close_timeout=5 ) await scheduler.spawn( es_index_wrapper(document, es_client) )

slide-20
SLIDE 20 | AsyncIO in production - War Stories | Michał Wysokiński | PyWaw #78 20

A story of a really simple microservice that gets

  • ne and only one job done

ips = await dns_resolver.query(edns[domain_info.name], 'A') # with this line we reduced running time from 12 hours to 8 minutes

slide-21
SLIDE 21 | AsyncIO in production - War Stories | Michał Wysokiński | PyWaw #78 21

A story of a group of services communicating only with messages

slide-22
SLIDE 22 | AsyncIO in production - War Stories | Michał Wysokiński | PyWaw #78 22

PROS

  • Performance gain for applications

relying on IO (network, DB)

  • Better resource utilization

(less time spent on communication and synchronization)

  • Being on the technological edge

gives you new ways to solve old problems

  • It makes you follow Python

progress and contribute

CONS

  • Still many missing features or know

issues

  • async iterators have messy indeterministic

cleanups

  • itertools for async is missing
  • very early implementations or complete lack of

modules for interacting with popular services (zookeeper, ElasticSearch, requests, and more)

  • Still young implementation

with many bugs and incompatibilities

  • The community becomes

more and more divided

slide-23
SLIDE 23 | AsyncIO in production - War Stories | Michał Wysokiński | PyWaw #78 23

What projects are best suited for AsyncIO (IMHO)

  • MICROservices
  • Projects with a small list of dependencies
  • Simple http APIs
  • Projects with big load but light processing
  • Projects where threads are not enough
  • Projects where the rest of technology stack is well understood
slide-24
SLIDE 24 | AsyncIO in production - War Stories | Michał Wysokiński | PyWaw #78 24

What projects are not suited for AsyncIO (IMHO)

  • Projects heavily relying on threads
  • Projects with dependencies heavily using threads, unless you know

their implementation really well

  • Projects where processing of a single task takes a lot of time

minutes/hours

  • Projects doing uncommon stuff
slide-25
SLIDE 25 | AsyncIO in production - War Stories | Michał Wysokiński | PyWaw #78 25

Q&A

slide-26
SLIDE 26 | AsyncIO in production - War Stories | Michał Wysokiński | PyWaw #78 26