Using AWS to Build a Large Scale Dockerized Microservices - - PowerPoint PPT Presentation

Using AWS to Build a Large Scale Dockerized Microservices Architecture

• Dr. Oliver Wahlen

moovel Group GmbH Frankfurt, 30. Juni 2016

The moovel Group GmbH

Our vision is an ecosystem that

• simplifies mobility now
• solves the mobility challenges of the future

Mobility as a Service

Urban areas will become more crowded:

• Today 54% of people live in cities  In 2050 there will be 66%
• More people on less space
• Driving and parking will become more regulated
• High demand for new transportation concepts

Autonomous driving will become reality:

• Taxi and car sharing business will change completely
• Cost efficiency similar to public transport but faster
• Owning a car becomes less relevant

The future will be Mobility as a Service

Entwicklung 3: Autonomes Fahren

Wir verändern die Art, wie wir uns fortbewegen.

We are building apps, widgets and services that integrate existing transportation providers Benefits for the user:

• Only one registration: no need to remember passwords
• r provide payment data several times
• Transparency on mobility alternatives
• Intermodal routing

Benefits for the partners:

• Get new users from other mobility providers
• Get new users from different geographic regions

moovel Technology

Entwicklung 3: Autonomes Fahren

Wir verändern die Art, wie wir uns fortbewegen.

• Scalability up to millions of users
• Future prove: Easy to modify components and update technologies
• Usable on global scale
• Minimal operational effort
• Teams can develop independently

Design Goals of Our System Architecture

Entwicklung 3: Autonomes Fahren

Wir verändern die Art, wie wir uns fortbewegen.

Avoid Vendor Lock-In vs. Reduced Operational Effort

Services currently used by moovel

Entwicklung 3: Autonomes Fahren What and Why Docker? Docker Containers

• Provide application(s) with a virtual operating system
• Isolate applications from the host OS and other docker containers

(e.g. Files, Network)

• Are self contained  run on any machine without other installs
• Are simple to build  images reference a base image

Docker Registry

• Stores docker images produced by the build system
• Provides access control and maintenance of tags
• Dockerhub (Docker Inc.) vs. Amazon EC2 Container Registry

Entwicklung 3: Autonomes Fahren What and Why EC2 Container Service (ECS)? ECS is a highly scalable, fast, container management service that makes it easy to run, stop, and manage Docker containers on a cluster of Amazon EC2 instances. – AWS documentation Technology similar to Kubernetes on Google Container Engine but tailored for AWS and fully managed Terminology

• ECS cluster: Group of EC2 instances that allow docker execution
• ECS container instance: EC2 instance with ECS container agent
• ECS task: The running docker container on an instance
• ECS task definition: docker image, CPU, mem, port requirements, …
• ECS service: number of parallel tasks, scalability, health checks, …

Entwicklung 3: Autonomes Fahren What and Why Microservices?

http://martinfowler.com/articles/microservices.html

• Services independently deployable
• Continuous Deployment
• Scalable on service level
• Specific to service needs:

CPU, memory

• Services have firm boundary

allows for:

• different languages per service
• different teams per service group
• evolution of logic and technology

ECS Cluster EC2 Instance Docker Task = ECS Task Web Service

Entwicklung 3: Autonomes Fahren Conway’s Law And Microservices: A Good Fit Organizations which design systems [...] are constrained to produce designs which are copies

• f the communication structures
• f these organizations

– Melvin Conway 1967

Product Owner App-Developers Backend-Devs Search Squad Book Squad User Squad Pay Squad

moovel product dev is organized in squads (feature teams):

• with a long term business vision
• ability to build and modularize mobile apps / componets
• build their own set of microservices
• choose the technology that best fits their needs

Entwicklung 3: Autonomes Fahren Continuous Integration with Docker and AWS

Developer

code push to

SaaS Build System Version Control

triggers dockerfile.tgz

S3 Bucket Build Path SaaS Build System Dockerhub

Deployment triggers: develop branch: automatic master branch: manually

Deployment Path

docker image push to start update

Cloudformation Elastic Container Services: dev/prod

update docker

Entwicklung 3: Autonomes Fahren moovel System Architecture

moovel VPC Client App External ELB

• TLS termination

Kong API Gateway

• reverse proxying
• LUA plugins

Consul

• service discovery
• client load balancing
• service configuration

synchronous REST call

ECS microservice SNS Topic SQS Queue

asynchronously publish changes logging

Kinesis ELK Stack

Entwicklung 3: Autonomes Fahren Limitations of Existing Solution

• Difficult to realize dynamic port allocation
• Currently each service has a fixed port
• Two levels of scaling introduce additional complexity:
• Cluster nodes = No. of EC2 instances
• ECS Tasks = No. of running Docker containers for a service
• had to introduced a custom metric taskless instance count
• Service discovery with Consul:
• Automates node and service registry
• Provides hierarchical key value store with ACL for configuration
• Consul is a very critical component
• Must be operated and maintained
• Kong API gateway:
• Extremely flexible and powerful nginx based reverse proxy
• Requires cassandra DB
• Must be operated and maintained
• Elasticsearch Logstash Kibana (ELK):
• Do not use it for long term data storage!

Entwicklung 3: Autonomes Fahren Check-up of Design Goals Scalability  moovel runs 80 services on 21 ECS clusters (dev+prod) Future prove  Exchanging microservice technology is easy  We have microservices in Java, Node.js, Python, Kotlin, … Global Scale  The world is currently operated from eu-west-1  Pub/Sub is good starting point for multi region deployment Minimal Operational Effort  Technology is operated by 10 squads  Currently no DevOps team needed: squads build it and run  Deployments are easy, frequently done and without downtime Teams can develop independently  Technology is operated by 10 squads  Microservices, REST and Pub/Sub are a good basis  UI decomposition in app is difficult: modularization ongoing

Thank You