Using AWS to Build a Large Scale Dockerized Microservices Architecture Dr. Oliver Wahlen moovel Group GmbH Frankfurt, 30. Juni 2016
The moovel Group GmbH Our vision is an ecosystem that • simplifies mobility now • solves the mobility challenges of the future
Mobility as a Service Urban areas will become more crowded: • Today 54% of people live in cities In 2050 there will be 66% • More people on less space • Driving and parking will become more regulated • High demand for new transportation concepts Autonomous driving will become reality: • Taxi and car sharing business will change completely • Cost efficiency similar to public transport but faster • Owning a car becomes less relevant The future will be Mobility as a Service
moovel Technology Entwicklung 3: Autonomes Fahren We are building apps, widgets and services that integrate existing transportation providers Benefits for the user: • Only one registration: no need to remember passwords or provide payment data several times • Transparency on mobility alternatives • Intermodal routing Benefits for the partners: • Get new users from other mobility providers Wir verändern die Art, • Get new users from different geographic regions wie wir uns fortbewegen.
Design Goals of Our System Architecture Entwicklung 3: Autonomes Fahren • Scalability up to millions of users • Future prove: Easy to modify components and update technologies • Usable on global scale • Minimal operational effort • Teams can develop independently Wir verändern die Art, wie wir uns fortbewegen.
Avoid Vendor Lock-In vs. Reduced Operational Effort Entwicklung 3: Autonomes Fahren Services currently used by moovel Wir verändern die Art, wie wir uns fortbewegen.
What and Why Docker? Entwicklung 3: Autonomes Fahren Docker Containers • Provide application(s) with a virtual operating system • Isolate applications from the host OS and other docker containers (e.g. Files, Network) • Are self contained run on any machine without other installs • Are simple to build images reference a base image Docker Registry • Stores docker images produced by the build system • Provides access control and maintenance of tags • Dockerhub (Docker Inc.) vs. Amazon EC2 Container Registry
What and Why EC2 Container Service (ECS)? Entwicklung 3: Autonomes Fahren ECS is a highly scalable, fast, container management service that makes it easy to run, stop, and manage Docker containers on a cluster of Amazon EC2 instances. – AWS documentation Technology similar to Kubernetes on Google Container Engine but tailored for AWS and fully managed Terminology • ECS cluster: Group of EC2 instances that allow docker execution • ECS container instance: EC2 instance with ECS container agent • ECS task: The running docker container on an instance • ECS task definition: docker image, CPU, mem, port requirements, … • ECS service: number of parallel tasks, scalability, health checks, …
What and Why Microservices? Entwicklung 3: Autonomes Fahren • Services independently deployable Continuous Deployment • Scalable on service level Specific to service needs: CPU, memory • Services have firm boundary EC2 Instance allows for: different languages per service Web Service different teams per service group evolution of logic and technology Docker Task = ECS Task ECS Cluster http://martinfowler.com/articles/microservices.html
Conway’s Law And Microservices: A Good Fit Entwicklung 3: Autonomes Fahren Organizations which design systems [...] are constrained to Search Book User Pay Squad Squad Squad Squad produce designs which are copies of the communication structures Product Owner of these organizations – Melvin Conway 1967 App-Developers Backend-Devs moovel product dev is organized in squads (feature teams): • with a long term business vision • ability to build and modularize mobile apps / componets • build their own set of microservices • choose the technology that best fits their needs
Continuous Integration with Docker and AWS Entwicklung 3: Autonomes Fahren Build Path dockerfile.tgz code push to triggers Developer Version Control SaaS Build System S3 Bucket Deployment triggers: develop branch: automatic master branch: manually Deployment Path start update docker image update docker push to Elastic Container SaaS Build System Dockerhub Cloudformation Services: dev/prod
moovel System Architecture Entwicklung 3: Autonomes Fahren Client App moovel VPC External ELB • TLS termination Consul • service discovery Kong API Gateway • client load balancing • reverse proxying • service configuration • LUA plugins ELK Stack Kinesis logging synchronous REST call asynchronously publish changes SQS SNS Queue Topic ECS microservice
Limitations of Existing Solution Entwicklung 3: Autonomes Fahren • Difficult to realize dynamic port allocation Currently each service has a fixed port • Two levels of scaling introduce additional complexity: Cluster nodes = No. of EC2 instances ECS Tasks = No. of running Docker containers for a service had to introduced a custom metric taskless instance count • Service discovery with Consul: Automates node and service registry Provides hierarchical key value store with ACL for configuration Consul is a very critical component Must be operated and maintained • Kong API gateway: Extremely flexible and powerful nginx based reverse proxy Requires cassandra DB Must be operated and maintained • Elasticsearch Logstash Kibana (ELK): Do not use it for long term data storage!
Check-up of Design Goals Entwicklung 3: Autonomes Fahren Scalability moovel runs 80 services on 21 ECS clusters (dev+prod) Future prove Exchanging microservice technology is easy We have microservices in Java, Node.js, Python, Kotlin, … Global Scale The world is currently operated from eu-west-1 Pub/Sub is good starting point for multi region deployment Minimal Operational Effort Technology is operated by 10 squads Currently no DevOps team needed: squads build it and run Deployments are easy, frequently done and without downtime Teams can develop independently Technology is operated by 10 squads Microservices, REST and Pub/Sub are a good basis UI decomposition in app is difficult: modularization ongoing
Thank You
Recommend
More recommend
