Decompose that WAR! A pattern language for microservices Chris - - PowerPoint PPT Presentation

@crichardson

Decompose that WAR! A pattern language for microservices

Chris Richardson

Author of POJOs in Action Founder of the original CloudFoundry.com @crichardson chris@chrisrichardson.net http://plainoldobjects.com http://microservices.io

@crichardson

Presentation goal

Why patterns and pattern languages? A pattern language for microservices

@crichardson

About Chris

@crichardson

About Chris

Founder of a startup that’s creating a platform for developing event-driven microservices

@crichardson

For more information

https://github.com/cer/event-sourcing-examples http://microservices.io http://plainoldobjects.com/ https://twitter.com/crichardson

@crichardson

Agenda

Why a pattern language for microservices? Core patterns Deployment patterns Communication patterns

@crichardson

In 1986…

http://en.wikipedia.org/wiki/Fred_Brooks

@crichardson

Yet almost 30 years later developers are still passionately arguing over “silver bullets”

@crichardson

Suck/Rock Dichotomy

Spring vs. Java EE JavaScript vs. Java Functional programming vs. Object-oriented

http://nealford.com/memeagora/2009/08/05/suck-rock-dichotomy.html

Containers vs. Virtual Machines

@crichardson

Gartner Hype Cycle

http://upload.wikimedia.org/wikipedia/commons/b/bf/Hype-Cycle-General.png

It’s awesome It’s not awesome Trade-offs understood

@crichardson

How we make decisions

Decide using emotions Rationalize with

• ur intellect

http://en.wikipedia.org/wiki/Mahout

@crichardson

We need a better way to discuss and think about technology

@crichardson

What’s a pattern?

Reusable solution to a problem

• ccurring

in a particular context

@crichardson

The structure of a pattern = Great framework for discussing and thinking about technology

@crichardson

The structure of a pattern

Resulting context

aka the situation

Name Context Problem

Related patterns

(conflicting) issues etc to address

Forces Solution

@crichardson

Resulting context

Benefits Drawbacks Issues to resolve

@crichardson

Related patterns

Alternative solutions Solutions to problems introduced by this pattern

Pattern language

A collection of related patterns that solve problems in a particular domain Relationships Pattern A results in a context that has a problem solved by Pattern B Patterns A and B solve the same problem Pattern A is a specialization of pattern B

http://en.wikipedia.org/wiki/A_Pattern_Language

Access to Water Promenade Local townhall Intimacy gradient Light on two sides

@crichardson

Meta-pattern

Problem: How to talk/reason about technology? Solution: Use the pattern format Benefit: More objective Drawback: Less exciting Context: Emotional software development culture Related patterns: It’s awesome!

@crichardson

Monolithic architecture Microservice architecture API gateway Client-side discovery Server-side discovery Service registry Self registration 3rd party registration Multiple Services per host Single Service per Host Service-per- Container Deployment Discovery Core Communication Service-per-VM Partitioning Messaging Remote Procedure Invocation Style Motivating Pattern Solution Pattern Solution A Solution B General Specific

Work in progress

@crichardson

Agenda

Why a pattern language for microservices? Core patterns Deployment patterns Communication patterns

@crichardson

@crichardson

Let’s imagine you are building an online store

Browser/ Client SQL Database

Review Service Product Info Service

Recommendation Service

StoreFrontUI Order Service HTML REST/JSON

@crichardson

Problem: what’s the deployment architecture?

@crichardson

Forces

There is a team of developers that must be productive The application must be easy to understand and modify Do continuous deployment Run multiple instances for scalability and availability Use emerging technologies (frameworks, programming languages, etc)

@crichardson

Tomcat

Pattern: Monolithic architecture

Browser/ Client

WAR/EAR

MySQL Database

Review Service Product Info Service

Recommendation Service

StoreFrontUI Order Service HTML REST/JSON

develop test deploy

Simple to

scale

@crichardson

Examples everywhere

@crichardson

But when the application is large …

@crichardson

Intimidates developers

@crichardson

Obstacle to frequent deployments

Need to redeploy everything to change one component Interrupts long running background (e.g. Quartz) jobs Increases risk of failure

Fear of change

Updates will happen less often - really long QA cycles e.g. Makes A/B testing UI really difficult

Eggs in

• ne basket

@crichardson

Overloads your IDE and container

Slows down development

@crichardson

Lots of coordination and communication required

Obstacle to scaling development

I want to update the UI But the backend is not working yet!

@crichardson

Requires long-term commitment to a technology stack

@crichardson

Pattern: Microservice architecture

@crichardson

Apply functional decomposition

X axis

• horizontal duplication

Z a x i s

• d

a t a p a r t i t i

• n

i n g Y axis - functional decomposition S c a l e b y s p l i t t i n g s i m i l a r t h i n g s Scale by splitting different things

@crichardson

Product Info

Microservice architecture

Product Info Service Recommendation Service Review Service Order Service Browse Products UI Checkout UI Order management UI Account management UI

Apply X-axis and Z-axis scaling to each service independently

@crichardson

Examples

http://highscalability.com/amazon-architecture http://techblog.netflix.com/ http://www.addsimplicity.com/downloads/ eBaySDForum2006-11-29.pdf http://queue.acm.org/detail.cfm?id=1394128 ~600 services 100-150 services to build a page

@crichardson

Benefits

Smaller, simpler apps Easier to understand and develop Less jar/classpath hell - who needs OSGI? Faster to build and deploy Scales development: develop, deploy and scale each service independently Improves fault isolation Eliminates long-term commitment to a single technology stack System level architecture vs. service level architecture Easily and safely experiment with new technologies

@crichardson

Drawbacks

Complexity of developing a distributed system Implementing inter-process communication Handling partial failures Implementing business transactions that span multiple databases (without 2PC) Complexity of testing a distributed system Complexity of deploying and operating a distributed system Managing the development and deployment of features that span multiple services

Fortunately solutions exists

@crichardson

The benefits typically

• utweigh the drawbacks

for large, complex applications

@crichardson

Issues to address

How to deploy the services? How do the services communicate? How do clients of the application communicate with the services? How to partition the system into services? ….

@crichardson

Agenda

Why a pattern language for microservices? Core patterns Deployment patterns Communication patterns

@crichardson

We have applied the microservices pattern: How to deploy the services?

@crichardson

Forces

Services are written using a variety of languages, frameworks, and framework versions Each service consists of multiple service instances for throughput and availability Building and deploying a service must be fast Service must be deployed and scaled independently Service instances need to be isolated Resources consumed by a service must be constrained Deployment must be cost-effective

@crichardson

@crichardson

Pattern: Multiple service instances per host

Host (Physical or VM) Service-A Instance-1 Service-B Instance-2 Service-C Instance-2 Process WAR OSGI bundle

Benefits and drawbacks

Benefits Efficient resource utilization Fast deployment Drawbacks Poor/Terrible isolation Poor visibility (with WAR/OSGI deployment) Difficult to limit resource utilization Risk of dependency version conflicts Poor encapsulation of implementation technology

@crichardson

Pattern: Service instance per host

@crichardson

Pattern: Service per VM host

Service VM image VM Service VM Service VM Service

packaged as deployed as

@crichardson

Example

http://techblog.netflix.com/ ~600 services

packer.io is a great tool

Benefits and drawbacks

Benefits Great isolation Great manageability VM encapsulates implementation technology Leverage AWS infrastructure for Autoscaling/Load balancing
 Drawbacks Less efficient resource utilization Slow deployment

@crichardson

VM VM

Pattern: Service per Container host

Service Container image Container Service Container Service Container Service

packaged as deployed as

@crichardson

Examples

Benefits and drawbacks

Benefits Great isolation Great manageability Container encapsulates implementation technology Efficient resource utilization Fast deployment Drawbacks Immature infrastructure for deploying containers

@crichardson

Agenda

Why a pattern language for microservices? Core patterns Deployment patterns Communication patterns

@crichardson

Communication issues

System Client Service A Service B Service C

The System

How do clients of the system interact with the services? How do services within the system interact?

@crichardson

@crichardson

How do clients of the system interact with the services?

@crichardson

Forces

Mismatch between the fine-grained microservices and needs

• f the clients

Different clients need different data LAN vs. WAN vs. Mobile network performance The number of service instances and their locations (host +port) is changes dynamically Partitioning into services can change over time and should be hidden from clients

@crichardson

Directly connecting the front-end to the backend

Model View Controller

Product Info service Recommendation Service Review service

REST REST Thrift

Model View Controller

Browser/Native App Traditional server-side web application

Chatty API Web unfriendly protocols What’s the host/port??

@crichardson

Pattern: API gateway

Model View Controller

Product Info service Recommendation Service Review service

REST REST Thrift API Gateway

Model View Controller

Browser/Native App

Single entry point Client specific APIs Protocol translation

Traditional server-side web application

@crichardson

Example: Netflix API

http://techblog.netflix.com/2013/01/optimizing-netflix-api.html

Device specific end points

@crichardson

Benefits of the API gateway

Insulates the clients from the partitioning Insulates the clients from the problem of discovery Provides the optimal API for each client Reduces the number of requests/roundtrips Simplifies the client by moving logic for calling multiple services from the client to API gateway Gateway translates between web-unfriendly protocols and HTTP/WebSockets

@crichardson

Drawbacks of the API gateway

Increased complexity - the API gateway is yet another highly available component that must be developed, deployed and managed Increased response time due to the additional network hop through the API gateway

@crichardson

Benefits and drawbacks of messaging

Benefits Decouples client from services Message broker buffers messages Supports a variety of communication patterns
 Drawbacks Additional complexity of message broker Request/reply-style communication is more complex Client needs to discover location of message broker

Benefits and drawbacks of RPC

Benefits Simple and familiar Request/reply is easy No intermediate broker Drawbacks Only supports request/ reply Service must be available Client needs to discover locations of service instances

@crichardson

@crichardson

The problem of discovery

Service Client Service Instance A Service Instance B Service Instance C

?

10.4.3.1:8756 10.4.3.99:4545 10.4.3.20:333

Client or API gateway Dynamically changing Service Instance A Service Instance B Service Instance C

10.4.3.1:8756 10.4.3.99:4545 10.4.3.20:333

How to load balance? Dynamically assigned

@crichardson

Pattern: Client-side discovery

Service Client

Registry- aware HTTP Client

Service Registry Service Instance A Service Instance B Service Instance C

10.4.3.1:8756 10.4.3.99:4545 10.4.3.20:333

register query request load balance

@crichardson

Example: Netflix Eureka and Ribbon

https://github.com/Netflix/eureka/wiki/Eureka-at-a-glance https://github.com/Netflix/ribbon

Benefits and drawbacks

Benefits Flexible, application-specific load balancing Fewer network hops and moving parts compared to Server-side discovery
 Drawbacks Couples the client to the Service Registry Need implement client-side discovery and load balancing logic in multiple languages/ frameworks Service Registry is yet another moving part to setup and operate - highly available

@crichardson

Pattern: Server-side discovery

Service Client

Router

Service Registry Service Instance A Service Instance B Service Instance C

10.4.3.1:8756 10.4.3.99:4545 10.4.3.20:333

register query request

request load balance

@crichardson

Example

http://docs.aws.amazon.com/ElasticLoadBalancing/latest/ DeveloperGuide/vpc-loadbalancer-types.html

Internal ELB Internal ELB Public ELB

Benefits and drawbacks

Benefits Simpler client code Built-in to some cloud/ container environments, e.g. AWS ELB, Kubernetes, Marathon Drawbacks Limited to (generic) load balancing algorithms provided by router More network hops Router and Service Registry is yet another moving part to setup and operate - highly available

@crichardson

The problem of registration

Service Registry Service Instance A Service Instance B Service Instance C

10.4.3.1:8756 10.4.3.99:4545 10.4.3.20:333

register

How does registration happen?

@crichardson

@crichardson

Forces

Service instances must be registered with the service registry on startup unregistered on shutdown Service instances that crash must be unregistered from the service registry Service instances that are running but incapable of handling requests must be unregistered from the service registry

@crichardson

Pattern: Self registration

Service Registry Service Instance A

10.4.3.1:8756

register heart-beat unregister

@crichardson

Examples

Netflix Eureka Java client API for registering/unregistering with Eureka server Specify state: STARTING, UP , DOWN, OUT_OF_SERVICE Register a periodically invoked health check callback Zookeeper-based Java applications Service = znode Service instance = ephemeral child of znode

Benefits and drawbacks

Benefits Simple - no extra components Service can implement a more sophisticated state model Drawbacks Couples the service to service registry Must implement registration logic in multiple languages/ frameworks Service might lack the self- awareness to unregister itself

@crichardson

Pattern: 3rd party registration

Service Registry Service Instance A

10.4.3.1:8756

register heart-beat unregister

Registrar

healthcheck

@crichardson

Examples

AWS Autoscaling groups Automatically register/unregister EC2 instance with ELB Registrator Registers and unregisters Docker containers with service registry Kubernetes/Marathon Automatically register/unregister services Netflix Eureka Prana Sidecar application for non-Java clients Registers/unregisters service with Eureka Performs health checks

Benefits and drawbacks

Benefits Simpler service Registrar can perform health checks Some cloud/container environments provide a Registrar Drawbacks Registrar is yet another component that must be setup/operated. Must be highly available

@crichardson

Summary: Patterns and pattern languages are a great way to …

Think about technology Discuss technology Apply technology

@crichardson

Summary: The Microservices pattern language is a great way to …

Think about microservices Discuss microservices Apply microservices (or not)

@crichardson

@crichardson chris@chrisrichardson.net http://plainoldobjects.com http://microservices.io