Microservices and Serveless Computing Corso di Sistemi Distribuiti - - PDF document

Corso di Sistemi Distribuiti e Cloud Computing A.A. 2019/20 Valeria Cardellini Laurea Magistrale in Ingegneria Informatica

Microservices and Serveless Computing

Macroarea di Ingegneria Dipartimento di Ingegneria Civile e Ingegneria Informatica

Microservices

• A “new” emerging architectural style for distributed

applications that structures an application as a collection of loosely coupled services

• Not so new: deriving from SOA

– But with some significant differences

• Address how to build, manage, and evolve

architectures out of small, self-contained units

– Modularization: decompose app into a set of independently deployable services, that are loosely coupled and cooperating and can be rapidly deployed and scaled – Services equipped with dedicated memory persistence tools (e.g., databases and NoSQL data stores)

Valeria Cardellini – SDCC 2019/20 1

Service Oriented Architecture

• Service Oriented Architecture (SOA): paradigma

architetturale per progettare sistemi sw distribuiti lascamente accoppiati

• Definizione OASIS: paradigma per l’organizzazione e

l’utilizzazione di risorse distribuite che possono essere sotto il controllo di domini di proprietà differenti. Fornisce un mezzo uniforme per offrire, scoprire, interagire ed usare le capacità di produrre gli effetti voluti in modo consistente con presupposti e aspettative misurabili

• Proprietà (secondo W3C, http://www.w3.org/TR/ws-arch/)

– Vista logica – Orientamento ai messaggi e alla descrizione – Granularità dei servizi, orientamento alla rete, neutralità della piattaforma

2

Service Oriented Architecture

l Tre entità che interagiscono tra loro

l

Service requestor o consumer: richiede l’esecuzione di un servizio

l

Service provider: implementa il servizio e lo rende disponibile

l

Service registry: offre un servizio di pubblicazione e di ricerca di servizi disponibili

Valeria Cardellini – SDCC 2019/20 3

Web services

• Web services: implementation of SOA
• W3C definition:

– A web service is a software system designed to support interoperable machine-to-machine interaction over a network – It has an interface described in a machine- processable format (specifically WSDL) – Other systems interact with the web service in a manner prescribed by its description using SOAP- messages, typically conveyed using HTTP

Valeria Cardellini – SDCC 2019/20 4

Web services

• More than 60 standards and specifications, most used:

– To describe: WSDL (Web Service Description Language) – To communicate: SOAP (Simple Object Access Protocol) – To register: UDDI (Universal Description, Discovery and Integration) – To define business processes: BPEL (Business Process Execution Language), BPMN (Business Process Model and Notation)

Valeria Cardellini – SDCC 2019/20 5

– To define SLA: WSLA

• Many technologies

– Among which ESB (Enterprise Service Bus), integration platform that provides fundamental interaction and communication services for complex applications

SOA vs. microservices

• Heavyweight vs. lightweight technologies
• SOA tends to rely strongly on heavyweight middleware (e.g.,

ESB), while microservices rely on lightweight technologies

• Protocol families

– SOA is often associated with web services protocols

• SOAP, WSDL, and WS-* family of standards

– Microservices typically rely on REST and HTTP

• Views

– SOA mostly viewed as integration solution – Microservices are typically applied to build individual software applications

Valeria Cardellini – SDCC 2019/20 6

Microservices: benefits

• Increase software agility

– Microservice: independent unit of development, deployment, operations, versioning, and scaling – Exploit container-based virtualization: popular approach is “microservice instance per container”

• Faster delivery
• Improved scalability
• Greater autonomy

Valeria Cardellini – SDCC 2019/20 7

Microservices and containers

• Microservices as ideal complementation of

container-based virtualization

– Package each service as a container image and deploy each service instance as a container – Manage each container at runtime (scaling and or migrating it)

• Pros:

– Service instance scaling out/in by changing the number

• f container instances

– Service instance isolation – Resource limits on service instance – Build and start rapidly

• Cons:

– Need container orchestration to manage multi-container app

Valeria Cardellini – SDCC 2019/20 8

How to decompose?

• How to decompose the application into microservices?
• Mostly an art, no winner strategy but rather a number
• f strategies:

– Decompose by business capability and define services corresponding to business capabilities

• E.g., Order Management is responsible for orders

– Decompose by domain-driven design (DDD) subdomain

• E.g., Order Management, Inventory, Product Catalogue, Delivery

– Decompose by use case and define services that are responsible for particular actions

• E.g., Shipping Service is responsible for shipping complete
• rders

– Decompose by nouns or resources and define a service that is responsible for all operations on entities/resources of a given type

• E.g., Account Service is responsible for managing user accounts

Valeria Cardellini – SDCC 2019/20 9

Microservices and scalability

• How to achieve service scalability?

– Use multiple instances of same service and load balance request across multiple instances

• How to improve service scalability?

– State is complex to manage and scale – Stateless services scale better and faster than stateful services

• We also need service discovery

– Service instances have dynamically assigned network locations and their set changes dynamically because of autoscaling, failures, and upgrades: we need a service discovery

Valeria Cardellini – SDCC 2019/20 10

Stateless vs. stateful service

Source: “Cloud Computing Patterns”, https://bit.ly/2yyxHkg

Valeria Cardellini – SDCC 2019/20

• Stateful service: multiple instances of a scaled-out

service need to synchronize their internal state to provide a unified behavior

• Issue: how can a stateful service that is scaled-out

maintain a synchronized internal state?

11

Stateless vs. stateful service

Source: “Cloud Computing Patterns”, https://bit.ly/2SekKo6

Valeria Cardellini – SDCC 2019/20

• Stateless service: state is handled external of service

to ease its scaling out and to make the application more tolerant to service failures

12

Integration

• Let’s consider two issues related to the

integration of microservices

– Synchronous vs. asynchronous communication – Orchestration vs. choreography

Valeria Cardellini – SDCC 2019/20 13

Synchronous vs. asynchronous

• Should communication be synchronous or

asynchronous?

– Synchronous: request/response style of communication – Asynchronous: event-driven style of communication

• Synchronous communication

– Synchronous request/response-based communication mechanisms, such as HTTP-based REST or gRPC

• Asynchronous communication

– Asynchronous, message-based communication mechanisms such as pub-sub systems, message queues and related protocols – Interaction style can be one-to-one or one-to-many

Valeria Cardellini – SDCC 2019/20 14

Synchronous vs. asynchronous

• Synchronous communication reduces availability

Valeria Cardellini – SDCC 2019/20 15

Orchestration and choreography

• Microservices can interact among them following two

patterns:

– Orchestration – Choreography

• Orchestration: centralized approach

– A single centralized process (orchestrator or message broker) coordinates the interaction among different services – The orchestrator is responsible for invoking and combining the services, which can be unaware of the composition

Valeria Cardellini – SDCC 2019/20 16

Orchestration and choreography

• Choreography: decentralized approach

– A global description of the participating services, which is defined by exchange of messages, rules of interaction and agreements between two or more endpoints – Services can exchange messages directly

Valeria Cardellini – SDCC 2019/20 17

Example: orchestration and choreography

• Example: customer creation

– The process for creating a new customer

Valeria Cardellini – SDCC 2019/20 18

From: S. Newman, “Building Microservices”, O’Really, 2015.

Example: orchestration and choreography

Orchestration

19

Choreography

Valeria Cardellini – SDCC 2019/20

Orchestration vs choreography

• Orchestration is simpler and more popular, but

– Single point of failure and performance bottleneck – Tight coupling – Higher network traffic and latency

• Choreography has lower coupling, less operational

complexity and increased flexibility and ease of changing, but

– Services need to know about each other’s locations – Extra work to monitor and track services – Implementing mechanisms, such as guaranteed delivery, is more challenging

Valeria Cardellini – SDCC 2019/20 20

Design patterns for microservices

• Let’s examine some design patterns

– Circuit breaker – Database per service – Saga – Log aggregation – Distributed request tracing

Valeria Cardellini – SDCC 2019/20 21

Patterns: Circuit breaker

• Problem: How to prevent a network or service

failure from cascading to other services?

• Solution: A service client invokes a remote

service via a proxy that functions in a similar fashion to an electrical circuit breaker

– When the number of consecutive failures crosses a threshold, the circuit breaker trips, and for the duration of a timeout period all attempts to invoke the remote service will fail immediately – After the timeout expires the circuit breaker allows a limited number of test requests to pass through. If those requests succeed the circuit breaker resumes normal operation. Otherwise, if there is a failure the timeout period begins again

Valeria Cardellini – SDCC 2019/20 22

https://microservices.io/patterns/reliability/circuit-breaker.html

Patterns: Database per service

• Problem: which database

architecture for microservice app?

• Solution: keep each

microservice’s persistent data private to that service and accessible only via its API. Service transactions only involve its database

Valeria Cardellini – SDCC 2019/20 23

• Pros:

– Helps ensure that services are loosely coupled – Each service can use the type of database that is best suited to its needs (e.g., SQL or NoSQL)

• Cons

– More complex to implement transactions that span multiple services – Complexity of managing multiple databases -> SAGA pattern

Patterns: Saga

• Problem: each service has its own database, however some

transactions span multiple service: how to ensure data consistency across services?

• Solution: implement each transaction that spans multiple

services as a saga. A saga is a sequence of local transactions. Each local transaction updates the database and publishes a message or event to trigger the next local transaction. If a local transaction fails, then the saga executes a series of compensating transactions that undo the changes that were made by the preceding local transactions

• 2 ways of coordination sagas:

– Choreography: each local transaction publishes events that trigger local transactions in other services – Orchestration: an orchestrator tells the participants what local transactions to execute

Valeria Cardellini – SDCC 2019/20 24

https://microservices.io/patterns/data/saga.html

Patterns: Saga

Valeria Cardellini – SDCC 2019/20 25

Choreography Orchestration

Patterns: Saga

• Let’s consider an example using Saga with
• rchestration
• The example also uses the event sourcing pattern

– Persist an aggregate as a sequence of domain events that represent state changes

http://microservices.io/patterns/data/event-sourcing.html

Valeria Cardellini – SDCC 2019/20 26

Source: MSc thesis by Andrea Cifola, see Microservice_SAGAexample.pdf

Monitoring microservices

• Service distribution (at large scale) → more difficult

to monitor microservice app

– Performance and latency – Transaction monitoring – Root cause analysis – Service dependency analysis – Distributed context propagation

• Let’s examine two patterns for monitoring

microservices

– Log aggregation – Distributed request tracing

Valeria Cardellini – SDCC 2019/20 27

Patterns: Log aggregation

• Problem: How to understand the behavior of an

application and troubleshoot problems?

• Solution: Use a centralized logging service that

aggregates logs from each service instance. The users can search and analyze the logs. They can configure alerts that are triggered when certain messages appear in the logs

• Cons:

– Centralized – Handling a large volume of logs requires substantial infrastructure

Valeria Cardellini – SDCC 2019/20 28

https://microservices.io/patterns/observability/application-logging.html

Patterns: Distributed tracing

• Problem: How to understand the behavior of an

application and troubleshoot problems?

• Solution: Instrument services with code that

– Assigns each external request a unique external request id – Passes the external request id to all services that are involved in handling the request – Includes the external request id in all log messages – Records information (e.g. start time, end time) about the requests and operations performed when handling a external request in a centralized service

• Con: aggregating and storing traces can require

significant infrastructure

Valeria Cardellini – SDCC 2019/20 29

https://microservices.io/patterns/observability/distributed-tracing.html

Monitoring microservices: tools

• Tools for distributed tracing

– Most existing microservice frameworks have application-layer tracing capability built in – Dapper: Google’s production distributed systems tracing infrastructure – Jaeger https://www.jaegertracing.io – Zipkin https://zipkin.io/ – OpenCensus https://opencensus.io/ – OpenTracing https://opentracing.io/

Valeria Cardellini – SDCC 2019/20 30

Some large-scale examples

• Netflix and Twitter: 500+ microservices (in 2015)

Valeria Cardellini – SDCC 2019/20 31

Example 1: Sock shop

• An online shop https://microservices-demo.github.io/

Valeria Cardellini – SDCC 2019/20 32

Example 1: Sock shop

• Built using SpringBoot, Go and Node.js and

packaged in Docker containers

• Cross-platform: deploy to all orchestrators
• All services communicate using REST over

HTTP

• Polyglot data stores

Valeria Cardellini – SDCC 2019/20 33

Example 2: Hipster shop

• Another online store

https://github.com/GoogleCloudPlatform/microservices-demo

– Composed of many microservices written in different languages that talk to each other over gRPC – Used by Google to demonstrate use of Kubernetes/GKE, Istio, Stackdriver, gRPC and OpenCensus

Valeria Cardellini – SDCC 2019/20 34

In both examples: polyglot programming

• Microservices often built in different programming

languages

• Programming languages are silos, but in the last 15

years renaissance in programming language diversity

– Polyglot programmers

• How to realize a polyglot application whose

microservices are written in different programming languages?

• Solution 1: REST and JSON (as message

interchange format): see example 1

• Solution 2: gRPC and protocol buffers (as IDL and

message interchange format): see example 2

Valeria Cardellini – SDCC 2019/20 35

gRPC

• Open-source high performance RPC framework for

microservice applications

Valeria Cardellini – SDCC 2019/20 36

– Can run in any environment – Work across languages and platforms

• Automatically generate client and

server stubs in a variety of languages and platforms

– Can connect services in and across data centers with pluggable support for load balancing, tracing, health checking and authentication – Bi-directional streaming and integrated authentication

Introduction to gRPC: https://www.youtube.com/watch?v=yfZB2_rT_Pc Introduction to Protocol Buffers in gRPC https://www.youtube.com/watch?v=yfZB2_rT_Pc

Microservice technologies timeline

Valeria Cardellini – SDCC 2019/20 37

From "Microservices: The Journey So Far and Challenges Ahead”.

Generations: at the beginning

Valeria Cardellini – SDCC 2019/20 38

• 4 generations of microservice architectures
• 1st generation based on:
• Container-based virtualization
• Service discovery (e.g., Eureka, ZooKeeper and etcd)
• Let services communicate with each other without explicitly

referring to their network locations

• Eureka: REST based service developed by Netflix and is

primarily used in the AWS cloud for locating services for the purpose of load balancing and failover of middle-tier servers

• etcd: distributed reliable key-value store for the most critical

data of a distributed system (e.g., used by Kubernetes as primary datastore) https://etcd.io

• Monitoring (e.g., Graphite, InfluxDB and Prometheus)
• Enable runtime monitoring and analysis of the behavior of

microservice resources at different levels of detail

Generations: container orchestration

Valeria Cardellini – SDCC 2019/20 39

• Then, container orchestration
• E.g., Kubernetes, Apache Swarm
• Automate container allocation and management tasks,

abstracting away the underlying physical or virtual infrastructure from service developers

• But application-level failure-handling mechanisms still

implemented in services source code

Generations: service discovery and fault tolerance

Valeria Cardellini – SDCC 2019/20 40

• 2nd generation based on discovery services and

fault-tolerant communication libraries

• Let services communicate more efficiently and reliably

Generations: service discovery and fault tolerance

Valeria Cardellini – SDCC 2019/20 41

• Examples of discovery services and fault-tolerant

communication libraries

• Consul: service discovery https://www.consul.io
• Finagle: fault tolerant, protocol-agnostic RPC

system http://twitter.github.io/finagle

• Hystrix: latency and fault tolerance library designed by Netflix

to isolate points of access to remote systems, services and 3rd party libraries, stop cascading failure and enable resilience in complex distributed systems where failure is inevitable https://github.com/Netflix/Hystrix

Generations: sidecar and service mesh

• 3rd generation based on sidecar (or service proxy or

ambassador) technologies (e.g., Prana and Envoy)

– Encapsulate communication-related features such as service discovery and use of protocol-specific and fault-tolerant communication libraries – Goal: abstract them from service developers, improve sw reusability and provide homogeneous interface

Valeria Cardellini – SDCC 2019/20 42

Generations: serverless

Valeria Cardellini – SDCC 2019/20 43

• 4th generation based on Function as a Service

(FaaS) and serverless computing to further simplify microservice development and delivery

References and resources

• J. Lewis, M. Fowler, “Microservices”,

https://martinfowler.com/articles/microservices.html

• Microservice Guides https://martinfowler.com/microservices/
• Microservice Architecture https://microservices.io
• P. Jamshidi, C. Pahl, N. C. Mendonça, J. Lewis and S. Tilkov,

"Microservices: The Journey So Far and Challenges Ahead,” IEEE Software, 2018. https://ieeexplore.ieee.org/document/8354433

• M. Roberts, “Serverless Architectures”,

https://martinfowler.com/articles/serverless.html

• E. Jonas et al., “Cloud Programming Simplified: A Berkeley View
• n Serverless Computing“, Tech. Rep. No. UCB/EECS-2019-3,
• 2019. https://www2.eecs.berkeley.edu/Pubs/TechRpts/2019/EECS-

2019-3.pdf

Valeria Cardellini – SDCC 2019/20 44