Scalable Hosting of Web Applications Guillaume Pierre (with Zhou - - PowerPoint PPT Presentation

Scalable Hosting of Web Applications

Guillaume Pierre

(with Zhou Wei, Jiang Dejun, Swaminathan Sivasubramanian, Tobias Groothuyse, Sandjai Bhulai, Chi-Hung Chi and Maarten van Steen)

CANOE and EuroSys Summer School

21 august 2009 http://www.cs.vu.nl/~gpierre/

Scalable Hosting of Web Applications 1 / 33

Advertisement

This school is co-organized by EuroSys

◮ The European Professional Society on Computer Systems ◮ Scope: operating systems, distributed systems, event-based systems,

embedded systems, etc.

◮ Membership: 40 euros (senior), 10 euros (students)

Upcoming activities:

◮ EuroSys VMware Premier Conference Award (application deadline:

August 28th)

◮ EuroSys Shadow PC (application deadline: September 15th) ◮ EuroSys 2010 conference (submission deadline: October 23rd) ◮ Roger Needham PhD award (application deadline: December 12th) ◮ Note: it is not necessary to be a member to participate!

www.eurosys.org

Scalable Hosting of Web Applications 2 / 33

The Problem

1 You build a great Web site, advertise it 2 . . . Scalable Hosting of Web Applications Introduction 3 / 33

The Problem

1 You build a great Web site, advertise it 2 . . .

Performance # of users What we want What we get

Scalable Hosting of Web Applications Introduction 3 / 33

Scalability

“A system is said to be scalable if it can handle the addition of users and resources without suffering a noticeable loss of performance or increase in administrative complexity.”

• B. Clifford Neuman,

“Scale in Distributed Systems”

Scalable Hosting of Web Applications Introduction 4 / 33

A typical Web application

One application server runs application code One database server holds the application state The code can issue any query to the database

◮ SELECT (read queries) ◮ UPDATE, DELETE, INSERT (UDI queries) ◮ Transactions

Database server Application server

HTTP requests SQL queries

Users

Scalable Hosting of Web Applications Introduction 5 / 33

Scaling the application server

The application server contains only the application code

◮ It does not hold state ◮ Different requests can be processed independently

Database server Application servers Users

Scalable Hosting of Web Applications Introduction 6 / 33

Replicating the database server

State is fully replicated across multiple database servers

◮ Read queries can be addressed at any replica ◮ UDIs must be issued at every replica

Application server Database server Users R e a d UDI

Each database server must process 1

N Read Queries + UDIs query load

◮ Increasing N does not help when the UDIs alone saturate the server’s

capacity

Scalable Hosting of Web Applications Introduction 7 / 33

Partially replicate the database

We must send less UDIs to each server

◮ Let’s partition the database ◮ Each server contains a subset of all tables

Users Tables T2, T3 Tables T1, T3 Read(T1) Read(T1,T3) UDI(T1) Table T1

◮ Updates to T1 must be addressed to only 2 servers ◮ We must place tables according to query templates ⋆ We cannot execute a query that joins T1 and T2. . . Scalable Hosting of Web Applications Introduction 8 / 33

Performance of partial database replication

Number of emulated browsers 500 400 300 200 100 2 3 4 5 6 7 8 1 Number of servers GlobeTP Full replication

TPC-W (e-commerce app)

Number of emulated browsers 50 100 150 200 250 300 350 1 2 3 4 5 6 7 8 Number of servers Full replication GlobeTP

RUBBoS (Slashdot-like) Problem: table-level granularity is too coarse

◮ Maximum gain = # of tables ◮ We need a finer granularity: column-level Scalable Hosting of Web Applications Introduction 9 / 33

Table of Contents

1 Introduction 2 Service-Oriented Data Denormalization 3 Resource Provisioning for Web Services 4 Conclusion

Scalable Hosting of Web Applications Service-Oriented Data Denormalization 10 / 33

Position

Position

We must split the application data into a number of independent services

◮ This implies restructuring the data schema at the column granularity

Each data services has its own private data store

◮ It can be accessed through a well-defined interface

This transformation does not improve performance!

◮ But it makes the workload of each service much simpler ◮ It is easier to scale each service independently Scalable Hosting of Web Applications Service-Oriented Data Denormalization 11 / 33

System model (traditional)

Scalable Hosting of Web Applications Service-Oriented Data Denormalization 12 / 33

System model (denormalized)

Scalable Hosting of Web Applications Service-Oriented Data Denormalization 13 / 33

Can we split data arbitrarily?

Answer: of course not!

◮ Queries and transactions access multiple data rows simultaneously ◮ We must make sure that the application queries can still execute ◮ Pay particular attention to transactional ACID properties

We must restructure the data according to the queries and transactions

Scalable Hosting of Web Applications Service-Oriented Data Denormalization 14 / 33

Step 1: restructure data according to transactions

A transaction may access any number of data items

◮ For consistency these items must

remain inside the same data service

◮ Let’s cluster data items according

to transaction patterns

Scalable Hosting of Web Applications Service-Oriented Data Denormalization 15 / 33

Step 2: restructure data according to regular queries

Problem: many queries may now access data from multiple data services

◮ Naive solution: cluster data services according to regular queries ◮ But this would result into a single monolithic cluster

Instead, we can apply other transformations

◮ Rewrite complex queries into multiple simple queries ◮ Replicate read-only columns across multiple data services ◮ In last resort, merge data services Scalable Hosting of Web Applications Service-Oriented Data Denormalization 16 / 33

Rewrite complex queries

Many join queries can be rewritten into several simple queries Example: SELECT C6 FROM T1,T2 WHERE T1.C1 = ? AND T1.C2 = T2.C5 This query can be rewritten into:

1

SELECT C2 FROM T1 WHERE T1.C1 = ?

2

SELECT C6 FROM T2 WHERE T2.C5 = ? The result of query 1 is the imput

• f query 2

Scalable Hosting of Web Applications Service-Oriented Data Denormalization 17 / 33

Replicate read-only column

Original query: SELECT T1.C1, T1.C2 FROM T1,T2 WHERE T1.C1 = T2.C4 AND T2.C6 = ? Columns T2.C4 and T2.C6 are read-only in the whole application

◮ We can replicate them across

multiple data services

Scalable Hosting of Web Applications Service-Oriented Data Denormalization 18 / 33

Scaling each data service

We studied the case of TPC-W

◮ A standard benchmark modeling an e-commerce site ◮ Standardized workload

Before denormalization:

◮ 10 tables, 6 transactions, 2 atomic sets, 6 UDI queries that are not part

• f a transaction, and 27 read-only queries

After denormalization:

◮ 8 data services, in total 15 tables

Important observation: most data services are read-dominant

◮ Database replication works well for them

Only one data service is update-intensive

◮ Database replication will not work here, we need to pay closer attention Scalable Hosting of Web Applications Service-Oriented Data Denormalization 19 / 33

Scaling the Financial service

The update-intensive service contains all financial-related operations

◮ Shopping carts, orders, item stocks

Most queries are index by shopping cart ID We can apply horizontal partitioning:

◮ Hash table records by their shopping cart ID ◮ Place each record on a different server according to the hash ◮ Consequence: UDIs must be addressed to only one server Scalable Hosting of Web Applications Service-Oriented Data Denormalization 20 / 33

Performance of individual data services

We define a response time objective: 90% of service invocations must return in less than 100 ms When using N servers, how many simultaneous clients can we support before violating the objective?

10000 20000 30000 40000 50000 60000 2 4 6 8 10 12 14 Maximum Throughput (EBs) Number of database servers

Read−dominant services Update−intensive service

Scalable Hosting of Web Applications Service-Oriented Data Denormalization 21 / 33

Performance of the entire application

Response time objective: 90% of client requests must return in less than 500 ms

10000 20000 30000 40000 50000 10 20 30 40 50 60 70 Maximum Throughput (EBs) Number of server machines

Monolithic with master−slave database replication Denormalized

Scalable Hosting of Web Applications Service-Oriented Data Denormalization 22 / 33

Table of Contents

1 Introduction 2 Service-Oriented Data Denormalization 3 Resource Provisioning for Web Services 4 Conclusion

Scalable Hosting of Web Applications Resource Provisioning for Web Services 23 / 33

The “secret sauce” behind the previous graph

How did we plot the previous graph?

◮ For each configuration we must select what each machine will do 10 20 30 40 50 60 70 80 90 10 20 30 40 50 60 70 Machine usage Number of server machines Financial service DB servers Other DB servers A p p l i c a t i

• n

s e r v e r s Load balancers Clients

Method: trial and error :-(

◮ This is not acceptable in a real Web hosting environment. . . Scalable Hosting of Web Applications Resource Provisioning for Web Services 24 / 33

Resource provisioning for a single Web service

One Web service can be seen as being composed of:

◮ 0 or more front-side cache(s) ◮ 1 or more application server(s) ◮ 0 or more database query cache(s) ◮ 0 or more database server(s) ◮ 0 or more external service response cache(s) Database Database

• b

Tier 2 Tier 0 Tier 1 Tier 3 Tier 2a

• cache

DB query cache DB query Service response cache Service response cache server Application server Application cache response Service cache response Service External service External service Scalable Hosting of Web Applications Resource Provisioning for Web Services 25 / 33

We can model a Web service as a queuing network

...

1−p

1

1−p

N

TN

1

T 1−p p

1

p pN (external service)

Service N Cache N

(service

caches)

(databases)

Service 1 Cache 1

(DB query

caches)

Cache 0

(front−side caches)

Service 0

(app.

servers)

Requests

Model:

◮ Poisson distribution of arrival times ◮ Infinite-server queue caches ◮ Processor-sharing application servers and database servers Scalable Hosting of Web Applications Resource Provisioning for Web Services 26 / 33

Mean response time

We can calculate the mean response time: ES = p0βc,0 + (1 − p0)(M + 1)βs,0 1 − ρs,0 + (1 − p0)

N

• i=1

ETi

• piβc,i + (1 − pi)

βs,i 1 − ρs,i

• .

The formula for the variance looks much worse. . .

Scalable Hosting of Web Applications Resource Provisioning for Web Services 27 / 33

Model-based resource provisioning

The performance model allows us to steer resource provisioning

1

Give an SLA to the service

2

Monitor its response time

3

When the SLA is violated: for each tier, compute the expected response time if this tier would have one more server

4

Select the tier that brings the most improvement, add a server there

Similar algorithm for removing servers when traffic decreases Note: there are a few subtleties

◮ How do you estimate the new cache hit rate if you add more caches?

(add more caches ≡ increase cache size)

◮ When should you initiate this process? Scalable Hosting of Web Applications Resource Provisioning for Web Services 28 / 33

Example: TPC-App

100 200 300 400 500 600 700 0 50 100 150 200 250 300 350 400 450 500 Average Response Time (ms) Number of EBs 1AS-1DB 1AS-1DBC-1DB 1AS-1DBC-2DB

Change in Resource Configuration SLA (HighRespTime)

Scalable Hosting of Web Applications Resource Provisioning for Web Services 29 / 33

Resource Provisioning of a multi-service application

Nowadays most service-oriented applications use a graph of services

◮ “If you hit the Amazon.com gateway page, the application calls more

than 100 services to collect data and construct the page for you.” [Werner Vogels, Amazon CTO]

Simple option: give an SLA to each service

◮ Service 1 has the same SLA as the whole

application

◮ How do you select SLAs for the other services? ◮ A wrong choice leads to inefficient resource

usage

Our option: give an SLA only to the front-side service

◮ Let services negotiate resource allocation with

each other

◮ “How much faster/slower can your sub-tree

perform with one more/less machine?”

1 2 5 4 3 7 6 Clients

Scalable Hosting of Web Applications Resource Provisioning for Web Services 30 / 33

Example: a 7-service invocation tree

100 200 300 400 500 600 700 800 900 20 40 60 80 100 120 Front−end response time (ms) Time Workload Response time Prediction

Remove server from service 7 Remove server from service 5 Add server to service 7 Add server to service 5 SLA=500ms

Scalable Hosting of Web Applications Resource Provisioning for Web Services 31 / 33

Table of Contents

1 Introduction 2 Service-Oriented Data Denormalization 3 Resource Provisioning for Web Services 4 Conclusion

Scalable Hosting of Web Applications Conclusion 32 / 33

Conclusion

Web applications are very diverse

◮ Most of them can easily be hosted by a single PC ◮ Some of them require complicated infrastructures with thousands of

servers

◮ It is impossible to predict if a small site will become popular tomorrow!

Even small Web applications should be ready to scale if necessary:

1

Denormalize the application’s data into independent services

2

Enable hosting infrastructures with automatic resource provisioning mechanisms

3

We need pools of resources that can be automatically assigned to applications (Grids, Clouds. . . )

Scalable Hosting of Web Applications Conclusion 33 / 33