QoS for (Web) Applications Velocity EU 2011 Intelligent Self - - PowerPoint PPT Presentation

▶

Dec 22, 2022 754 likes •1.17k views

QoS for (Web) Applications Velocity EU 2011 Intelligent Self Activity Regulated Metering Software Signals & Control william.louth@jinspired.com Self Adaptive Software Self Adaptive Software evaluates its own behavior and changes

SLIDE 1

QoS for (Web) Applications Velocity EU 2011

william.louth@jinspired.com

Intelligent Activity Metering Self Regulated Software Signals & Control

SLIDE 2

“Self Adaptive Software evaluates its own behavior and changes behavior when the evaluation indicates that it is not accomplishing what the software is intended to do, or when better functionality or performance is possible.” DARPA

Self Adaptive Software

SLIDE 3 Observation Judgement Reaction Self Regulated Evidence Relevance Consequence Action Feedback Loop 1 2 3 4

Feedback Loops

Evidence, Relevance, Conseqence, Action

SLIDE 4 Measure Observe & Optimize Control & Collect

Self Observation

Observe, Optimize, Control, Collect

SLIDE 5 Goals Control Disturbances Process Sensor Tolerance AutoTune Controller AutoTune Probe AutoTune Monitor Threshold Run Reading Metering

Self Control

Control, Process, Sensor

SLIDE 6 Present Past Predicted Local Remote

DIRT - Data in Realtime

Remote, Past, Local, Present, Predicted

SLIDE 7 Profile Protect Police Prioritize Predict Provision

99.9999% => PP.PPPP

Profile,Protect,Police,Prioritize,Predict,Provide

SLIDE 8

instrument, measure, and collect performance data for a significant portion of an application code base

Profile

SLIDE 9

supervisory routines that are embedded into the runtime to govern the execution flow & consumption of resources

Protect

SLIDE 10

watching for non-compliant behavior & taking corrective measures to enforce call flow contracts and agreements

Police

SLIDE 11

differentiating the quality of service that is offered to applications, activities and actors

Prioritize

SLIDE 12

controllers that use past & current in-flight data & models to predict behavior & consumption and then to optimize

Predict

SLIDE 13

resource capacity planning based on inflight activities & predicted their predicted requirements - in internet time

Provide

SLIDE 14

“The ability of the network to provide better or “special” service to a set of packets/dataflows to the detriment of

ther packets/dataflows”

QoS for Networks

SLIDE 15

Bandwidth the number of bits per second that can be expected to be transmitted Delay the elapse time between when a packet is first sent and when it arrives Jitter the variation in the arrival rate, or delay introduced, of packets sent at a uniform rate Packet loss all routers lose, drop of discard packets for a number of reasons

Traffic Characteristics

SLIDE 16

Identify traffic and its requirements in terms of service levels Divide and Associate traffic into classes of service levels Define policies for each class of service level which cause some change in the traffic characteristic mentioned previously

Traffic Planning

SLIDE 17

differentiating one packet from another by examining fields inside of the packet’s header in some cases this classification is performed automatically and adaptively based on traffic patterns and resource consumption behavior

Traffic Service Classification

SLIDE 18

using queues and schedulers which split traffic across queues and schedulers based on service classification associated with a packet or flow

Traffic Congestion Management

SLIDE 19

Policing traffic contracts which define how much data can be sent Shaping smoothing the peaks & troughs of data transmission in order to optimize or guarantee performance & bandwidth.

Traffic Policing and Shaping

SLIDE 20

“The ability of the runtime to provide better or “special” service to a set of calls/threads to the detriment of other calls/threads”.

QoS for Applications

SLIDE 21

Throughput the number of requests/calls per second that can expect to receive a valid response Response Time the time between a request/call is sent (received) & when it’s response is received (sent) Response Time Variation the variation in the response time of requests/calls Exceptions & Errors faults can occur in runtimes such as timeouts caused by contention and corruption

Call Characteristics

SLIDE 22

Identify request/call patterns and their requirements in terms of service levels Divide and Associate request/call patterns into classes of service levels. Define policies for each class of service level which cause some change in the request/call characteristic mentioned previously

Call Planning

SLIDE 23

differentiation is typically done by examining the execution context associated with a request/call

Call Service Classification

SLIDE 24

managed using concurrency and control constructs such as semaphores/latches/locks which are generally backed by implicit or explicit queues consisting of parked threads during contention

Call Congestion Management

SLIDE 25

Policing call contracts which define how many calls can be sent Shaping it tries to conform the call rate to the call contract using some form of delay mechanism such as waiting and parking

Call Policing & Shaping

SLIDE 26

Network Application Packet Call Flow Thread Router Method Datalink Call Site Buffer Resource Queue Semaphore

Network-to-Application Mapping

SLIDE 27 QoS service (re-)classification point QoS reservation extension or (re-)prioritization point bottleneck point (consumption, coordination, contention) code execution point (method, call site, activity) code execution flow path QoS queuing, prioritization, rate limiting

The Application is the Network

SLIDE 28

metering has the potential to revolutionize the software design, development, testing, and management a metering standard would define a model, concepts and mechanism which can be used across languages and processing boundaries to control and cost consumption

Activity Based (Resource) Metering

SLIDE 29

Method Metering Metric Analysis Causality Correlation Accuracy Event Sampled Allocation Thread Process Assignment Direct Apportioned

Methods of Measurement

SLIDE 30

Model Activity Resource Device Probe Meter Develop Code Counter Design Behavior Usage Data Group Metering

Metering Measurement Model

SLIDE 31 public int func(...) { ...... ........ ......... ......... ...... ........ ... ..... ... } Rate Limiting Priority Queueing QoS Resource Reservation Lanes reserve(func) release(units)

QoS Enablement

SLIDE 32

Demo: Prioritization

L1: L2: L3: L4: Ln: Pool

SLIDE 33

Demo: Resource Reservation

Give Take Permit Pool

SLIDE 34

Demo: Rate Limiting

“Leaky” Bucket

SLIDE 35 resource pool lane #1 lane #2 lane #N within range within range not in range within range priority level capacity limit not in range

QoS Reservation Lanes

SLIDE 36

Fairer Web Page Servicing with Performance Credits

User WebPage Resource

SLIDE 37

Fairer Web Page Servicing with Performance Credits

SLIDE 38 PaaS Application PaaS Runtime Extensions Open API Metering Open API Plugins APM CARS QoS CARS: Cost Aware Runtimes & Services APM: Application Performance Management QoS: Quality of Service BI BI: Business Intelligence Open API Open API Consumption Control Chargeback Context

A Cloud Cortex

SLIDE 39 Client cl ☁

☁

salesforce.com sf sf:clock.time= sf:cpu.time= sf:io.bytes= sf:charge.unit= sf:db.time= sf:db.count= s3:clock.time= s3:io.bytes= s3:charge.unit= cl:clock.time= sf:clock.time= sf:cpu.time= sf:io.bytes= sf:charge.unit= sf:db.time= sf:db.count= s3:clock.time= s3:io.bytes= s3:charge.unit= Metering Management Service Response

☁

aws.amazon.com s3 s3:clock.time= s3:io.bytes= s3:charge.unit= db Response sf://..........

Cloud Services

SLIDE 40

More Information

http://opencore.jinspired.com