6/10/2002 Motivation Motivation Tradable properties (ilities - PDF document

6/10/2002 Motivation Motivation � � Tradable properties (“ilities Tradable properties (“ ilities”) in system design: functionality, ”) in system design: functionality, usability, maintainability, performance, portability, security, usability, maintainability, performance, portability, security, availability, development cost, … availability, development cost, … A A Utility Utility- -Centered Approach Centered Approach � � Examples of multiway Examples of multiway tradeoffs: tradeoffs: to Designing Dependable Internet Services to Designing Dependable Internet Services : data quality ↔ ↔ performance+ availability+ cost � � Inktomi: data quality Inktomi performance+ availability+ cost security+ manageability ↔ ↔ performance+ availability+ cost � � Akamai: Akamai : security+ manageability performance+ availability+ cost cost+ portability ↔ ↔ performance+ functionality � � Yahoo: cost+ portability Yahoo: performance+ functionality George Candea, Armando Fox George Candea, Armando Fox � � Key observation: tradeoffs improve service by providing a Key observation: tradeoffs improve service by providing a better match between service properties and app requirements better match between service properties and app requirements and other ROC- and other ROC -ers ers � � Small systems: right mix is a matter of optimization Small systems: right mix is a matter of optimization Stanford University Stanford University Giant scale: indispensable to the very possibility of building s Giant scale: indispensable to the very possibility of building sys ys 2 ROC Retreat / Lake Tahoe / June 10, 2002 George Candea Issues Issues Proposed Process Proposed Process Identify set of relevant axes that span design space in Identify set of relevant axes that span design space in req req spec spec � � Making the right tradeoffs is mostly Making the right tradeoffs is mostly art art 1. 1. (“spanning set” � � any interesting tradeoff can be expressed in terms of the axes) (“spanning set” any interesting tradeoff can be expressed in terms of the axes) � � 75% of system deployments fail or don’t meet requirements 75% of system deployments fail or don’t meet requirements State system utility functions State system utility functions w.r.t w.r.t. each axis . each axis 2. 2. (Yankee Group, 1998) (Yankee Group, 1998) 3. Identify major design areas; Identify major design areas; 3. � � Deployment costs exceed expectations (Forrester Research: Deployment costs exceed expectations (Forrester Research: choose representative design for each; then choose representative design for each; then 25% of Fortune 1000 reported 10- 25% of Fortune 1000 reported 10 -49% higher costs) 49% higher costs) find their coordinates in design space � � find their coordinates in design space � � To make it engineering To make it engineering , we need three things: , we need three things: � compute overall utility by combining individual utilities compute overall utility by combining individual utilities � 1. A straightforward model for the design space A straightforward model for the design space 1. Choose design area that maximizes utility; repeat w/in scope of 4. 4. Choose design area that maximizes utility; repeat w/in scope of 2. Simple, but comprehensive vocabulary for describing properties Simple, but comprehensive vocabulary for describing properties 2. chosen area chosen area and the outcome of making tradeoffs and the outcome of making tradeoffs � iterative process, with successive refining 3. Step Step- -by by- -step process for trading properties among each other to step process for trading properties among each other to � 3. iterative process, with successive refining maximize usefulness of system maximize usefulness of system 3 ROC Retreat / Lake Tahoe / June 10, 2002 George Candea 4 ROC Retreat / Lake Tahoe / June 10, 2002 George Candea bofa.com : Quality of Data (Fidelity) : Quality of Data (Fidelity) Bank of America Bank of America ( bofa.com ( http:// http://www.bofa.com www.bofa.com ) ) � Utility = how useful is a given level of quality � Utility = how useful is a given level of quality � � System model: service takes inputs and must return System model: service takes inputs and must return outputs within specified amount of time outputs within specified amount of time Utility [normalized] � � Spanning set for design space: Spanning set for design space: 1 � � Q Q uality of data: consistency with real account uality of data: consistency with real account � � A A vailability: % of requests that are completed as required vailability: % of requests that are completed as required � � P P erformance: Throughput and latency for reads/writes erformance: Throughput and latency for reads/writes � � S S ecurity: ITSEC levels ecurity: ITSEC levels � C C ost of ownership: $ amount/year (including initial cost, � ost of ownership: $ amount/year (including initial cost, amortized over expected lifetime of system) amortized over expected lifetime of system) Quality [%] 0 100 ROC Retreat / Lake Tahoe / June 10, 2002 George Candea ROC Retreat / Lake Tahoe / June 10, 2002 George Candea 5 6 1

6/10/2002 bofa.com : Availability : Availability bofa.com : Performance/Latency : Performance/Latency bofa.com bofa.com � � Can choose salient points, then interpolate Can choose salient points, then interpolate Utility [normalized] Utility [normalized] 1 1 Availability [%] Max. Latency [sec] 0 98 100 0 10 -1 5 10 7 ROC Retreat / Lake Tahoe / June 10, 2002 George Candea 8 ROC Retreat / Lake Tahoe / June 10, 2002 George Candea bofa.com : Performance/Throughput : Performance/Throughput bofa.com : Security : Security bofa.com bofa.com Utility [normalized] Utility [normalized] 1 1 Min. Throughput Security [ITSEC EAL] 6 0 1000 2000 0 1 2 3 4 5 7 [# responses / sec] 9 ROC Retreat / Lake Tahoe / June 10, 2002 George Candea 10 ROC Retreat / Lake Tahoe / June 10, 2002 George Candea bofa.com : Cost : Cost bofa.com Proposed Process Overview Proposed Process Overview Identify set of relevant axes that span design space Identify set of relevant axes that span design space 1. 1. (“spanning set” � � any interesting tradeoff can be expressed in terms of the axes) any interesting tradeoff can be expressed in terms of the axes) Utility [normalized] (“spanning set” 2. State system utility functions with respect to each axis State system utility functions with respect to each axis 2. 1 Identify major design areas; Identify major design areas; 3. 3. choose representative design for each; then choose representative design for each; then � � find their coordinates in design space find their coordinates in design space � � compute overall utility by combining individual utilities compute overall utility by combining individual utilities Choose design area that maximizes utility; repeat w/in scope of Choose design area that maximizes utility; repeat w/in scope of 4. 4. chosen area chosen area TCO [M$/year] � � iterative process, with successive refining 0 iterative process, with successive refining 1 3 4 ROC Retreat / Lake Tahoe / June 10, 2002 George Candea ROC Retreat / Lake Tahoe / June 10, 2002 George Candea 11 12 2