Toward Architecture-based Reliability Estimation Roshanak Roshandel & Nenad Medvidovic Computer Science Department University of Southern California {roshande,neno}@usc.edu
Motivation • Software reliability: probability that the system performs its intended functionality without failure • Software reliability techniques aim at reducing or eliminating failure of software systems • Complementary to testing , rely on implementation • How do we go about building reliable systems? • How do we measure reliability early?
Software Architecture • High-level abstractions describing – Structure, Behavior, Constraints • Coarse-grain building blocks, promote separation of concerns, reuse – Components, Connectors, Interfaces, Configurations • Architectural decisions directly affect aspects of software dependability – Reliability • ADLs, Formal modeling notations, related analysis – Often lack quantification and measurement
Architectural Reliability • Lightly explored • Require availability of implementation to: – Build behavioral model of the software system – Obtain each component’s reliability • Software architecture offers compositional approaches to modeling and analysis • The challenge is quantifying these results – Presence of uncertainty – Unknown operational profile – Improper behavior
Archi chitect ectur ure Static Dynamic Behaviors Behaviors Static Behavio Static Dynamic Interface rs Behavio Behaviors Interfac rs e “The Interfac Quartet” e Protocols Component onent Protoco “The Quartet” Component onent ls Protocols Component onent M M ar kov ar kov M M odel odel M M ar kov ar kov M M odel odel Local Reliability Local Local Reliability Reliability M M ar kov ar kov Global M M odel odel Reliability
Quantification Classification Cost framework State Architectural Defects Analysis Reliability r Models θ ( ( ), ) G t f Hidden Markov Modeling State-based Reliability Markov model Estimator Comp Reliability Model Extractor Training data Baum- Welch Transition Domain Knowledge Algorithm Probabilities ITP OR Random Legend Component Numerical Artifacts values Major steps of Iterative Reliability the approach process Initial transition ITP probabilities
The Quartet 1. Interface • Point by which a component interacts with other components 2. Static behavior • Discrete functionality of a component • i.e., at particular “snapshots” during the system’s execution 3. Dynamic behavior • Continuous view of how a component arrives at different states throughout its execution 4. Interaction protocol • E xternal view of the component • Specifies its legal interactions with other components in the system
DYNAM IC BEHAVIOR decelerate() accelerate () gas/accelerate gas/accelerate Cruise Control brake[ val +curSpeed >0 ] cruise() brake() manual stop Comp /decelerate ≤ brake[ val +curSpeed 0 ] /decelerate gas() maintain() brake/decelerate cruise/maintain cruise INT ERFACES PRO V gas( val : SpeedType) : SpeedType; PRO V br ake( val : SpeedType) : SpeedType; gas/accelerate PRO V cr ui se( speed: SpeedType) ; Bool ean; gas() ST ATIC BEHAVIOR INT ERACT ION STATE- VAR: PROT OCOLS cur Speed: SpeedType; i sCr ui si ng: Bool ean; I NVARI ANT: S 1 ≤ ≤ 0 cur Speed M AX; O PERATI O NS: brake() gas. pr eCond ( val > 0) ; brake() gas. post Cond ( ~cur Speed = cur Speed + val ) ; gas br ake. pr eCond ( val < 0) ; br ake. post Cond ( ~cur Speed = cur Speed + val AND i sCr ui si ng = f al se) ; cr ui se. pr eCond ( speed > 0) ; cruise cr ui se. post Cond ( ~cur Speed = speed S 2 AND i sCr ui si ng = t r ue) ;
Quantification Classification Cost framework State Architectural Defects Analysis Reliability r Models θ ( ( ), ) G t f Hidden Markov Modeling State-based Reliability Markov model Estimator Comp Reliability Model Extractor Training data Baum- Welch Transition Domain Knowledge Algorithm Probabilities ITP OR Random Legend Component Numerical Artifacts values Major steps of Iterative Reliability the approach process Initial transition ITP probabilities
Static Dynamic Behaviors Behaviors Interface Interaction Protocols Syntactic Semantic
Quantification Classification Cost framework State Architectural Defects Analysis Reliability r Models θ ( ( ), ) G t f Hidden Markov Modeling State-based Reliability Markov model Estimator Comp Reliability Model Extractor Training data Baum- Welch Transition Domain Knowledge Algorithm Probabilities ITP OR Random Legend Component Numerical Artifacts values Major steps of Iterative Reliability the approach process Initial transition ITP probabilities
Defect Quantification • Architectural defects could affect system Reliability • Different defects affect the Reliability differently – e.g., interface mismatch vs. protocol mismatch • The cost of mitigating defects varies based on the defect type • Other (domain specific) factors may affect the quantification • Classification + Cost framework
Classification + Cost Framework Architectural • Pluggable/Adaptable Defect • Identify the important Topological Topological Error Error factors within a domain • For a defect class t Directional Directional r Usage Usage = θ ( ( ), ), c G t f where t Structural Structural r θ = θ θ θ ( ) [ ( ), ( ),..., ( )] t t t t Incomplete Incomplete 1 2 Behavioral Behavioral n Inconsistency Inconsistency • f: Frequency of occurrence Interface Interface Signatures Signatures r θ • And vector of all ( ) t Pre / Pre Post / Post relevant factors Static Behavior Static Behavior Conditions Conditions • Result will be used in Interaction Interaction Protocol Protocol reliability estimation Protocols Protocols
Quantification Classification Cost framework State Architectural Defects Analysis Reliability r Models θ ( ( ), ) G t f Hidden Markov Modeling State-based Reliability Markov model Estimator Comp Reliability Model Extractor Training data Baum- Welch Transition Domain Knowledge Algorithm Probabilities ITP OR Random Legend Component Numerical Artifacts values Major steps of Iterative Reliability the approach process Initial transition ITP probabilities
Reliability Techniques • Non-Homogenous Poisson Processes, Binomial Models, Software Reliability Growth Models, … • Markovian Models – Suited to architectural approaches – Consider a system’s structure, compositional – Stochastic processes – Informally, a finite state machine extended with transition probabilities
Our Reliability Model • Built based on the dynamic behavioral model • Assume Markov property – Discrete Time Markov Chains • Transition probabilities may be unknown • Complex behavior results in lack of a correspondence between events and states • Event/action pairs to describe component interactions � Augmented Hidden Markov Models (AHMM)
Evaluation • Uncertainty analysis – Operational profile – Incorrect behavior • Sensitivity analysis – Traditional Markov-based sensitivity analysis combined with the defect quantification • Complexity • Scalability
Conclusion and Future Work • Step toward closing the gap between architectural specification and its effect on system’s reliability • Handles two types of uncertainties associated with early reliability estimation • Preliminary results are promising • Need further evaluation • Build compositional models to estimate system reliability based on estimated component reliabilities
Questions?
AHMM = : , { ,..., } S Set of all possible States S S S 1 N : N Number of states : q state at time t t = : , { ,..., } E Set of all events E E E 1 M : M Number of events : , :{ ,..., } F Set of all actions F F F 1 K : K Number of actions : We now define λ = π ( , , ) : A B is a Hidden Markov M odel such that : A statetransition probability distribution = = = = ≤ ≤ { }, Pr[ | ], 1 , A a a q S q S i j N + ij ij t 1 j t i : B Interface probability distribution in state j = { ( )} B b m j = = ≤ ≤ ≤ ≤ ≤ ≤ ( ) Pr[ / | ], 1 ,1 ,1 b m E F at t q S j N m M k K j m k t j π π = π : { } The initial proba bility distribution i π = = ≤ ≤ Pr[ ],1 . q S i n i 1 i
Cruise Control Example DYNAM IC BEHAVIOR gas/accelerate gas/accelerate manual stop brake/decelerate brake/decelerate brake/decelerate cruise/maintain cruise gas/accelerate
Partial Markov Extension gas/accelerate TRUE TRUE gas/accelerate gas/accelerate brake/ brake/decelerate manual stop decelerate gas/ accelerate brake/ brake/ decelerate decelerate gas/ gas/accelerate accelerate cruise/ maintain gas/accelerate cruise TRUE gas/accelerate
Recommend
More recommend
