Optimization models for non-functional requirements validation - PowerPoint PPT Presentation

Optimization models for non-functional requirements validation Vittorio Cortellessa Computer Science and Engineering Department Università dell'Aquila – Italy vittorio.cortellessa@univaq.it http://www.di.univaq.it/cortelle/

Presentation roadmap • Software non-functional requirements (NFR) • Non-functional attribute (NFA) composition • Optimization models • Conclusions and perspectives V. Cortellessa, Optimization models for non-functional requirement validation 2 CREST Workshop : Requirements and Test Optimization - UCL, February 11-12, 2013 SEA Group

Software non-functional requirements “Good enough” Non Functional Requirements (NFR) specification: • Quantification rather than qualification The average response time of BrowseCatalog service must not be higher than 1.5 seconds… rather than The BrowseCatalog service must be quick • Workload specification The average response time of BrowseCatalog service must not be higher than 1.5 seconds under a maximum workload of 50 requests/second V. Cortellessa, Optimization models for non-functional requirement validation 3 CREST Workshop : Requirements and Test Optimization - UCL, February 11-12, 2013 SEA Group

Software non-functional requirements Non-functional requirements validation cannot be effectively carried out without these good practices But what is it expected from NF validation in general? Early artifacts (e.g. models) in the lifecycle – Quantitatively compare different software solutions vs requirements Late artifacts (e.g. code) in the lifecycle – Estimate realistic values of NF attributes V. Cortellessa, Optimization models for non-functional requirement validation 4 CREST Workshop : Requirements and Test Optimization - UCL, February 11-12, 2013 SEA Group

Non-functional attribute composition 1) On one NF attribute 2) On multiple NF attributes V. Cortellessa, Optimization models for non-functional requirement validation 5 CREST Workshop : Requirements and Test Optimization - UCL, February 11-12, 2013 SEA Group

Non-functional attribute composition 1) On one NF attribute Expressing (possibly in a closed form) the whole system attribute in terms of component/service attributes Ф (C1) Ф () : non functional attribute (e.g., C1 reliability) Ф (Cn) Cn Connectors Ф (C2) C2 Ф (S) = Ф (C1) ⊗ Ф (C2) ⊗ … ⊗ Ф (Cn) V. Cortellessa, Optimization models for non-functional requirement validation 6 CREST Workshop : Requirements and Test Optimization - UCL, February 11-12, 2013 SEA Group

Non-functional attribute composition 2) Across different NF attributes Expressing (possibly in a closed form) the relationships/tradeoffs/dependencies among attributes Requirement: <<data confidentiality>> C 1 C 2 Throughput sendData initial system t 1 C 1 C 2 t 2 secure system Encryption* sendData Decryption* λ Workload V. Cortellessa, Optimization models for non-functional requirement validation 7 CREST Workshop : Requirements and Test Optimization - UCL, February 11-12, 2013 SEA Group

DOMAIN Component-based Software Optimization models Choice of components driven by non-functional properties We have introduced several optimization models… R EQUIREMENTS P HASE Based on satisfaction functions Cost vs satisfaction of requirements [Filkenstein et al.] ARCHITECTURAL P HASE Cost vs reliability and delivery time DEPLOYMENT P HASE Closed-form of performance indices cannot capture waiting Cost vs reliability and performance times on shared resources Opening to evolving/adaptive MAINTENANCE PHASE systems where new requirements Change management come ofter deployment V. Cortellessa, Optimization models for non-functional requirement validation 8 CREST Workshop : Requirements and Test Optimization - UCL, February 11-12, 2013 SEA Group

A RCHITECTURAL P HASE On the basis of an architectural design, for each software component we assume to have different COTS available to buy or different in-house versions to build . We also assume that all components are equivalent by a functional viewpoint. We intend to determine the combination of available COTS products and in-house developed components that minimizes the software costs under delivery time and reliability constraints. As a ”side effect”, we provide the amount of testing to be performed on each in-house component in order to achieve the required reliability level. V. Cortellessa, Optimization models for non-functional requirement validation 9 CREST Workshop : Requirements and Test Optimization - UCL, February 11-12, 2013 SEA Group

O PTIMIZATION M ODEL $ ' min n & ) − ∑ ∑ tot ) x ij + ∑ c ij ( t ij + τ ij N ij c ij x ij & ) & ) _ i = 1 j ∈ J i j ∈ J i % ( ∑ tot ) x ij + ∑ max i = 1,.., n ( ( t ij + τ ij N ij d ij x ij ) ≤ T − j ∈ J i j ∈ J i ∑ ∑ − ( θ ij s i x ij + µ ij s i x ij ) n j ∈ Ji j ∈ Ji ∏ e ≥ R i = 1 ∑ x ij = 1, ∀ i = 1,..., n − j ∈ J i  J i − x ij ∈ 0,1 { } , ∀ i = 1,..., n , ∀ j ∈ J i ∪ J i V. Cortellessa, Optimization models for non-functional requirement validation 10 CREST Workshop : Requirements and Test Optimization - UCL, February 11-12, 2013 SEA Group

V ARIABLES In general, a “ build-or-buy ” decisional strategy can be described as a set of 0/1 variables defined as follows ( ∀ i = 1,…,n): − 1 if instance C ij is chosen ( j J or j J ) x ij = ! ∈ ∈ i i " 0 otherwise # The variables must fulfill the following constraints: For each component i , x ij 1, ∈ ∑ = i 1,..., n exactly one instance is ∀ = either bought as COTS _ or in-house developed. U j J J i i V. Cortellessa, Optimization models for non-functional requirement validation 11 CREST Workshop : Requirements and Test Optimization - UCL, February 11-12, 2013 SEA Group

C OST O BJECTIVE F UNCTION We express the Cost Objective Function as follows: ! $ n # & tot c t ( N ) x c x ∑ ∑ ∑ Cost of a + τ + # & ij ij ij ij ij ij ij COTS # & i 1 j J − = ∈ component i j J ∈ " % i For each instance j and component i let: c ij be the buying cost V. Cortellessa, Optimization models for non-functional requirement validation 12 CREST Workshop : Requirements and Test Optimization - UCL, February 11-12, 2013 SEA Group

D ELIVERY T IME C ONSTRAINT A maximum threshold T has been given on the delivery time of the whole system. The following expression represents the delivery time of the component i : tot ( t N ) x d x ∑ ∑ Delivery time + τ + ij ij ij ij ij ij of an in-house instance. j J − ∈ i j J ∈ i For each instance j and component i let: t ij be the estimated development time τ ij be the average time required to perform a test case V. Cortellessa, Optimization models for non-functional requirement validation 13 CREST Workshop : Requirements and Test Optimization - UCL, February 11-12, 2013 SEA Group

D ELIVERY T IME C ONSTRAINT A maximum threshold T has been given on the delivery time of the whole system. The following expression represents the delivery time of the component i : Delivery tot ( t N ) x d x ∑ ∑ + τ + time of a ij ij ij ij ij ij COTS j J − ∈ component i j J ∈ i For each instance j and component i let: d ij be the delivery time V. Cortellessa, Optimization models for non-functional requirement validation 14 CREST Workshop : Requirements and Test Optimization - UCL, February 11-12, 2013 SEA Group

RELIABILITY C ONSTRAINT A minimum reliability R is required for the whole system. A closed-form expression represents the reliability of the whole system: ∑ ∑ − ( θ ij s i x ij + µ ij s i x ij ) n j ∈ Ji j ∈ Ji ∏ e ≥ R i = 1 Here is the requirement/testing joint point… V. Cortellessa, Optimization models for non-functional requirement validation 15 CREST Workshop : Requirements and Test Optimization - UCL, February 11-12, 2013 SEA Group

RELIABILITY C ONSTRAINT Probability that no failure occurs during the execution of the i -th component [Jung et al., 1999] : fnum e − φ i = i average number of fnum s x s x ∑ ∑ = θ + µ failures of a i ij i ij ij i ij component instance j J − ∈ i j J ∈ i probability of failure on-demand The probability of failure on demand θ ij of the in-house developed instance C ij [Bertolino et al., 1996] : suc (other reliability N ij θ ij = Testab ij * p ij (1 − Testab ij ) growth models in suc closed forms can be N ij (1 − p ij ) + p ij (1 − Testab ij ) adopted here) N succ ij the number of succesful (i.e. failure free) tests performed on an in-house instance V. Cortellessa, Optimization models for non-functional requirement validation 16 CREST Workshop : Requirements and Test Optimization - UCL, February 11-12, 2013 SEA Group

Just another (newer) example of closed-form reliability growth model that we are using now… θ i = V. Cortellessa, Optimization models for non-functional requirement validation 17 CREST Workshop : Requirements and Test Optimization - UCL, February 11-12, 2013 SEA Group