CS 147: Computer Systems Performance Analysis Fractional Factorial - PowerPoint PPT Presentation

CS147 2015-06-15 CS 147: Computer Systems Performance Analysis Fractional Factorial Designs CS 147: Computer Systems Performance Analysis Fractional Factorial Designs 1 / 26

Overview CS147 Overview 2015-06-15 2 k − p Designs Example Preparing the Sign Table Confounding Algebra of Confounding Overview Design Resolution 2 k − p Designs Example Preparing the Sign Table Confounding Algebra of Confounding Design Resolution 2 / 26

2 k − p Designs Example Introductory Example of a 2 k − p Design Introductory Example of a 2 k − p Design CS147 2015-06-15 2 k − p Designs Exploring 7 factors in only 8 experiments: Run A B C D E F G 1 -1 -1 -1 1 1 1 -1 Example 2 1 -1 -1 -1 -1 1 1 3 -1 1 -1 -1 1 -1 1 4 1 1 -1 1 -1 -1 -1 Introductory Example of a 2 k − p Design 5 -1 -1 1 1 -1 -1 1 6 1 -1 1 -1 1 -1 -1 7 -1 1 1 -1 -1 1 -1 8 1 1 1 1 1 1 1 Exploring 7 factors in only 8 experiments: Run A B C D E F G 1 -1 -1 -1 1 1 1 -1 2 1 -1 -1 -1 -1 1 1 3 -1 1 -1 -1 1 -1 1 4 1 1 -1 1 -1 -1 -1 5 -1 -1 1 1 -1 -1 1 6 1 -1 1 -1 1 -1 -1 7 -1 1 1 -1 -1 1 -1 8 1 1 1 1 1 1 1 3 / 26

2 k − p Designs Example Analysis of 2 7 − 4 Design Analysis of 2 7 − 4 Design CS147 2015-06-15 2 k − p Designs ◮ Column sums are zero: � i x ij = 0 ∀ j ◮ Sum of 2-column product is zero: � x ij x il = 0 ∀ j � = l Example i ◮ Sum of column squares is 2 7 − 4 = 8 Analysis of 2 7 − 4 Design ◮ Orthogonality allows easy calculation of effects: q A = − y 1 + y 2 − y 3 + y 4 − y 5 + y 6 − y 7 + y 8 8 etc. ◮ Column sums are zero: � i x ij = 0 ∀ j ◮ Sum of 2-column product is zero: � x ij x il = 0 ∀ j � = l i ◮ Sum of column squares is 2 7 − 4 = 8 ◮ Orthogonality allows easy calculation of effects: q A = − y 1 + y 2 − y 3 + y 4 − y 5 + y 6 − y 7 + y 8 8 etc. 4 / 26

2 k − p Designs Example Effects and Confidence Intervals for 2 k − p Designs Effects and Confidence Intervals for 2 k − p Designs CS147 2015-06-15 2 k − p Designs ◮ Effects are as in 2 k designs: 1 q α = � y i x α i Example 2 k − p i ◮ % variation proportional to squared effects Effects and Confidence Intervals for 2 k − p ◮ For standard deviations & confidence intervals: ◮ Use formulas from full-factorial designs ◮ Replace 2 k with 2 k − p Designs ◮ Effects are as in 2 k designs: 1 � q α = y i x α i 2 k − p i ◮ % variation proportional to squared effects ◮ For standard deviations & confidence intervals: ◮ Use formulas from full-factorial designs ◮ Replace 2 k with 2 k − p 5 / 26

2 k − p Designs Preparing the Sign Table Preparing the Sign Table for a 2 k − p Design Preparing the Sign Table for a 2 k − p Design CS147 2015-06-15 2 k − p Designs Preparing the Sign Table ◮ Prepare sign table for k − p factors ◮ Assign remaining factors Preparing the Sign Table for a 2 k − p Design ◮ Prepare sign table for k − p factors ◮ Assign remaining factors 6 / 26

2 k − p Designs Preparing the Sign Table Sign Table for k − p Factors CS147 Sign Table for k − p Factors 2015-06-15 2 k − p Designs ◮ Same as table for experiment with k − p factors ◮ I.e., 2 ( k − p ) table Preparing the Sign Table ◮ 2 k − p rows and 2 k − p columns ◮ First k − p columns get k − p chosen factors ◮ Rest are interactions (products of previous columns) ◮ “I” column can be included or omitted as desired Sign Table for k − p Factors ◮ Same as table for experiment with k − p factors ◮ I.e., 2 ( k − p ) table ◮ 2 k − p rows and 2 k − p columns ◮ First k − p columns get k − p chosen factors ◮ Rest are interactions (products of previous columns) ◮ “I” column can be included or omitted as desired 7 / 26

2 k − p Designs Preparing the Sign Table Assigning Remaining Factors CS147 Assigning Remaining Factors 2015-06-15 2 k − p Designs ◮ 2 k − p − ( k − p ) − 1 interaction (product) columns will remain Preparing the Sign Table ◮ Choose any p columns ◮ Assign remaining p factors to them ◮ Any others stay as-is, measuring interactions Assigning Remaining Factors ◮ 2 k − p − ( k − p ) − 1 interaction (product) columns will remain ◮ Choose any p columns ◮ Assign remaining p factors to them ◮ Any others stay as-is, measuring interactions 8 / 26

2 k − p Designs Preparing the Sign Table Example of Preparing a Sign Table CS147 Example of Preparing a Sign Table 2015-06-15 2 k − p Designs A 2 4 − 1 design: Run A B C 1 -1 -1 -1 Preparing the Sign Table 2 1 -1 -1 3 -1 1 -1 4 1 1 -1 5 -1 -1 1 Example of Preparing a Sign Table 6 1 -1 1 7 -1 1 1 8 1 1 1 A 2 4 − 1 design: Run A B C 1 -1 -1 -1 2 1 -1 -1 3 -1 1 -1 4 1 1 -1 5 -1 -1 1 6 1 -1 1 7 -1 1 1 8 1 1 1 9 / 26

2 k − p Designs Preparing the Sign Table Example of Preparing a Sign Table CS147 Example of Preparing a Sign Table 2015-06-15 2 k − p Designs A 2 4 − 1 design: Run A B C AB AC BC ABC 1 -1 -1 -1 1 1 1 -1 Preparing the Sign Table 2 1 -1 -1 -1 -1 1 1 3 -1 1 -1 -1 1 -1 1 4 1 1 -1 1 -1 -1 -1 5 -1 -1 1 1 -1 -1 1 Example of Preparing a Sign Table 6 1 -1 1 -1 1 -1 -1 7 -1 1 1 -1 -1 1 -1 8 1 1 1 1 1 1 1 A 2 4 − 1 design: Run A B C AB AC BC ABC 1 -1 -1 -1 1 1 1 -1 2 1 -1 -1 -1 -1 1 1 3 -1 1 -1 -1 1 -1 1 4 1 1 -1 1 -1 -1 -1 5 -1 -1 1 1 -1 -1 1 6 1 -1 1 -1 1 -1 -1 7 -1 1 1 -1 -1 1 -1 8 1 1 1 1 1 1 1 9 / 26

2 k − p Designs Preparing the Sign Table Example of Preparing a Sign Table CS147 Example of Preparing a Sign Table 2015-06-15 2 k − p Designs A 2 4 − 1 design: Run A B C AB AC BC D 1 -1 -1 -1 1 1 1 -1 Preparing the Sign Table 2 1 -1 -1 -1 -1 1 1 3 -1 1 -1 -1 1 -1 1 4 1 1 -1 1 -1 -1 -1 5 -1 -1 1 1 -1 -1 1 Example of Preparing a Sign Table 6 1 -1 1 -1 1 -1 -1 7 -1 1 1 -1 -1 1 -1 8 1 1 1 1 1 1 1 A 2 4 − 1 design: Why did we choose the ABC column to Run A B C AB AC BC D rename as D? In one sense, the choice 1 -1 -1 -1 1 1 1 -1 is completely arbitrary. But in reality, this 2 1 -1 -1 -1 -1 1 1 leads to a discussion of confounding. 3 -1 1 -1 -1 1 -1 1 4 1 1 -1 1 -1 -1 -1 5 -1 -1 1 1 -1 -1 1 6 1 -1 1 -1 1 -1 -1 7 -1 1 1 -1 -1 1 -1 8 1 1 1 1 1 1 1 9 / 26

2 k − p Designs Confounding Confounding CS147 Confounding 2015-06-15 2 k − p Designs ◮ The confounding problem Confounding ◮ An example of confounding ◮ Confounding notation ◮ Choices in fractional factorial design Confounding ◮ The confounding problem ◮ An example of confounding ◮ Confounding notation ◮ Choices in fractional factorial design 10 / 26

2 k − p Designs Confounding The Confounding Problem CS147 The Confounding Problem 2015-06-15 2 k − p Designs ◮ Fundamental to fractional factorial designs Confounding ◮ Some effects produce combined influences ◮ Limited experiments means only combination can be counted ◮ Problem of combined influence is confounding ◮ Inseparable effects called confounded The Confounding Problem ◮ Fundamental to fractional factorial designs ◮ Some effects produce combined influences ◮ Limited experiments means only combination can be counted ◮ Problem of combined influence is confounding ◮ Inseparable effects called confounded 11 / 26

2 k − p Designs Confounding An Example of Confounding CS147 An Example of Confounding 2015-06-15 2 k − p Designs ◮ Consider this 2 3 − 1 table: I A B C 1 -1 -1 1 1 1 -1 -1 Confounding 1 -1 1 -1 1 1 1 1 ◮ Extend it with an AB column: I A B C AB 1 -1 -1 1 1 An Example of Confounding 1 1 -1 -1 -1 1 -1 1 -1 -1 1 1 1 1 1 ◮ Consider this 2 3 − 1 table: I A B C There is an animation on this slide 1 -1 -1 1 1 1 -1 -1 1 -1 1 -1 1 1 1 1 ◮ Extend it with an AB column: I A B C AB 1 -1 -1 1 1 1 1 -1 -1 -1 1 -1 1 -1 -1 1 1 1 1 1 12 / 26

2 k − p Designs Confounding An Example of Confounding CS147 An Example of Confounding 2015-06-15 2 k − p Designs ◮ Consider this 2 3 − 1 table: I A B C 1 -1 -1 1 1 1 -1 -1 Confounding 1 -1 1 -1 1 1 1 1 ◮ Extend it with an AB column: I A B C AB 1 -1 -1 1 1 An Example of Confounding 1 1 -1 -1 -1 1 -1 1 -1 -1 1 1 1 1 1 ◮ Consider this 2 3 − 1 table: I A B C There is an animation on this slide 1 -1 -1 1 1 1 -1 -1 1 -1 1 -1 1 1 1 1 ◮ Extend it with an AB column: I A B C AB 1 -1 -1 1 1 1 1 -1 -1 -1 1 -1 1 -1 -1 1 1 1 1 1 12 / 26

2 k − p Designs Confounding Analyzing the Confounding Example CS147 Analyzing the Confounding Example 2015-06-15 2 k − p Designs ◮ Effect of C is same as that of AB: q C = ( y 1 − y 2 − y 3 + y 4 ) / 4 q AB = ( y 1 − y 2 − y 3 + y 4 ) / 4 Confounding ◮ Formula for q C really gives combined effect: Analyzing the Confounding Example q C + q AB = ( y 1 − y 2 − y 3 + y 4 ) / 4 ◮ Effect of C is same as that of AB: There is an animation on this slide q C = ( y 1 − y 2 − y 3 + y 4 ) / 4 q AB = ( y 1 − y 2 − y 3 + y 4 ) / 4 ◮ Formula for q C really gives combined effect: q C + q AB = ( y 1 − y 2 − y 3 + y 4 ) / 4 13 / 26