Gregory M. Kapfhammer Department of Computer Science Allegheny - PowerPoint PPT Presentation

Introduction Challenges Potential Way Forward Conclusion Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward Gregory M. Kapfhammer Department of Computer Science Allegheny College http://www.cs.allegheny.edu/~gkapfham/ 1st International Workshop on Regression Testing Co-Located with the 4th IEEE International Conference on Software Testing, Verification and Validation Berlin, Germany, March 2011 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward 1 / 10

Introduction Challenges Potential Way Forward Conclusion Presentation Overview Regression Testing Community Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward 2 / 10

Introduction Challenges Potential Way Forward Conclusion Presentation Overview Current Trends Regression Testing Community Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward 2 / 10

Introduction Challenges Potential Way Forward Conclusion Presentation Overview Current Trends Regression Increased Testing Interest Community Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward 2 / 10

Introduction Challenges Potential Way Forward Conclusion Presentation Overview Current Trends Regression Increased Testing Interest Community Software Empirical Testing Studies Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward 2 / 10

Introduction Challenges Potential Way Forward Conclusion Presentation Overview Current Techniques Trends Regression Increased Testing Interest Community Software Empirical Testing Studies Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward 2 / 10

Introduction Challenges Potential Way Forward Conclusion Presentation Overview Current Techniques Trends Regression Increased Challenges Testing Interest Community Software Empirical Testing Studies Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward 2 / 10

Introduction Challenges Potential Way Forward Conclusion Presentation Overview Current Techniques Trends Regression Increased Challenges Testing Interest Community Solutions Software Empirical Testing Studies Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward 2 / 10

Introduction Challenges Potential Way Forward Conclusion Presentation Overview Current Techniques Trends Regression Increased Challenges Testing Interest Community Solutions Software Empirical Testing Studies Statistics Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward 2 / 10

Introduction Challenges Potential Way Forward Conclusion Presentation Overview Current Techniques Trends Regression Increased Challenges Testing Interest Community Solutions Software Empirical Testing Studies Statistics Medicine Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward 2 / 10

Introduction Challenges Potential Way Forward Conclusion Presentation Overview Current Techniques Trends Regression Increased Challenges Testing Interest Community Solutions Software Empirical Testing Studies Statistics Social Sciences Medicine Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward 2 / 10

Introduction Challenges Potential Way Forward Conclusion Presentation Overview Mutually Beneficial Sharing of All Artifacts Used in Experimentation Current Techniques Trends Regression Increased Challenges Testing Interest Community Solutions Software Empirical Testing Studies Statistics Social Sciences Medicine Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward 2 / 10

Introduction Challenges Potential Way Forward Conclusion Model of Regression Testing Start Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward 3 / 10

Introduction Challenges Potential Way Forward Conclusion Model of Regression Testing Coverage Start Report Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward 3 / 10

Introduction Challenges Potential Way Forward Conclusion Model of Regression Testing Selection Coverage Start Reduction Report Prioritization Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward 3 / 10

Introduction Challenges Potential Way Forward Conclusion Model of Regression Testing Original Test Suite Selection Coverage Start Reduction Report Prioritization Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward 3 / 10

Introduction Challenges Potential Way Forward Conclusion Model of Regression Testing Original Test Suite Selection Modified Coverage Start Reduction Test Report Suite Prioritization Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward 3 / 10

Introduction Challenges Potential Way Forward Conclusion Model of Regression Testing Original Test Suite Test Suite Selection Modified Execution Coverage Start Reduction Test Report Test Coverage Suite Prioritization Monitoring Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward 3 / 10

Introduction Challenges Potential Way Forward Conclusion Model of Regression Testing Program Adequacy Criterion Original Test Suite Test Suite Selection Modified Execution Coverage Start Reduction Test Report Test Coverage Suite Prioritization Monitoring Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward 3 / 10

Introduction Challenges Potential Way Forward Conclusion Model of Regression Testing Program Adequacy Criterion Original Test Suite Test Suite Selection Modified Execution Coverage Start Reduction Test Test Results Report Test Coverage Suite Prioritization Monitoring Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward 3 / 10

Introduction Challenges Potential Way Forward Conclusion Model of Regression Testing Program Adequacy Criterion Original Test Suite Test Suite Selection Modified Execution Coverage Start Reduction Test Test Results End Report Test Coverage Suite Prioritization Monitoring Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward 3 / 10

Introduction Challenges Potential Way Forward Conclusion Model of Regression Testing Use the Coverage Report During the Next Round of Regression Testing Program Adequacy Criterion Original Test Suite Test Suite Selection Modified Execution Coverage Start Reduction Test Test Results End Report Test Coverage Suite Prioritization Monitoring Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward 3 / 10

Introduction Challenges Potential Way Forward Conclusion Model of Regression Testing Use the Same Test Suite for the Next Round of Regression Testing Program Adequacy Criterion Original Test Suite Standard Repeat Test Suite Selection Modified Execution Coverage Start Reduction Test Test Results End Report Test Coverage Suite Prioritization Monitoring Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward 3 / 10

Introduction Challenges Potential Way Forward Conclusion Model of Regression Testing Make a New Test Suite for the Next Round of Regression Testing Program Adequacy Criterion Original Test Suite Standard Repeat Test Suite Selection Modified Execution Coverage Start Reduction Test Test Results End Report Test Coverage Suite Prioritization Monitoring Version Specific Repeat Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward 3 / 10

Introduction Challenges Potential Way Forward Conclusion Model of Regression Testing Practitioners are unwilling to use methods for “lack of empirical studies” [12] Program Adequacy Criterion Original Test Suite Standard Repeat Test Suite Selection Modified Execution Coverage Start Reduction Test Test Results End Report Test Coverage Suite Prioritization Monitoring Version Specific Repeat Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward 3 / 10