Learning objectives Understand the purpose of integration testing - PowerPoint PPT Presentation

Learning objectives • � Understand the purpose of integration testing – � Distinguish typical integration faults from faults that Integration and Component-based should be eliminated in unit testing – � Understand the nature of integration faults and how Software Testing to prevent as well as detect them • � Understand strategies for ordering construction and testing – � Approaches to incremental assembly and testing to reduce effort and control risk • � Understand special challenges and approaches for testing component-based systems (c) 2007 Mauro Pezzè & Michal Young Ch 21, slide 1 (c) 2007 Mauro Pezzè & Michal Young Ch 21, slide 2 What is integration testing? Integration versus Unit Testing Module test Integration test System test • � Unit (module) testing is a necessary foundation Specification: Module Interface specs, module Requirements – � Unit level has maximum controllability and visibility interface breakdown specification – � Integration testing can never compensate for inadequate unit testing Visible structure: Coding details Modular structure — none — (software architecture) • � Integration testing may serve as a process check – � If module faults are revealed in integration testing, Scaffolding Some Often extensive Some they signal inadequate unit testing required: – � If integration faults occur in interfaces between correctly implemented modules, the errors can be Looking for faults Modules Interactions, System in: compatibility functionality traced to module breakdown and interface specifications (c) 2007 Mauro Pezzè & Michal Young Ch 21, slide 3 (c) 2007 Mauro Pezzè & Michal Young Ch 21, slide 4

Integration Faults Example: A Memory Leak • � Inconsistent interpretation of parameters or values – � Example: Mixed units (meters/yards) in Martian Lander Apache web server, version 2.0.48 • � Violations of value domains, capacity, or size limits Response to normal page request on secure (https) port – � Example: Buffer overflow • � Side effects on parameters or resources static void ssl io filter disable(ap filter t *f) – � Example: Conflict on (unspecified) temporary file { bio filter in ctx t *inctx = f->ctx; • � Omitted or misunderstood functionality – � Example: Inconsistent interpretation of web hits No obvious error, but • � Nonfunctional properties inctx->ssl = NULL; Apache leaked memory – � Example: Unanticipated performance issues inctx->filter ctx->pssl = NULL; slowly (in normal use) or • � Dynamic mismatches quickly (if exploited for a } – � Example: Incompatible polymorphic method calls DOS attack) (c) 2007 Mauro Pezzè & Michal Young Ch 21, slide 5 (c) 2007 Mauro Pezzè & Michal Young Ch 21, slide 6 Example: A Memory Leak Example: A Memory Leak Apache web server, version 2.0.48 Apache web server, version 2.0.48 Response to normal page request on secure (https) port Response to normal page request on secure (https) port static void ssl io filter disable(ap filter t *f) static void ssl io filter disable(ap filter t *f) { bio filter in ctx t *inctx = f->ctx; { bio filter in ctx t *inctx = f->ctx; SSL_free(inctx -> ssl); SSL_free(inctx -> ssl); The missing code is for a Almost impossible to find inctx->ssl = NULL; inctx->ssl = NULL; structure defined and with unit testing. inctx->filter ctx->pssl = NULL; created elsewhere , inctx->filter ctx->pssl = NULL; (Inspection and some accessed through an dynamic techniques could } } opaque pointer. have found it.) (c) 2007 Mauro Pezzè & Michal Young Ch 21, slide 7 (c) 2007 Mauro Pezzè & Michal Young Ch 21, slide 8

Maybe you’ve heard ... Translation... • � Yes, I implemented • � Yes, I implemented • � I didn’t think at all � module A � , but I � module A � , but I about the strategy didn’t test it didn’t test it for testing. I didn’t thoroughly yet. It thoroughly yet. It design � module A � for will be tested along will be tested along testability and I with � module A � when with � module A � when didn’t think about that’s ready. that’s ready. the best order to build and test modules � A � and � B � . (c) 2007 Mauro Pezzè & Michal Young Ch 21, slide 10 (c) 2007 Mauro Pezzè & Michal Young Ch 21, slide 11 Integration Plan + Test Plan Big Bang Integration Test An extreme and desperate approach: • � Integration test Test only after integrating all modules plan drives and is ... driven by the + � Does not require scaffolding ... project “build • � The only excuse, and a bad one plan” Build Plan Test Plan - � Minimum observability, diagnosability, efficacy, – � A key feature of the feedback system architecture ... - � High cost of repair and project plan System Architecture • � Recall: Cost of repairing a fault rises as a function of time between error and repair (c) 2007 Mauro Pezzè & Michal Young Ch 21, slide 12 (c) 2007 Mauro Pezzè & Michal Young Ch 21, slide 13

Structural and Functional Strategies Top down . • � Structural orientation: Top Modules constructed, integrated and tested based on a hierarchical project structure – � Top-down, Bottom-up, Sandwich, Backbone • � Functional orientation: stub A stub B stub C Modules integrated according to application characteristics or features Working from the top level (in terms of “use” or “include” relation) toward the bottom. – � Threads, Critical module No drivers required if program tested from top-level interface (e.g. GUI, CLI, web app, etc.) (c) 2007 Mauro Pezzè & Michal Young Ch 21, slide 14 (c) 2007 Mauro Pezzè & Michal Young Ch 21, slide 15 Top down .. Top down ... Top Top A A stub B stub C B C Write stubs of called or As modules replace used modules at each stubs, more stub X stub Y step in construction stub X stub Y functionality is testable (c) 2007 Mauro Pezzè & Michal Young Ch 21, slide 16 (c) 2007 Mauro Pezzè & Michal Young Ch 21, slide 17

Top down ... complete Bottom Up . Top Starting at the leaves of the A Driver B C “uses” hierarchy, we never need stubs ... until the program is complete, and all functionality can be X Y X tested (c) 2007 Mauro Pezzè & Michal Young Ch 21, slide 18 (c) 2007 Mauro Pezzè & Michal Young Ch 21, slide 19 Bottom Up .. Bottom Up ... Driver ... but we must Driver A ... an intermediate Driver construct drivers for module replaces a each module (as in driver, and needs its unit testing) ... own driver ... X X Y Y (c) 2007 Mauro Pezzè & Michal Young Ch 21, slide 20 (c) 2007 Mauro Pezzè & Michal Young Ch 21, slide 21

Bottom Up .... Bottom Up ..... Driver Driver Driver Driver Driver A B A B C ... so we may have several working X Y subsystems ... X Y (c) 2007 Mauro Pezzè & Michal Young Ch 21, slide 22 (c) 2007 Mauro Pezzè & Michal Young Ch 21, slide 23 Bottom Up (complete) Sandwich . Top Top (parts) A Stub B C C Working from the ... that are eventually extremes (top and integrated into a X Y Y bottom) toward center, single system. we may use fewer drivers and stubs (c) 2007 Mauro Pezzè & Michal Young Ch 21, slide 24 (c) 2007 Mauro Pezzè & Michal Young Ch 21, slide 25

Sandwich .. Thread ... Top (more) Top A A C C A “thread” is a portion of several Sandwich integration modules that together provide a is flexible and user-visible program feature. X Y adaptable, but X complex to plan (c) 2007 Mauro Pezzè & Michal Young Ch 21, slide 26 (c) 2007 Mauro Pezzè & Michal Young Ch 21, slide 27 Thread ... Thread ... Top Top A A B C B C Integrating one As in sandwich thread, then another, integration testing, we etc., we maximize X Y can minimize stubs X Y visibility for the user and drivers, but the integration plan may be complex (c) 2007 Mauro Pezzè & Michal Young Ch 21, slide 28 (c) 2007 Mauro Pezzè & Michal Young Ch 21, slide 29