Automatic Program Instrumentation to the Rescue! Gregory M. Kapfhammer Department of Computer Science Allegheny College Mary Lou Soffa Department of Computer Science University of Virginia Automatic Program Instrumentation to the Rescue!, RICSS, February 9, 2007 – p. 1/15
What is My Program Doing? Contribution : An instrumentation framework to support testing, analysis, debugging, and understanding Automatic Program Instrumentation to the Rescue!, RICSS, February 9, 2007 – p. 2/15
Potential Probe Locations Program and Test Suite Database Manager Java JDBC Driver Java Virtual Machine Virtual Machine Operating System Operating System Instrumentation probes can be placed in many locations How can we “ best ” capture the behavior of a program? Is it possible to automatically introduce the probes? What tools already exist ? What approach is the most efficient ? Automatic Program Instrumentation to the Rescue!, RICSS, February 9, 2007 – p. 3/15
Understanding Static Instrumentation Insert the probes into the source code or bytecode Instrumentation occurs before program execution Less flexible if a program regularly changes What is the impact on space overhead ? Aspect-oriented programming versus bytecode instrumentation Automatic Program Instrumentation to the Rescue!, RICSS, February 9, 2007 – p. 4/15
Dynamic Instrumentation P Input Output Byte Code Fast? Interpreter? Virtual Machine Adaptive? JIT? methodA testOne Class Loader Program Heap Native Code Cache Stack Perform dynamic instrumentation at established interface(s) Automatic Program Instrumentation to the Rescue!, RICSS, February 9, 2007 – p. 5/15
Constructing Dynamic Call Trees A B B H C F I D E G H G Number of Nodes = 13, Number of Edges = 12 Automatic Program Instrumentation to the Rescue!, RICSS, February 9, 2007 – p. 6/15
Using Calling Context Trees A B H C F I D E G Number of Nodes = 9, Number of Edges = 10 Automatic Program Instrumentation to the Rescue!, RICSS, February 9, 2007 – p. 7/15
Static Instrumentation Time FF PI RM ST TM GB All � sec � 10 8.687 8 Time 5.583 6 5.169 Instrumentation 4.391 4.404 4.396 4.394 4 2 Static FF PI RM ST TM GB All Application Instrumentation never takes longer than nine seconds Automatic Program Instrumentation to the Rescue!, RICSS, February 9, 2007 – p. 8/15
Space Overhead of the Instrumentation As A ZIP GZIP PACK � 48887 50000 � bytes 45730 45650 40000 Size 30000 20000 Application 14582 11424 10000 6058 ZIP GZIP PACK Compression Technique � TM � Increase in the number of bytecodes is substantial Automatic Program Instrumentation to the Rescue!, RICSS, February 9, 2007 – p. 9/15
Size of the Instrumented Applications Compr Tech Before Instr (bytes) After Instr (bytes) None 29275 887609 Zip 15623 41351 Gzip 10624 35594 Pack 5699 34497 Average static size across all case study applications Compressed the bytecodes with general purpose techniques Specialized compressor nicely reduces space overhead Automatic Program Instrumentation to the Rescue!, RICSS, February 9, 2007 – p. 10/15
Size of the Instrumentation Probes Compression Technique Probe Size (bytes) None 119205 Zip 40017 Gzip 34982 Pack 35277 420% average increase in space overhead! Why? Reflection vs. extra bytecode instructions Is this increase in space overhead acceptable ? Automatic Program Instrumentation to the Rescue!, RICSS, February 9, 2007 – p. 11/15
Static and Dynamic Time Overhead Norm Sta � CCT Sta � DCT Dyn � CCT Dyn � DCT 14 11.916 12 11.245 10 � sec � 8.796 8.02 8 6.933 Time 6 TCM 4 2 Norm Sta � CCT Sta � DCT Dyn � CCT Dyn � DCT Instrumentation Technique Tree Type � TM � � What trends can you find in this graph? Which tree and instrumentation technique would you pick? Automatic Program Instrumentation to the Rescue!, RICSS, February 9, 2007 – p. 12/15
Test Execution Time Overhead Percent Increase ( % ) Instr Tech Tree Type TCM Time (sec) Static CCT 7.44 12.5 Static DCT 8.35 26.1 Dynamic CCT 10.17 53.0 Dynamic DCT 11.0 66.0 Normal average testing time of 6 . 62 seconds Which tree and instrumentation technique is most efficient ? Which configuration would you select? Automatic Program Instrumentation to the Rescue!, RICSS, February 9, 2007 – p. 13/15
Average Tree Storage Time Tree Type Tree Representation Tree Storage Time (msec) CCT Binary 144.9 DCT Binary 1011.72 CCT XML 408.17 DCT XML 2569.22 Strengths and weaknesses of tree representations Is it ever better to store the tree in XML? Which configuration would you use? Automatic Program Instrumentation to the Rescue!, RICSS, February 9, 2007 – p. 14/15
Conclusions and Future Work Automatic program instrumentation can save the day ! A complete framework for recording call trees Useful applications : test coverage monitoring, performance analysis, regression test suite reduction Characterizing the DCT and CCT for object-oriented programs Automatic visualization of method input and output What are your suggestions? I value your comments and participation! Automatic Program Instrumentation to the Rescue!, RICSS, February 9, 2007 – p. 15/15
Recommend
More recommend
