Automatic Program Instrumentation to the Rescue! Gregory M. - - PowerPoint PPT Presentation

Automatic Program Instrumentation to the Rescue!

Gregory M. Kapfhammer Department of Computer Science Allegheny College Mary Lou Soffa Department of Computer Science University of Virginia

Automatic Program Instrumentation to the Rescue!, RICSS, February 9, 2007 – p. 1/15

What is My Program Doing?

Contribution: An instrumentation framework to support

testing, analysis, debugging, and understanding

Automatic Program Instrumentation to the Rescue!, RICSS, February 9, 2007 – p. 2/15

Potential Probe Locations

Program and Test Suite Java Virtual Machine Operating System Database Manager Java Virtual Machine Operating System JDBC Driver Instrumentation probes can be placed in many locations How can we “best” capture the behavior of a program? Is it possible to automatically introduce the probes? What tools already exist? What approach is the most efficient?

Automatic Program Instrumentation to the Rescue!, RICSS, February 9, 2007 – p. 3/15

Understanding Static Instrumentation

Insert the probes into the source code or bytecode Instrumentation occurs before program execution Less flexible if a program regularly changes What is the impact on space overhead? Aspect-oriented programming versus bytecode instrumentation

Automatic Program Instrumentation to the Rescue!, RICSS, February 9, 2007 – p. 4/15

Dynamic Instrumentation

Program Stack Native Code Cache Fast? Interpreter? Machine Virtual JIT? Adaptive? Class Loader Heap

methodA testOne

Input Output Byte Code

P

Perform dynamic instrumentation at established interface(s)

Automatic Program Instrumentation to the Rescue!, RICSS, February 9, 2007 – p. 5/15

Constructing Dynamic Call Trees

B B G G H I H A C F D E

Number of Nodes = 13, Number of Edges = 12

Automatic Program Instrumentation to the Rescue!, RICSS, February 9, 2007 – p. 6/15

Using Calling Context Trees

B G H I A C F D E

Number of Nodes = 9, Number of Edges = 10

Automatic Program Instrumentation to the Rescue!, RICSS, February 9, 2007 – p. 7/15

Static Instrumentation Time

FF PI RM ST TM GB All Application 2 4 6 8 10 Static Instrumentation Time sec FF PI RM ST TM GB All 4.391 4.404 4.396 4.394 5.169 5.583 8.687

Instrumentation never takes longer than nine seconds

Automatic Program Instrumentation to the Rescue!, RICSS, February 9, 2007 – p. 8/15

Space Overhead of the Instrumentation

ZIP GZIP PACK Compression Technique TM 10000 20000 30000 40000 50000 Application Size bytes

• ZIP

GZIP PACK 14582 11424 6058 48887 45730 45650

A As

Increase in the number of bytecodes is substantial

Automatic Program Instrumentation to the Rescue!, RICSS, February 9, 2007 – p. 9/15

Size of the Instrumented Applications

Compr Tech Before Instr (bytes) After Instr (bytes)

None 29275 887609 Zip 15623 41351 Gzip 10624 35594 Pack 5699 34497

Average static size across all case study applications Compressed the bytecodes with general purpose techniques Specialized compressor nicely reduces space overhead

Automatic Program Instrumentation to the Rescue!, RICSS, February 9, 2007 – p. 10/15

Size of the Instrumentation Probes

Compression Technique Probe Size (bytes)

None 119205 Zip 40017 Gzip 34982 Pack 35277

420% average increase in space overhead! Why? Reflection vs. extra bytecode instructions Is this increase in space overhead acceptable?

Automatic Program Instrumentation to the Rescue!, RICSS, February 9, 2007 – p. 11/15

Static and Dynamic Time Overhead

Norm Sta CCT Sta DCT Dyn CCT Dyn DCT Instrumentation Technique

• Tree

Type TM 2 4 6 8 10 12 14 TCM Time sec Norm Sta CCT Sta DCT Dyn CCT Dyn DCT 6.933 8.02 8.796 11.245 11.916

What trends can you find in this graph? Which tree and instrumentation technique would you pick?

Automatic Program Instrumentation to the Rescue!, RICSS, February 9, 2007 – p. 12/15

Test Execution Time Overhead

Instr Tech Tree Type TCM Time (sec) Percent Increase (%)

Static CCT 7.44 12.5 Static DCT 8.35 26.1 Dynamic CCT 10.17 53.0 Dynamic DCT 11.0 66.0

Normal average testing time of 6.62 seconds Which tree and instrumentation technique is most efficient? Which configuration would you select?

Automatic Program Instrumentation to the Rescue!, RICSS, February 9, 2007 – p. 13/15

Average Tree Storage Time

Tree Type Tree Representation Tree Storage Time (msec)

CCT Binary 144.9 DCT Binary 1011.72 CCT XML 408.17 DCT XML 2569.22

Strengths and weaknesses of tree representations Is it ever better to store the tree in XML? Which configuration would you use?

Automatic Program Instrumentation to the Rescue!, RICSS, February 9, 2007 – p. 14/15

Conclusions and Future Work

Automatic program instrumentation can save the day! A complete framework for recording call trees Useful applications: test coverage monitoring, performance analysis, regression test suite reduction Characterizing the DCT and CCT for object-oriented programs Automatic visualization of method input and output What are your suggestions? I value your comments and participation!

Automatic Program Instrumentation to the Rescue!, RICSS, February 9, 2007 – p. 15/15