F10 5/18/2007 11:15:00 AM "C HALLENGES IN P ERFORMANCE T ESTING - PDF document

BIO PRESENTATION PAPER F10 5/18/2007 11:15:00 AM "C HALLENGES IN P ERFORMANCE T ESTING OF AJAX A PPLICATIONS " Rajendra Gokhale Aztecsoft International Conference On Software Test Analysis And Review May 14-18, 2007 Orlando, FL USA

Rajendra Gokhale Rajendra Gokhale Rajendra Gokhale currently heads the Research Division at Aztecsoft-Itest, the independent testing services division of Aztecsoft (http://www.aztecsoft.com). He has over twenty years of experience in software development and testing. His current work involves researching issues in the areas of Security and Performance Testing of Web- based applications. He holds a Masters degree in Computer Science and Engineering from I.I.T. Bombay.

Performance Performance Testing for AJAX Testing for AJAX Applications Applications - Rajendra Gokhale Research and Development

Agenda Agenda • Performance testing of traditional web based applications • How AJAX applications differ from traditional web applications • Google Suggest as an example AJAX application • Special challenges in performance testing AJAX applications Slide 2 http:/ / itest.aztecsoft.com

Performance Testing Phases Performance Testing Phases Phase1 Phase 2 Phase1 Phase 2 PT Strategy Planning Phase PT Execution Phase PT Strategy Planning Phase PT Execution Phase Needs Analysis and Lab setup Needs Analysis and Lab setup Technical Review Technical Review Performance Scenario Scripting Performance Scenario Scripting requirements and goals requirements and goals Designing Test Scenarios Data generation Designing Test Scenarios Data generation Workload Distribution Test execution Workload Distribution Test execution Tool Selection Monitoring and Analysis Tool Selection Monitoring and Analysis Reporting Infrastructure Reporting Infrastructure requirements requirements

Phase 1: Performance Test Strategy Planning Phase 1: Performance Test Strategy Planning • Needs Analysis • Technical review of application • Definition of application's Performance Requirements and Goals • High level PT road map (Planning) – Designing Test scenarios – Workload Distribution – Data Generation Needs – Test Tool Selection – Monitoring and Reporting Needs • Infrastructure Requirements and Environment

Phase 2: Performance Test Execution Phase Phase 2: Performance Test Execution Phase • Lab setup • Designing/ Creating PT Scripts – Scripting/ coding • Data Generation • Test Execution – Test script management/ test suite/ Harness + version control – Baseline and Benchmark tests • Monitoring and Analysis – Results analysis – Reporting – Performance Deliverables

Traditional Web Applications vs. AJAX Apps Traditional Web Applications vs. AJAX Apps

Traditional Request Response Model (S ynchronous) Traditional Request Response Model (S ynchronous) User Activity User Activity User Activity CLIENT n n Transmission Transmission o o i i s s s s a a Data Data i i t m t m a a D D s s n n TIME a a r r T T Server Side Processing Server Side Processing SERVER

AJAX Request Response Model (Asynchronous) AJAX Request Response Model (Asynchronous) Browser UI User Activity User Activity User Activity Input Display Input Display Input Display AJAX Engine (Client Side Processing) CLIENT n Transmission n o Transmission o i s i s s a Data s i a Data t m i a t m a D s D s n n a TIME a r T r T Server Side Processing Server Side Processing SERVER

Traditional Web Applications vs. AJAX Apps… … Traditional Web Applications vs. AJAX Apps Traditional Application AJAX application Requests are triggered by explicit Requests happen asynchronously – not user actions triggered by user action Browser-server interaction consists Requests for complete web-pages of requests for complete pages typically interspersed with many AJAX requests Response results in a complete The response is interpreted by AJAX page reload engine to update the DOM

Google S uggest – – A Case S A Case S tudy Google S uggest tudy • New interface to well known Google Search engine • Application guesses what user is intending to type and offers suggestions in real time • Example: as one types "bass," “Google Suggest” might offer a list of completions that include "bass shoes" or "bass guitar".

uggest … … uggest Google S Google S

Key Implementation Details of Google S uggest Key Implementation Details of Google S uggest • Asynchronous requests for phrase completions are sent to the server at regular time intervals • The time interval between these requests is determined dynamically and is a function of the latency observed by the client • No request is sent if the user has not typed anything since the last request was sent • The results obtained dynamically are cached (useful when the user erases something he has typed)

Challenges in Performance Testing of AJAX Application Challenges in Performance Testing of AJAX Application • Definition of Performance Goals and Metrics • User Modeling • Scripting and Load Simulation

Impact of AJAX on PT Phases Impact of AJAX on PT Phases Phase1 Phase 2 Phase1 Phase 2 PT Strategy Planning Phase PT Execution Phase PT Strategy Planning Phase PT Execution Phase Needs Analysis and Lab setup Needs Analysis and Lab setup Technical Review Technical Review Perform ance Scenario Scripting Perform ance Scenario Scripting requirem ents/ goals requirem ents/ goals Designing Test Scenarios Data generation Designing Test Scenarios Data generation W orkload Distribution Test execution W orkload Distribution Test execution Tool Selection Monitoring and Analysis Tool Selection Monitoring and Analysis Reporting Infrastructure Reporting Infrastructure requirements requirements

How AJAX impacts Performance Goals and Metrics How AJAX impacts Performance Goals and Metrics • Performance measurement metrics for AJAX applications different from traditional applications • Widely used traditional measures for web applications may be meaningless • Some performance goals (and metrics) relevant for AJAX applications inapplicable for non-Ajax applications – Optimization of AJAX engine – Measuring the Cross-browser performance

Google S uggest vs. Vanilla Google Google S uggest vs. Vanilla Google • Performance Goals – Google (Vanilla) • Check if server can handle ‘M’ search requests per seconds with average response time for each query not exceeding ‘x’ seconds – Google Suggest • Check if server can handle ‘M’ search requests per seconds with average response time for each query not exceeding ‘x’ seconds • Optimize AJAX engine so that it does not make excessive demands on the server, client or network

How AJAX impacts User Modeling How AJAX impacts User Modeling • Need to factor in more variables when modeling the load on AJAX application – Google (Vanilla) • average number of searches performed by a typical user in a given duration of time – Google Suggest • average number of searches performed by a typical user in a given duration of time • average length of search strings/ search string length distribution? (longer strings = more # intermediate Ajax requests) • response time distribution

Load S ting an u lation S cri d p i m Load S ting an u lation S cri d p m i • Script designers need to gain good understanding of the AJAX engine implementation • Debugging test scripts is harder • How to simulate AJAX engine? • Simulation of the AJAX engine involves hard design choices – Heavy Model – Light Model

Google S uggest vs. Vanilla Google Google S uggest vs. Vanilla Google • Developing the script for “Vanilla Google” was fairly straightforward • Implementing corresponding scripts for “Google Select” was much harder, primarily because – involved understand the complex algorithms for determining request frequencies – re-implementing the same algorithm • Developing test scripts for “Google Select” involved three tim es as much effort required for “Vanilla Google” .

THE END THE END

Performance Testing for AJAX- based Applications Rajendra Gokhale and Budhaditya Das Aztecsoft-itest Pune, India November 19th, 2006 1 www.aztecsoft.com