F4 Friday, October 31, 2003 10:00 AM W EB S ERVICES : O VERVIEW AND - - PDF document

BIO PRESENTATION

F4

Friday, October 31, 2003 10:00 AM

WEB SERVICES: OVERVIEW AND

ING STRATEGY

TEST

Alan Newman Spirent Communications

International Conference On Software Testing Analysis & Review October 27-31, 2003 San Jose, CA USA

Alan New man

Alan Newman is currently Director of Product Marketing at Spirent Communications where he leads the Enterprise marketing team. Prior to Spirent, Alan worked at Cisco in a variety of Engineering and Marketing roles. Alan has an MBA and an MS in Computer & Information Science.

Analyze Assure Accelerate

Web Services: Overview and Testing Strategy

Presented at STAR West October 31, 2003 Alan Newman Director, Product Marketing alan.newman@spirentcom.com

Analyze Assure Accelerate

About Alan Newman

• Experience as a:

Software QA Engineer (HP) Software QA Manager (HP & Octel) Development Manager (Octel & Cisco) Program Manager (Cisco) Marketing Manager (Cisco & Spirent) What’s next?

• Currently working at Spirent Communications leading the

Web Services & Application Testing Marketing Team

• alan.newman@spirentcom.com

Analyze Assure Accelerate

Agenda

• Web Services Overview
• Testing Strategy
• A Testing Solution

Analyze Assure Accelerate

Web Services Overview

Analyze Assure Accelerate

Web Services Principles

• Systems connect through the Internet (or

Intranet) to access services

• Employ a common protocol to exchange

information (SOAP)

• Use a common language to share information

(XML)

• Provide a simple way to find services (UDDI)

Analyze Assure Accelerate

Web Services Components

• SOAP: Simple Object Access Protocol

Protocol for passing XML encoded data Usually transported over HTTP/S

• UDDI: Universal Description, Discovery and

Integration Service

Mechanism for dynamically finding web services-

“Yellow Pages for Web Services”

Provider publishes; Client searches

• WSDL: Web Services Description Language

XML document describing what a Web Service can

do, where it resides and how to invoke it

Analyze Assure Accelerate

Web Services Overview

Service Requestor Service Provider Service Registry (UDDI)

Find Publish Service Request Service

SOAP over HTTP(S)

Service Description (WSDL)

Analyze Assure Accelerate

Service Oriented Architecture

Web Service Web Services Client

Analyze Assure Accelerate

Service Oriented Architecture

Web Services Web Services Client

Analyze Assure Accelerate

Service Oriented Architecture: Many to Many

Analyze Assure Accelerate

Web Services Security

• Transport Security

HTTPS: Simple to implement and widely available

• WS-Security

Standard proposed by IBM, Microsoft, and VeriSign Support for multiple signature formats and

encryption technologies

Just starting to be implemented

Analyze Assure Accelerate

Workflows and Business Processes

• Single Web Service provides request &

response or RPC functionality

• Sequence of multiple Web Services to

accomplish a task is called a Workflow or Choreography

• Multiple competing Workflow standards

proposals:

MSFT & IBM: Business Process Execution

Language for Web Services (BPEL4WS)

BEA, Sun, others: Web Services Choreography

Interface

Analyze Assure Accelerate

Source: IDC

Web Services Adoption Forecast by Enterprise Size

Pervasive use in non

• traditional

devices

Within the firewall Contained external users Public dynamic search & use

2006 2008

Simplified application integration Increased developer productivity Simplified business partner connectivity Richer application functionality Subscription

• based services

Casual/ad -hoc use of services New business models possible Commoditization

• f software

2002 2004

Analyze Assure Accelerate

XML Increases to 25% of Network Traffic in 2006

* Excluding XML Traffic

• Source: ZapThink, LLC 2003

Analyze Assure Accelerate

Business Opportunities for Web Services

• Information sharing with consumers or

businesses

News, weather, financial information, web searches

(Google), etc.

• Business integration- transactions for a fee

Airline (SABRE) & hotel reservations, auctions, etc.

• Business process externalization- dynamically

link partners into business processes

Purchasing, manufacturing, banking, etc.

Analyze Assure Accelerate

Web Services Requirements

• Availability- Requests can come at any time
• Capacity- Number of simultaneous requests

can be huge

• Performance- Need to insure fast time to the

end user, so the Web Service must respond quickly

Analyze Assure Accelerate

The Need for Capacity

The capacity to scale to unpredictable levels in a production Web services deployment is a requirement for

• rganizations that are planning to connect enterprise

applications, customers, suppliers, and potentially business partners. Without the ability to scale quickly to meet the requests in a growing Web services environment, large-scale Web services deployments are doomed.”

Source: ”XML Networking: Moving up the Stack” The HTRC Group, 2003

Analyze Assure Accelerate

Web Services Testing Strategy

Analyze Assure Accelerate

Similarities between Web Services & Web Site Testing

• Many “users” in remote locations
• Multiple “user types”- different browsers & different

applications

• Need to capture dynamic data in a response and use in

subsequent request (Session ID, search results, etc.)

• Often 3-Tier Architecture: Web Server, Application

Server, Database Server

• Use HTTP/S for transport
• Network affects performance- System Test needs to

test Network infrastructure

• Testers have good job security! ☺

Analyze Assure Accelerate

Differences between Web Services & Web Site Testing

• Web Services “user” is an application- not a person
• Web Services “Think Time” is application processing

time

• Web Services can use other transports besides HTTP-

JMS, SMTP, etc.

• Harder to spot bugs in a SOAP Message (XML) than a

Web Page (Browser)

• Even for a simple interaction you need a test tool- hard

to do “manually”

• Performance is even more important, because there

may be a real person interacting with the application that is making the Web Services request

Analyze Assure Accelerate

Testing Process for Web Services

• Define the scope of testing
• Plan the functional tests: test cases, time &

resources

• Plan the system tests: test cases, time & resources
• Run the functional and system tests; tell the

developers to fix the bugs

• Run the functional and system tests; tell the developers to fix the

bugs

• Run the functional and system tests; tell the developers to fix the bugs
• Run the functional and system tests; tell the developers to fix the bugs
• Run the functional and system tests; tell the developers to fix the bugs
• Run the functional and system tests; tell the developers to fix the bugs
• Run the functional and system tests; tell the developers to fix the bugs
• Write a test summary report

Analyze Assure Accelerate

Web Services Testing: Define the Scope of Testing

• Which Web Service(s) will you test?
• Will you use UDDI to get the WSDL or is that a

given?

• What security mechanism will you use?

HTTPS? WS-Security?

• Is there a Workflow, or are you testing a simple

Request/Response service?

Analyze Assure Accelerate

Web Services Testing: Plan the Functional Tests

• Define the parameters to vary in each request
• Specify content of each response to verify
• Negative testing:

Incorrect parameters Badly formed SOAP messages

• UDDI: Verify publish and find- positive &

negative test cases

• Workflows: Test valid and invalid Workflows

Analyze Assure Accelerate

Typical Web Services Topology

Web Server Firewall Load Balancer Database Server Application Server Router

Web Services Providers & Service Registries

Switch

Web Services Clients

Analyze Assure Accelerate

Typical Functional Test Topology

Web Server Database Server Application Server

Web Services Functional Test Tool Web Services Providers & Service Registries

Switch

Analyze Assure Accelerate

“Economical” Functional Test Topology

Web Services Functional Test Tool Web Services Providers & Service Registries

Switch

Analyze Assure Accelerate

Functional Test Tool: Some Selection Criteria

• Easily generate Web Service requests from UDDI look-up

responses

• Extensive verification of SOAP message response
• Quickly (automatically?) generate negative tests
• Workflow support
• Easy to generate tests
• Flexibly reporting
• Cost

Analyze Assure Accelerate

Web Services Testing: Plan the System Tests

• Build a “model” of the real world:

Estimate the load per Web Service that will be seen in

production

Estimate the mix of connection speeds, network latency and

network packet loss

Estimate the requesting application delays between requests

(“think time”)

• Do performance and reliability tests using the “model”
• Test at and beyond “model” limits:

Insure graceful degradation Verify no security flaws exposed at heavy loads

• Verify system can withstand security attacks

Analyze Assure Accelerate

Typical Web Services Topology

Web Server Firewall Load Balancer Database Server Application Server Router

Web Services Providers & Service Registries

Switch

Web Services Clients

Analyze Assure Accelerate

Typical System Test Topology: Lots of PCs

Web Server Firewall Load Balancer Database Server Application Server Router

Web Services Clients Web Services Providers & Service Registries

Switch

Analyze Assure Accelerate

Typical System Test Topology: Appliance Solution (like Spirent’s Avalanche 2200)

Web Server Firewall Load Balancer Database Server Application Server Router

Web Services Clients Web Services Providers & Service Registries

Switch

Analyze Assure Accelerate

System Test Tool: Some Selection Criteria

• “Fill-in” requests with supplied data (Stock ticker, book

to buy, etc.)

• Capture dynamic data, and use in subsequent request

(Conversation ID, PO #, etc.)

• Generate load at and beyond architected limits
• Simulate realistic network conditions including

connection speeds, link latency and packet loss

• Generate security attacks at the same time as normal

Web Services traffic

• Sanity checking of responses
• Isolate problems to Network or Web Service
• Cost per simulated Web Services requestor

Analyze Assure Accelerate

Case Study: Web Services Based Peer to Peer-to-Peer Product

Analyze Assure Accelerate

Product Overview

• Product provides collaboration software for

enterprises over the Internet and within the enterprises’ Intranets.

• Users share files, calendars, send instant

messages, and have forum discussions

• SOAP is used for communication between

clients and from client to server

Analyze Assure Accelerate

Identified the Need for Performance Testing (a little late!)

• Company was deluged with support calls from

large customers complaining that the performance of their product had declined in their latest release, which added multiple new features

• Company decided to do performance testing!
• Had difficulty generating sufficient stress from

PC-based software tools, so looked for a tool to generate the load

Analyze Assure Accelerate

Solution: Appliance-based Test Tool Generates the Heavy Load Required

Web Services Clients Peer-to-Peer Application

Switch

Analyze Assure Accelerate

Results

• Identified non-linear performance that is only

visible at extremely heavy load (could not generate with PC-based products)

• Isolated bottleneck in the software and fixed it
• Customers are happy with the updated product!

Analyze Assure Accelerate

Case Study: XML Firewall Evaluation

Analyze Assure Accelerate

XML Firewall Overview

• An XML Firewall acts as an XML proxy that intercepts

XML traffic

• Povides authentication services by interfacing with

LDAP directories, HTTP basic authorization and limiting access to specific IP addresses or ranges

• It also logs incoming and outgoing messages to

provide non-repudiation

• Unlike conventional firewalls that examine packets, the

XML Firewall looks at the contents of SOAP messages

Analyze Assure Accelerate

XML Firewall Evaluation

• Company was deploying a Web Service on the Internet
• Identified the need for an XML firewall, and wanted to

evaluate multiple vendors’ product

• Wanted to test security features and do performance &

stress testing testing

• Concerned that an attack might crash the XML firewall

and expose the Web Service directly to the Internet

• Web Service was not ready for testing, so needed to

simulate both clients and the web service

Analyze Assure Accelerate

Solution: Appliance-based Clients and Web Services Providers

Web Services Clients Web Services Providers

XML Firewall Under Test

Load Balancer Router

Switch

Analyze Assure Accelerate

Summary

• Web Services is an important, emerging

technology

• SOAP messages are encoded using XML, and

usually transported over HTTP/S

• UDDI repository used for publishing and finding

services

• WSDL describes how to access a service
• Workflows/Choreography defines the sequence
• f actions to perform a business process

Analyze Assure Accelerate

Thank you!