Open Source Test Automation: Riding the Second Wave - - PDF document

¡ ¡

¡ ¡ ¡ K2 ¡

Keynote ¡ 5/4/16 ¡10:00 ¡ ¡ ¡ ¡ ¡ ¡ ¡

Open ¡Source ¡Test ¡Automation: ¡Riding ¡ the ¡Second ¡Wave ¡ ¡

Presented ¡by: ¡

¡

David ¡Dang ¡

Zenergy ¡Technologies ¡ ¡

Brought ¡to ¡you ¡by: ¡ ¡ ¡ ¡

¡

¡ ¡

350 ¡Corporate ¡Way, ¡Suite ¡400, ¡Orange ¡Park, ¡FL ¡32073 ¡ ¡ 888-­‑-­‑-­‑268-­‑-­‑-­‑8770 ¡·√·√ ¡904-­‑-­‑-­‑278-­‑-­‑-­‑0524 ¡-­‑ ¡info@techwell.com ¡-­‑ ¡http://www.stareast.techwell.com/ ¡ ¡ ¡

¡

¡ ¡

¡ ¡

David ¡Dang ¡

Zenergy ¡Technologies ¡

¡

For ¡more ¡than ¡seventeen ¡years, ¡David ¡Dang ¡has ¡been ¡a ¡leader ¡in ¡the ¡test ¡ automation ¡industry. ¡As ¡VP ¡of ¡automated ¡solutions ¡for ¡Greensboro, ¡NC-­‑based ¡ Zenergy ¡Technologies, ¡David ¡spearheads ¡the ¡development ¡of ¡advanced ¡ frameworks ¡that ¡emphasize ¡reusability ¡and ¡reduce ¡maintenance ¡efforts. ¡He ¡is ¡an ¡ expert ¡in ¡all ¡major ¡commercial ¡automation ¡tools ¡as ¡well ¡as ¡open ¡source ¡tools ¡ such ¡as ¡Selenium ¡and ¡Jenkins. ¡On ¡the ¡mobile ¡front, ¡David ¡uses ¡advanced ¡ concepts ¡to ¡design ¡optimal ¡frameworks ¡using ¡mobile ¡automation ¡toolsets ¡ including ¡Perfecto ¡and ¡Appium. ¡In ¡addition ¡to ¡his ¡high-­‑level ¡consulting ¡ engagements ¡for ¡Zenergy’s ¡clients, ¡David ¡is ¡in ¡high ¡demand ¡as ¡a ¡presenter ¡at ¡ major ¡software ¡quality ¡assurance ¡and ¡testing ¡conferences. ¡Read ¡more ¡about ¡ David ¡and ¡Zenergy ¡at ¡zenergytechnologies.com. ¡ ¡ ¡

¡

¡

Open Source Test Automation: Riding the Second Wave

StarEast 2016 Presented By: David Dang

A Testing Partner You Can Trust

Overview

• What is an open source test automation framework/tool?
• Why is it gaining in popularity?
• First wave of open source frameworks/tools
• Current open source frameworks/tools
• Strategies for success
• Should you catch the second wave?
• Q & A

What is Open Source?

• Software for which the original source code is made freely available

and may be redistributed and modified

• Open source code is meant to be shared within the community.

Enhancements, bug fixes, branching of the code are made by the community

• Currently on www.opensourcetesting.org/functional.php, there are

around 135 open source test automation tools/frameworks

Why is Open Source Test Automation Gaining Popularity?

• FREE! Companies have to do more with less
• More applications are being converted to web-based
• Development teams are pushing for open source test

automation

• Industry trends include more open source in all areas

(Eclipse, Java, Jenkins, OpenStack, Drupel, etc.)

First Wave of Open Source Test Automation Tools/Frameworks

• FitNesse: Operates below the user interface level. The user

provides various inputs to the application and determines if the correct results are returned

• Ruby Watir: Ruby libraries used to automate web browsers. It

allows users to automate clicks, fill in forms, press buttons, etc.

• Selenium RC: Writes automated web application UI tests. RC comes

in two parts: server acts as an HTTP proxy for web, and client libraries interact with the web elements

Why the First Wave Tapered Off

• Tools were too fragmented, difficult to integrate into complete

solutions.

• Level of instrumentation increased the technical challenge and

decreased reliability

• Learning curve tended to be higher due to overly complex and less

refined solutions

• IT landscape wasn’t ready to embrace open source at the time

Why the First Wave Tapered Off

• FitNesse
• Does not run at the UI level; runs at the source code level
• Fixtures must directly access the source code
• Watir
• Only available for Ruby
• Each browser had a different library
• Selenium RC:
• Required heavy instrumentation
• Had a large, clunky API
• Effectiveness and reliability limited by use of Javascript injection

Current Open Source Test Automation Tools/Frameworks

• Selenium WebDriver: Drives a browser directly using its built in

support for automation

• Cucumber: Runs automated acceptance tests written in a behavior-

driven development (BDD) style. It uses Ruby to create step definition

• Robot Framework: Generic test automation framework based on

keyword-driven testing approach. The core framework is implemented using Python

Sample of Selenium WebDriver

Sample of Cucumber

Sample of Robot Framework

Why Second Wave Tools are More Popular

• WebDriver laid foundation for numerous additional tools
• Increased support from well-known organizations
• WebDriver
• Developed by Thought Works
• Microsoft is developing a driver for their new Edge browser
• W3C has adopted it as the standard for browser automation
• Appium – Sauce Labs
• Supports popular methodologies (Ex. Cucumber BDD)
• Cleaner abstractions and more concise APIs
• Less instrumentation required
• Better integration

Selenium RC vs Selenium WebDriver: Architecture

Selenium RC Server Test Code (e.g.

Java, C#, Python, etc.)

Web Browser Selenium RC Selenium Core Application Client Application Server Browser Launcher HTTP Proxy 1 2 3 4 5 Selenium WebDriver Selenium WebDriver Test Code (e.g.

Java, C#, Python, etc.)

Web Browser

(IE, Firefox, Chrome, etc.)

Application Server

Selenium RC vs Selenium WebDriver: API

Selenium RC Selenium WebDriver

WebDriver: The Foundation of the Second Wave

• As the industry standard tool for

browser automation, countless tools are being created that extend or integrate with Selenium WebDriver

• Tools like Appium and Protractor

extend WebDriver for use in other technologies

• Tools like Cucumber integrate with

WebDriver so it can be used with methodologies such as BDD

Application Under Test Selenium WebDriver

Selenium vs Packaged Tools: Feature Comparison

Feature Packaged Tools Selenium Environment Integrated Development IDE (Eclipse, IntelliJ, RubyMine) Browser Compatibility Limited (e.g., Firefox, IE, Chrome) Firefox, IE, Chrome, Safari, Opera, Edge, HtmlUnit, PhantomJS, and others Language support Limited (e.g., VBScript

• nly or .NET only)

Java, Javascript, C#, Python, Ruby, and others Reporting Included Separate package (ReportNG, Allure) or custom Test Management Integrated Custom

Selenium vs Packaged Tools: Feature Comparison

Feature Packaged Tools Selenium Continuous Integration Integrated Separate package (Jenkins, Bamboo) Support 800 number The Community… Object Management Included Separate package or custom Test Control Included Separate package (TestNG, phpUnit, RSpec)

Strategies for Success

• People
• Requires more technical resources
• Resources must be comfortable with object-oriented

programming

• Resources must understand framework/tool configuration and

environment setup

• Resources must be able to diagnose and fix technical issues

without vendor support

• May have to identify and even fix bugs in the tool
• May need to contribute to the community with new features
• May need to work more closely with development team

Strategies for Success

• Technology
• Open source requires more time and effort to get started and maintain
• No official support, only the community
• Components are not built-in; they must use other libraries or be

developed from scratch

• Some technologies are not well supported by open source tools
• Some technologies have features that lack support from otherwise

useful open source tools

Strategies for Success

• Process
• Requires more dedicated resources
• Tool should fit into your existing process
• For e.g., if you are a .NET shop, you will often find less of a learning curve

with a tool that offers .NET as an option

• A tool like Cucumber will have a greater return if it is adopted early in the

SDLC

• Requires a detailed plan for implementation and maintenance
• Framework will be required including standards and page object models

Framework Development Process

Gather Information Architecture Design Core Framework Implementation Ongoing Development

Ongoing Maintenance

Should You Catch the Second Wave?

• People
• Is your team comfortable with object-oriented programming?
• Are they comfortable dealing with technical challenges with

minimal support?

• Are they able to contribute to the tool with bug fixes and

enhancements?

• Do you have a plan in place for training and mentoring?

Should You Catch the Second Wave?

• Process
• Do you have the resources to dedicate toward ensuring

automation is successful?

• Do you have a plan for creating a stable, maintainable

framework?

• Does open source make sense from a budget standpoint?
• Is open source a good fit for your overall IT strategy?

Should you Catch the Second Wave?

• Technology
• Have you chosen a tool that is appropriate for your platform?
• How active is the community? Is there somewhere to turn for

help?

• Is the tool updated frequently and responsive to platform

changes?

• Does investing in open source fit into your organization’s

technology road map?

• Are you comfortable supporting multiple tools sets?

Summary

• WebDriver is the foundation of the current wave of open

source automation

• People, process and technology are the keys to success
• Moving to open source is not a decision to take lightly

Questions?