Mobile App T esting with Xamarin T est Cloud Martin imeek - - PowerPoint PPT Presentation

Mobile App T esting with Xamarin T est Cloud

Martin Šimeček martin.simecek@microsoft.com @deeedx

Mobile Quality Challenges

• Device Fragmentation
• App Complexity
• Fast Release Cycles
• Short Sessions
• High Mobile User Expectations

Results! The Differences in Time to Value

19K unique

Android devices available on the market

20 iPhone and

iOS configuration combinations on the market In the United States, a business must test on

134 different devices 80% of devices in use

to cover

The Manual Approach

• Limited device reach, based on user availability
• Slower and less accurate
• Delayed feedback collection
• Inconsistent

The Device Remoting Approach

• Prone to human error
• Limited test complexity
• No simultaneous testing slows

time to identify issues

The Simulator Approach

• Limited real world applicability
• Virtual, not real devices
• Limited accuracy

Xamarin T est Cloud Approach

AUTOMATED WITH MANAGED DEVICES

• Consistent, scalable and accurate

tests

• Simultaneous testing across

device platforms, operating systems and form factors

• Real usage scenarios, including

complex gestures

• Secure with device wipe after

each test run

Xamarin Test Cloud

Xamarin T est Cloud

2,000+ devices

Automate your app testing and run them on over 2,000 different real devices. T est everything users do, as well as any performance problems with step-by- step memory and performance tracking.

Real devices, real quality

• Ensure higher quality by testing on real devices
• Automate app testing on over 2,000 real devices
• Use C#, Ruby, or Cucumber to build automated

tests

• Integrates with any continuous integration system

Tap Scroll Swipe Pinch Multi finger Text Entry Rotation GPS

Examples of User Interactions

Visually compare test results against hundreds of devices at a time

• Zoom in and full screen resolution of all

screens

• Video playback coming in 2015

Review each step of a test case and compare visual results on all devices

Access precise device specifications, full-size screenshots, and the native device log Understand CPU and memory usage during every step

Object-level user interface testing Tests adapt as user interface changes

Tests Adapt to UI Changes

March 2015

How does it work?

Xamarin T est Cloud

• 1. Create a Xamarin T

est Cloud account

• 2. Prepare tests
• 3. Deploy to XTC
• 4. Integrate with your release pipeline

• 1. Create Xamarin T

est Cloud Account

• free trial

– 30 days – max. 3 concurrent devices – max. 3 testing hours per day

DEMO Registering XTC account

• 2. Prepare tests
• UI T

ests in a separate project

• using NUnit
• can be hand-written
• can be recorded using T

est Recorder

• iOS tests cannot be run on Windows

REPL

• Read-Eval-Print-Loop
• useful when defining tests
• app.Repl()

– stops running test and opens a console window

• tree

– lists the current view‘s structure

• copy

– copies all commands to clipboard

Commands (Methods)

• app.Flash()

– „blinks“ an element – app.Flash(c => c.Class(„EditT ext“))

• app.EnterText(c => c.Marked(„creditCardNumberText“), „12345“)

– app.ClearT ext(...)

• app.Tap(c => c.Marked(„validateButton“))
• app.WaitForElement(c => c.Marked(„label“).Text(„ABC“))

– 1 minute in test cloud – 15 seconds locally – can be changed with a parameter

DEMO Working with Xamarin REPL

T est Recorder

• available as a preview extension for Visual Studio
• https://developer.xamarin.com/guides/testcloud/testrecorder

/visual-studio/

• 3. Deploy to Xamarin T

est Cloud

• via web portal
• from Visual Studio / Xamarin

Studio

• from command line

DEMO Deploy to XTC from VS

• 4. Integrate with your release pipeline
• Visual Studio T

eam Services

• Xamarin T

est Cloud as one of the build steps

DEMO VSTS & Xamarin T est Cloud

Mobile App T esting with Xamarin T est Cloud

Martin Šimeček martin.simecek@microsoft.com @deeedx