automated acceptance testing of mobile apps Karl Krukow, CTO, - - PowerPoint PPT Presentation

▶
automated acceptance testing of mobile apps
SMART_READER_LITE
LIVE PREVIEW

automated acceptance testing of mobile apps Karl Krukow, CTO, - - PowerPoint PPT Presentation

Mar 20, 2023 15 likes •229 views

automated acceptance testing of mobile apps Karl Krukow, CTO, LessPainful Goto Aarhus, 2012 karl@lesspainful.com, @karlkrukow 1 tirsdag den 2. oktober 12 Agenda Automated testing for mobile desirable properties for an acceptance

slide-1
SLIDE 1

automated acceptance testing of mobile apps

Karl Krukow, CTO, LessPainful Goto Aarhus, 2012 karl@lesspainful.com, @karlkrukow

1

tirsdag den 2. oktober 12

slide-2
SLIDE 2

Agenda

  • Automated testing for mobile
  • desirable properties for an acceptance testing tool
  • Introduce Calabash via Short Live Demos
  • Focus on iOS
  • LessPainful: test service and device cloud

2

tirsdag den 2. oktober 12

slide-3
SLIDE 3

Professional practices?

  • Invest the time once! Get the practices going.

Examples:

  • Unit testing
  • Acceptance tests (see this talk!).
  • Continuous build
  • Continuous unit test, functional test (code coverage).
  • Static analysis using clang/FindBugs.
  • Automated deployment to users via in-app updates or link in email.
  • Automatic Crash/Error reporting.
  • ...

3

Go talk to the guys from eBay Classifieds Group DenBlåAvis+BilBasen

tirsdag den 2. oktober 12

slide-4
SLIDE 4

The QA story for mobile

4

tirsdag den 2. oktober 12

slide-5
SLIDE 5

The acceptance testing story for mobile apps

5

  • As realistic an environment as practically possible.
  • simulators/emulators are good, but not enough!
  • Many devices, screens, resolutions, OS’es, CPUs
  • Varying Conditions (Settings, Network, Storage...)
  • Often a manual process: repetitive, expensive
  • Regressions, e.g., app crashes
  • Visual appearance of screens matter (alot!)
  • User experience, Design guidelines, branding,...

tirsdag den 2. oktober 12

slide-6
SLIDE 6

Significant Differences

  • Problems that arise from differences are

significant

  • Crashes
  • Graphical Errors
  • Wrong Computation
  • UI Components
  • Inaccessible text, buttons...

tirsdag den 2. oktober 12

slide-7
SLIDE 7
  • Minimize distance between use cases and actual test

code (DSLs?).

  • Expressive and efficient to write & maintain.
  • Extensible (open vs closed source)
  • High-level, declarative (robustness:“minor” UI changes).
  • Support testing in realistic environments (real devices,
  • n multiple OS versions, languages).
  • Support Continuous integration.

7

Automated acceptance testing desiderata

tirsdag den 2. oktober 12

slide-8
SLIDE 8

tirsdag den 2. oktober 12

slide-9
SLIDE 9

Calabash

  • Vision: Cross-platform Automated Acceptance Testing
  • One interface: Cucumber, for Android and iOS.
  • BDD. Business understandable spec, executable as tests
  • Reuse of Cucumber features across platforms possible.
  • Runs on physical devices and simulators/emulators.
  • Support for native and hybrid apps (embedded webviews)
  • Free, open source (Eclipse Public License)
  • with optional commercial extras support, training, consulting,

device cloud, private device cloud

9

tirsdag den 2. oktober 12

slide-10
SLIDE 10

LessPainful Test Execution Service

  • Execute Calabash tests in parallel on

many devices, OS’es, languages.

  • Visual test reports.
  • Comparison across models and operating

systems.

  • Authentic: Not jailbroken/rooted, supports

both iOS and Android devices.

  • Continuous integration:

lesspainful app.ipa KEY

tirsdag den 2. oktober 12

slide-11
SLIDE 11

tirsdag den 2. oktober 12

slide-12
SLIDE 12

tirsdag den 2. oktober 12

slide-13
SLIDE 13

Calabash iOS: by way of Demo

13

tirsdag den 2. oktober 12

slide-14
SLIDE 14

The Setup

  • install and run
  • Cucumber Basics
  • predefined steps

14

tirsdag den 2. oktober 12

slide-15
SLIDE 15

Calabash Architecture

JSON

  • ver

HTTP

Your App

Calabash Servers (iOS, Android) Calabash Clients

Calabash Android Calabash iOS Calabash JVM

...

tirsdag den 2. oktober 12

slide-16
SLIDE 16

Cucumber + Calabash

16

Step Defs Support Features Features Features

Gherkin Ruby

Calabash Clients

Uses

Cucumber Tool

Executes Produces

Test Results

tirsdag den 2. oktober 12

slide-17
SLIDE 17

Calabash iOS Basics

  • interactive/exploratory development
  • core API
  • screenshots

17

tirsdag den 2. oktober 12

slide-18
SLIDE 18

Calabash iOS Advanced

  • record/transform/playback
  • web view support
  • location support

18

tirsdag den 2. oktober 12

slide-19
SLIDE 19

Roadmap for Calabash

  • Support for dynamic query and action on Android.
  • Better support for reuse of test code across platforms
  • More uniform APIs (old apis will not dissappear)
  • Naming and conceptual model
  • Protocol-level
  • Support platform specific actions, where useful
  • Support for JVM, Better WebView support, Better tooling.
  • Clojure POC already exists - supports any JVM language
  • Your inputs!

19

tirsdag den 2. oktober 12

slide-20
SLIDE 20

References

  • https://github.com/calabash
  • https://github.com/calabash/calabash-ios
  • https://github.com/calabash/calabash-ios/wiki
  • https://github.com/calabash/calabash-ios-server
  • https://github.com/calabash/calabash-android
  • http://blog.lesspainful.com/
  • https://www.lesspainful.com/

20

tirsdag den 2. oktober 12

slide-21
SLIDE 21

Making app testing less painful... Please contact us with any questions: contact@lesspainful.com karl@lesspainful.com - iOS jonas@lesspainful.com - Android http://www.lesspainful.com

Questions?

tirsdag den 2. oktober 12