iOS UI Testing Nick McConnell nicholasgmcconnell@gmail.com - PowerPoint PPT Presentation

iOS UI Testing Nick McConnell nicholasgmcconnell@gmail.com https://www.linkedin.com/in/nickgmcconnell

Intro to UI Testing • Automated UI Testing, that is! • Xcode 7, iOS9 only • Similar UIAutomation (which used javascript) > this has been mostly dropped • Uses accessibility • “Blackbox” approach (?)

You are going to like: • Runs externally, so you are running the exact same code • Some thoughtful options - querying of UI elements • Recording :) • Can use swift or Objective-C • Can debug • Can help with repeated “manual” testing • Managed in the same way as XCTest Unit Tests • Relatively fast (caveats, can remove animations) • Improvements through the beta 7’s Feels natural (kind of)

(a silver bullet) UI Testing (not a silver bullet)

One of Many Ways of Testing… • QA Testing • Stability Testing • Destructive Testing • Black box testing • Smoke Testing • A/B Testing • White box testing • Sanity Testing • Functional Testing • Gray box testing • Integration Testing • Component Testing • Manual Testing • System Testing • Exploratory Testing • Automated Testing • Compatibility Testing • Load Testing • Regression Testing • Alpha Testing • Negative Testing • Acceptance Testing • Beta Testing • Internationalization Testing • Performance Testing • Monkey Testing • Non-functional Testing • Scalability Testing • Gorilla Testing • Stress Testing • Unit Testing • Baboon Testing • Usability Testing • UI Testing • Security Testing • Ad-hoc Testing

A Common Perspective….

Compare • UI Automation • Calabash • Javascript • Uses Cucumber • Uses Accessibility • Android • No longer supported • Appium • KIF • Supports several languages • objective-C • Android • developed by Square • uses undocumented API (future issues?) • “Breaks with every major iOS revision”, “slow” • Possibly more flexible, but complexities to consider

Management • Separate Target (can add automatically when creating new project or later) • Runs Externally from App • Sim or Physical • Source control easy (code part of project) • Right-click on the method name…. run and disable/enable • Test Navigator (right-click… run test) • Report Navigator (right-click… jump to reports) • Report Navigator - Check screenshots • Schemes to make test execution. • Keyboard shortcuts: ⌘ U to run all, ⌃⌥⌘ U to run individual tests

Basics • 2 classes: XCUIElementQuery & XCUIElement • Queries are stored paths to elements • Elements are the singular paths to UI Objects obtained through queries. • Elements are NOT UIViews, they are “proxies” • Queries and Elements are only resolved when needed • Elements have types that map to UI Objects but are not “class typed” (Buttons, Images, StaticTexts, Cells, Tables)

Basic Queries App NavBar Table Cell Cell Button USES THE VEIW HIERARCHY >>> Label Label descendantsMatchingType (type) > All the way down the tree childrenMatchingType (type) > One level down only containingType(type, identifier) > Looks below Oh yeah! Convenience methods instead of descendantsMatchingType… .buttons .tables .staticTexts ( Labels) .cells .alerts .otherElements ( Views - useful) .images .navigationBars

From Query to Element by Text .staticTexts[“Ape”] .containingType(.StaticText, “Monkey”) >>> Will use text labels, accessibility IDs, “processed”image names by Index .elementBoundByIndex(index) Using Predicate .containingPredicate(NSPredicate(format: "label contains[c] 'hello'")) Assume One (will fail if more) .element

Actions on Elements Is it there? .exists() Taps etc .tap(), doubleTap(), twoFingerTap(), .pressForDuration(duration) etc Swipes .swipeUp(), .swipeDown(), .swipeLeft() etc Other .rotate(rotation,velocity), .pinchWithScale(scale,velocity) Count a query .count()

First Query let app = XCUIApplication() let labels = app.descendantsMatchingType(.StaticText) let label = labels[“Baboon”] XCTAssertTrue (label.exists()) OR XCTAssertTrue (app.StaticTexts[“Baboon”].exists())

Record • Magic Red Button • Can you use to add to existing test code • Caution: Sometimes creates code that doesn’t run (eg system alerts) • Caution: Sometimes creates brittle, awkward code. Consider rewriting. • Great for getting started with UI Testing

Recording Craziness app.childrenMatchingType(.Window).elemen tBoundByIndex(0).childrenMatchingType(.O ther).element.childrenMatchingType(.Othe r).element.childrenMatchingType(.Other). element.childrenMatchingType(.Other).ele ment.childrenMatchingType(.Other).elemen t.childrenMatchingType(.Other).elementBo undByIndex(2).tables.childrenMatchingTyp e(.Cell).elementBoundByIndex(4).children MatchingType(.Button).elementBoundByInde x(0).tap()

Surprisingly… • No official docs for XCUIElement etc • Can’t get access to state, color, font • Can’t check directly for views, activity indicator etc (caveats) • Can’t call any application classes directly • UI Automation is not for non-techies • No code coverage (maybe not surprising) • Some instability • What you record doesn’t always run • v1 > Hope there is more to come

Real World • Logic paths based on signed-on or not etc • Look at backend and database’s data stability • Start from a solid refresh point: can delete app from sim: xcrun simctl uninstall booted com.fueled.OnyDo || true • • If adding data, look at repeatability. May need a clean- up step afterwards. May need separate DB/ environment. • Could look to Mocks (change your code)

Command Line & CI • Command Line: xcodebuild test • destination to pick device. • Xbots - good integration with unit, UI, code coverage and screenshots • Jenkins (use agent not daemon, with OCUnit2Junit XSLT can trend, can give access to screenshots) • Bitrise - just works (basic) • Bamboo - ??? • UI Testing is generally less stable than Unit testing • Screen shots

Long Term Value & Maintenance • Would guess 90% of historically written automated UI Tests don’t survive further than the first launch. • Automated UI Test code is just code of course, and code needs TLC by coders. • Consider how easy your UI Test code will respond to future UI changes. • Consider directly rewriting “recorded” code. Good start but often very brittle. • Break up tests into separate methods for each step… encapsulate sign-in process • Consider design patterns such as Page Object pattern (http:// code.tutsplus.com/articles/maintainable-automated-ui-tests--net-35089 • Whole team / org decision

Final For all the caveats, UI Testing may be the Automated UI Test tool you actually want to use. It’s easy to get into and it’s right there in Xcode. And with that, you may end up with real, valuable UI Tests that will become part of your daily workflow.

Misc Comments • WWDC 2015: https://developer.apple.com/videos/wwdc/2015/?id=406 • Pinching and Zooming, Hardware buttons, UIPicker…. new in the final Beta. • Performance Tests - possible • Test case order is random • Can use it as a quick-start if you are repeating a lot of manual tests (command line) • System Alerts - automaticallu clicks OK/Allow • Can even “join in” even running UI Tests