Testing the Mobile Application's Performance Testing the Mobile - - PowerPoint PPT Presentation

▶
testing the mobile application s performance testing the
SMART_READER_LITE
LIVE PREVIEW

Testing the Mobile Application's Performance Testing the Mobile - - PowerPoint PPT Presentation

Jul 17, 2023 426 likes •651 views

Testing the Mobile Application's Performance Testing the Mobile Application's Performance Case Study on Communicator Mobile Case Study on Communicator Mobile Rama Krishna Pagadala Rama Krishna Pagadala Microsoft Corporation Microsoft

slide-1
SLIDE 1

Testing the Mobile Application's Performance Testing the Mobile Application's Performance

Case Study on Communicator Mobile Case Study on Communicator Mobile Rama Krishna Pagadala Rama Krishna Pagadala

Microsoft Corporation Microsoft Corporation

slide-2
SLIDE 2
  • Performance Metrics for Mobile Applications
  • Communicator Mobile 2007 R2
  • Product Performance Strategy
  • Performance Testing Strategy
  • Architectural improvements
  • Takeaways from this case study

Agenda

slide-3
SLIDE 3

Performance Metrics

  • Time - UI Response Time

– (Mobile) Users tend to be less patient – Very demanding

slide-4
SLIDE 4

User: “Download is too slow!”

How can we make the download faster? Better compression, faster connection, smaller file size, etc. How can we make the download appear faster? Provide progress indication, diversion, etc. How can the download be more tolerable? Offer completion notification, provide “emergency exits”, etc.

Designing and Engineering Time by Steven C. Seow

“Operational Management” Most compelling and tangible, but expensive “Tolerance management”. Best remedy when perception is rigid. Inexpensive “Perception management”. Inexpensive, effective for mainstream usage context

slide-5
SLIDE 5
  • Battery – critical
  • Mitigations

– Minimize the radio wakeups – System events instead of polling

  • Drains non-linear

Performance Metrics

slide-6
SLIDE 6

Cell Radio (Data)

slide-7
SLIDE 7
  • Impacts

– Time – User Experience – Battery

Performance Metrics

  • Network Bandwidth

– Low speed networks – Pay-per-use ($) – Roaming ($$$)

slide-8
SLIDE 8
  • (no Virtual) Memory & CPU

– Device sluggishness  Responsiveness – Application crashes

Performance Metrics

slide-9
SLIDE 9
  • Binary size (footprint)

– Application load time – RAM usage – Download/Install time

Performance Metrics

slide-10
SLIDE 10

Communicator Mobile 2007 R2

  • Communications application for enterprise users
  • Always-on
  • Completes UC* story and complements other UC endpoints

*Microsoft Unified Communications

  • Windows marketplace
  • Contacts, Presence, Work calls, Conferencing, Search, …
slide-11
SLIDE 11

Product Strategy

  • Customer feedback for CoMo 2007
  • Other communication apps
  • Personas
  • 3 Power Users
  • Performance goals
  • Ex: Login < 10 seconds
  • Design for Performance
slide-12
SLIDE 12
  • User Profiles

Testing Strategy

– Baseline/Email Power User

slide-13
SLIDE 13
  • User Profiles

Testing Strategy

– Communicator Mobile Power User

slide-14
SLIDE 14
  • Scenarios

– End-to-End scenarios (User centric) – Script

Testing Strategy

– Specific scenarios (Dev/Test teams)

slide-15
SLIDE 15
  • 3rd party Tools

– PowerMon (device level)

  • In house Tools

– NetMon (network level) – Memory/CPU (per process) – GPRSMon (network/application level)

Testing Strategy

slide-16
SLIDE 16
  • Test Execution

– Daily/Weekly? – Manual vs. Automated? – Test configuration: PocketPC vs. SmartPhone? 3G vs. Edge?, …

Testing Strategy

  • Debugging

– Is hard – Inconsistent issues – Logging impacts the performance – Standard variation

slide-17
SLIDE 17
  • Reporting

– Weekly report (Email) – Trends, charts – Automated results (daily) – Product team analysis

Testing Strategy

slide-18
SLIDE 18

Test Matrix

  • Wide array of devices
  • Carriers
  • Networks - 2.5/Edge/3G
  • Cellular Data vs. WiFi
  • Platform intricacies (WM 6.x power

management)

slide-19
SLIDE 19
  • Poll vs. Push

– Presence updates – Contacts in view

Architectural improvements

  • Collapse Contact Groups

– improves sign-In time!

  • Piggy back the data

– Keep-alives with Presence updates – ActiveSync (Email/Calendar)

slide-20
SLIDE 20

Takeaways

  • Performance is key to mobile application's adoption

– Think/Design/Test early on – Set clear goals for select-few targeted users

  • Performance testing is challenging on mobile platforms

– Metrics are inter dependent – Tools, variety of platforms, … – Usage patterns are quite different – +/- 10-20% Variation? – Consider External factors affecting perf: signal strength, carrier networks, Network type (3G/HSPDA), …

  • Manual (UI) vs. Automated (UI/API)
  • Absolute (engineers) vs. Perceived (users!)
slide-21
SLIDE 21
  • Communicator Mobile Product Team
  • Windows Live Messenger Team

Credits

  • Designing and Engineering Time: The Psychology of Time

Perception in Software by Steven C. Seow

  • Power Monitor
  • SPB GPRSMon
slide-22
SLIDE 22

Questions Thanks!