managing app testing device clouds issues and
play

Managing App Testing Device Clouds: Issues and Opportunities Mattia - PowerPoint PPT Presentation

Jan 18, 2024 •261 likes •321 views

Managing App Testing Device Clouds: Issues and Opportunities Mattia Fazzini Alessandro Orso Problem Description Study Methodology Android Mobile App Testing Devices and Tests Considered AWS Compatibility Device Farm Android Version

  1. Managing App Testing Device Clouds: Issues and Opportunities Mattia Fazzini Alessandro Orso Problem Description Study Methodology Android Mobile App Testing Devices and Tests Considered AWS Compatibility Device Farm Android Version #Devices CST ver. #Modules #Tests J ELLY B EAN 4.2 2 50 17268 4.2.2_r1 Test Suite J ELLY B EAN 4.3 5 58 18013 4.5_r2.2-cts K IT K AT 4.4 38 73 24039 cts-4.4_r4 L OLLIPOP 5.0 11 93 33929 cts-5.0_r9 L OLLIPOP 5.1 6 96 34207 cts-5.1_r28 M ARSHMALLOW 6.0 15 114 37613 cts-6.0_r32 N OUGAT 7.0 7 124 43494 cts-7.0_r32 Could Testing Infrastructures N OUGAT 7.1 2 133 43598 cts-7.1_r29 Oreo 8.0 11 195 59459 cts-8.0_r20 AWS Device Farm Firebase Test Lab App Center O REO 8.1 4 210 60383 cts-8.1_r16 P IE 9 9 289 84182 cts-9.0_r9 ‣ We used the Compatibility Test Suite to analyze the behavior of 110 devices on the Device farm. We collected a total of 12 million test executions. Study Results and Analysis Future Work ‣ Create a taxonomy of the failures ‣ Identify per-vendor and per-version failures ‣ We observed 9,778 failures and all devices experienced at least one failure ‣ Investigate automatic generation of ‣ We investigated general issues with the cloud infrastructure compatibility-based tests ‣ 58 tests failed on all devices (27 confirmed by a device farm engineer) ‣ Study light-weight monitoring of cloud-based ‣ We investigated security-related issues test executions ‣ 1,307 security-related test failures (caused by 153 different tests)

  2. Managing App Testing Device Clouds: Issues and Opportunities Mattia Fazzini Alessandro Orso Problem Description Study Methodology Android Mobile App Testing Devices and Tests Considered AWS Compatibility Device Farm Android Version #Devices CST ver. #Modules #Tests J ELLY B EAN 4.2 2 50 17268 4.2.2_r1 Test Suite J ELLY B EAN 4.3 5 58 18013 4.5_r2.2-cts K IT K AT 4.4 38 73 24039 cts-4.4_r4 L OLLIPOP 5.0 11 93 33929 cts-5.0_r9 L OLLIPOP 5.1 6 96 34207 cts-5.1_r28 M ARSHMALLOW 6.0 15 114 37613 cts-6.0_r32 N OUGAT 7.0 7 124 43494 cts-7.0_r32 Could Testing Infrastructures N OUGAT 7.1 2 133 43598 cts-7.1_r29 Oreo 8.0 11 195 59459 cts-8.0_r20 AWS Device Farm Firebase Test Lab App Center O REO 8.1 4 210 60383 cts-8.1_r16 P IE 9 9 289 84182 cts-9.0_r9 ‣ We used the Compatibility Test Suite to analyze the behavior of 110 devices on the Device farm. We collected a total of 12 million test executions. Study Results and Analysis Future Work ‣ Create a taxonomy of the failures ‣ Identify per-vendor and per-version failures ‣ We observed 9,778 failures and all devices experienced at least one failure ‣ Investigate automatic generation of ‣ We investigated general issues with the cloud infrastructure compatibility-based tests ‣ 58 tests failed on all devices (27 confirmed by a device farm engineer) ‣ Study light-weight monitoring of cloud-based ‣ We investigated security-related issues test executions ‣ 1,307 security-related test failures (caused by 153 different tests)

  3. Managing App Testing Device Clouds: Issues and Opportunities Mattia Fazzini Alessandro Orso Problem Description Study Methodology Android Mobile App Testing Devices and Tests Considered AWS Compatibility Device Farm Android Version #Devices CST ver. #Modules #Tests J ELLY B EAN 4.2 2 50 17268 4.2.2_r1 Test Suite J ELLY B EAN 4.3 5 58 18013 4.5_r2.2-cts K IT K AT 4.4 38 73 24039 cts-4.4_r4 L OLLIPOP 5.0 11 93 33929 cts-5.0_r9 L OLLIPOP 5.1 6 96 34207 cts-5.1_r28 M ARSHMALLOW 6.0 15 114 37613 cts-6.0_r32 N OUGAT 7.0 7 124 43494 cts-7.0_r32 Could Testing Infrastructures N OUGAT 7.1 2 133 43598 cts-7.1_r29 Oreo 8.0 11 195 59459 cts-8.0_r20 AWS Device Farm Firebase Test Lab App Center O REO 8.1 4 210 60383 cts-8.1_r16 P IE 9 9 289 84182 cts-9.0_r9 ‣ We used the Compatibility Test Suite to analyze the behavior of 110 devices on the Device farm. We collected a total of 12 million test executions. Study Results and Analysis Future Work ‣ Create a taxonomy of the failures ‣ Identify per-vendor and per-version failures ‣ We observed 9,778 failures and all devices experienced at least one failure ‣ Investigate automatic generation of ‣ We investigated general issues with the cloud infrastructure compatibility-based tests ‣ 58 tests failed on all devices (27 confirmed by device farm engineers) ‣ Study light-weight monitoring of cloud-based ‣ We investigated security-related issues test executions ‣ 1,307 security-related test failures (caused by 153 different tests)

  4. Managing App Testing Device Clouds: Issues and Opportunities Mattia Fazzini Alessandro Orso Problem Description Study Methodology Android Mobile App Testing Devices and Tests Considered AWS Compatibility Device Farm Android Version #Devices CST ver. #Modules #Tests J ELLY B EAN 4.2 2 50 17268 4.2.2_r1 Test Suite J ELLY B EAN 4.3 5 58 18013 4.5_r2.2-cts K IT K AT 4.4 38 73 24039 cts-4.4_r4 L OLLIPOP 5.0 11 93 33929 cts-5.0_r9 L OLLIPOP 5.1 6 96 34207 cts-5.1_r28 M ARSHMALLOW 6.0 15 114 37613 cts-6.0_r32 N OUGAT 7.0 7 124 43494 cts-7.0_r32 Could Testing Infrastructures N OUGAT 7.1 2 133 43598 cts-7.1_r29 Oreo 8.0 11 195 59459 cts-8.0_r20 AWS Device Farm Firebase Test Lab App Center O REO 8.1 4 210 60383 cts-8.1_r16 P IE 9 9 289 84182 cts-9.0_r9 ‣ We used the Compatibility Test Suite to analyze the behavior of 110 devices on the Device farm. We collected a total of 12 million test executions. Study Results and Analysis Future Work ‣ Create a taxonomy of the failures ‣ Identify per-vendor and per-version failures ‣ We observed 9,778 failures and all devices experienced at least one failure ‣ Investigate automatic generation of ‣ We investigated general issues with the cloud infrastructure compatibility-based tests ‣ 58 tests failed on all devices (27 confirmed by a device farm engineer) ‣ Study light-weight monitoring of cloud-based ‣ We investigated security-related issues test executions ‣ 1,307 security-related test failures (caused by 153 different tests)

  5. Managing App Testing Device Clouds: Issues and Opportunities Mattia Fazzini Alessandro Orso Problem Description Study Methodology Android Mobile App Testing Devices and Tests Considered AWS Compatibility Device Farm Android Version #Devices CST ver. #Modules #Tests J ELLY B EAN 4.2 2 50 17268 4.2.2_r1 Test Suite J ELLY B EAN 4.3 5 58 18013 4.5_r2.2-cts K IT K AT 4.4 38 73 24039 cts-4.4_r4 L OLLIPOP 5.0 11 93 33929 cts-5.0_r9 L OLLIPOP 5.1 6 96 34207 cts-5.1_r28 M ARSHMALLOW 6.0 15 114 37613 cts-6.0_r32 N OUGAT 7.0 7 124 43494 cts-7.0_r32 Could Testing Infrastructures N OUGAT 7.1 2 133 43598 cts-7.1_r29 Oreo 8.0 11 195 59459 cts-8.0_r20 AWS Device Farm Firebase Test Lab App Center O REO 8.1 4 210 60383 cts-8.1_r16 P IE 9 9 289 84182 cts-9.0_r9 ‣ We used the Compatibility Test Suite to analyze the behavior of 110 devices on the Device farm. We collected a total of 12 million test executions. Study Results and Analysis Future Work ‣ Create a taxonomy of the failures ‣ Identify per-vendor and per-version failures ‣ We observed 9,778 failures and all devices experienced at least one failure ‣ Investigate automatic generation of ‣ We investigated general issues with the cloud infrastructure compatibility-based tests ‣ 58 tests failed on all devices (27 confirmed by a device farm engineer) ‣ Study light-weight monitoring of cloud-based ‣ We investigated security-related issues test executions ‣ 1,307 security-related test failures (caused by 153 different tests)

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Q-Clouds: Managing Performance Interference Effects for QoS-Aware Clouds Ripal Nathuji Aman

Q-Clouds: Managing Performance Interference Effects for QoS-Aware Clouds Ripal Nathuji Aman

Q-Clouds: Managing Performance Interference Effects for QoS-Aware Clouds Ripal Nathuji Aman Kansal Alireza Ghaffarkhah Presented by Joshua Davis Motivation and Background Cloud computing Off load processing and storage Charged per

549 views • 28 slides
When you look up into the sky, you will often see clouds. No two clouds are the same, and there

When you look up into the sky, you will often see clouds. No two clouds are the same, and there

When you look up into the sky, you will often see clouds. No two clouds are the same, and there are many different types of clouds. Different kinds of clouds lead to different types of weather. Cirrus clouds are the highest group of clouds that

271 views • 5 slides
Solving Device Tree Issues Use of device tree is mandatory for all new ARM systems. But the

Solving Device Tree Issues Use of device tree is mandatory for all new ARM systems. But the

Solving Device Tree Issues Use of device tree is mandatory for all new ARM systems. But the implementation of device tree has lagged behind the mandate. The first priority has been correct function. Lower priorities include device tree

2.4k views • 183 slides
Seamless Offloading of Web App Computations From Mobile Device to Edge Clouds via HTML5 Web

Seamless Offloading of Web App Computations From Mobile Device to Edge Clouds via HTML5 Web

Seamless Offloading of Web App Computations From Mobile Device to Edge Clouds via HTML5 Web Worker Migration Hyuk Jin Jeong Seoul National University SoCC 2019 Virtual Machine & Optimization Laboratory Department of Electrical and

824 views • 29 slides
Treatment Device Testing Protocols in the Garden State:

Treatment Device Testing Protocols in the Garden State:

Treatment Device Testing Protocols in the Garden State: Current Status of Laboratory Testing and Field Longevity Monitoring Washington Stormwater Center 2013

387 views • 16 slides
ykpang@beyondsecurity.com Fuzz Testing for Embedded Device Security Assurance (EDSA) ISASecure

ykpang@beyondsecurity.com Fuzz Testing for Embedded Device Security Assurance (EDSA) ISASecure

Empowering Security Teams ykpang@beyondsecurity.com Fuzz Testing for Embedded Device Security Assurance (EDSA) ISASecure EDSA Certification Communication/Network Robustness Testing (CRT) CRT examines the capability of the device to

859 views • 33 slides
Practical Issues with HIV Practical Issues with HIV Testing, CD4 Count and Viral Testing, CD4

Practical Issues with HIV Practical Issues with HIV Testing, CD4 Count and Viral Testing, CD4

Practical Issues with HIV Practical Issues with HIV Testing, CD4 Count and Viral Testing, CD4 Count and Viral Load testing Load testing 2 ND Phnom Penh Symposium on HIV 2 ND Phnom Penh Symposium on HIV Marie Downer, CDC; Chuop Sokheng, NIPH,

104 views • 8 slides
State of the Art and Open Issues in Software Testing State of the Art and Open Issues in

State of the Art and Open Issues in Software Testing State of the Art and Open Issues in

State of the Art and Open Issues in Software Testing State of the Art and Open Issues in Software Testing Software Testing d e a t g t i s e v i n d n a Most used approach 18 13.5 9 4.5 0 2000 2002 2004 2006 2008 2010

975 views • 84 slides
The Missing Satellites of the Magellanic Clouds Testing LCDM Predictions on Small Scales Nitya

The Missing Satellites of the Magellanic Clouds Testing LCDM Predictions on Small Scales Nitya

The Missing Satellites of the Magellanic Clouds Testing LCDM Predictions on Small Scales Nitya Kallivayalil Small Galaxies, Cosmic Questions, Durham, UK, 2019 Ultra-Faint Intrigue [M * (0.11) 10 4 M ; M halo ~ 10 8 M ]

241 views • 11 slides
Solving Device Tree Issues - part 3 Using devicetree is painful. The framework does not help to

Solving Device Tree Issues - part 3 Using devicetree is painful. The framework does not help to

Solving Device Tree Issues - part 3 Using devicetree is painful. The framework does not help to develop correct system descriptions (device tree source) and drivers. Errors are often not visible or are hard to understand. But tools and

2.09k views • 140 slides
Solving Device Tree Issues - part 2 Debugging devicetree issues is painful. Last year I

Solving Device Tree Issues - part 2 Debugging devicetree issues is painful. Last year I

Solving Device Tree Issues - part 2 Debugging devicetree issues is painful. Last year I presented some under-development tools and techniques to debug devicetree issues. This year I will provide an update on the status of those tools and

1.02k views • 85 slides
IDENTIFYING & MANAGING GCP COMPLIANCE RISKS FOR THE PHARMACEUTICAL, BIOTECH & DEVICE

IDENTIFYING & MANAGING GCP COMPLIANCE RISKS FOR THE PHARMACEUTICAL, BIOTECH & DEVICE

IDENTIFYING & MANAGING GCP COMPLIANCE RISKS FOR THE PHARMACEUTICAL, BIOTECH & DEVICE INDUSTRIES Karen Weaver Epstein, Becker & Green Health Care & Life Sciences Practice APPLICABLE REGULATIONS 21 CFR 312 - Investigational New

527 views • 24 slides
Improving Agility and Elasticity in Bare-metal Clouds Yushi Omote , Takahiro Shinagawa ,

Improving Agility and Elasticity in Bare-metal Clouds Yushi Omote , Takahiro Shinagawa ,

Improving Agility and Elasticity in Bare-metal Clouds Yushi Omote , Takahiro Shinagawa , Kazuhiko Kato University of Tsukuba, The University of Tokyo 1 Bare-metal Clouds An IaaS for high performance and device functionality

230 views • 21 slides
Handheld Infrared Device Dr Richard Stewart, Managing Director, Ziltek Pty Ltd Conference on

Handheld Infrared Device Dr Richard Stewart, Managing Director, Ziltek Pty Ltd Conference on

Rapid Mapping of Soils in China using a Handheld Infrared Device Dr Richard Stewart, Managing Director, Ziltek Pty Ltd Conference on Remediation of Chlorinated and Recalcitrant Compounds, 8-12 April 2018 Acknowledgements 2 Contaminated Soil in

399 views • 20 slides
Social Clouds Creating a research agenda Andrew Lippman MIT Media Lab October, 2010 Clouds and

Social Clouds Creating a research agenda Andrew Lippman MIT Media Lab October, 2010 Clouds and

Social Clouds Creating a research agenda Andrew Lippman MIT Media Lab October, 2010 Clouds and Mists: 1 st Cloud Defined by access Familiar as the internet Result of a nexus: PC, backbone, ISPs Web Opened vast doors to creativity Clouds and

592 views • 24 slides
SeeTest Continuous Testing Platform 1 Layout (Visual) Testing Integrate visual testing into your

SeeTest Continuous Testing Platform 1 Layout (Visual) Testing Integrate visual testing into your

Layout (Visual) Testing SeeTest Continuous Testing Platform 1 Layout (Visual) Testing Integrate visual testing into your CI/CD flow to detect and fix responsive design issues before production Create automated tests to verify UI responsiveness

621 views • 8 slides
CapiTainS Guidelines From digital edition to machine actionable edition Thibault Clrice, PhD

CapiTainS Guidelines From digital edition to machine actionable edition Thibault Clrice, PhD

CapiTainS Guidelines From digital edition to machine actionable edition Thibault Clrice, PhD at Humboldt Chair of Digital Humanities, Leipzig University @ponteineptique A problem (1) In France alone : 1575 PhD for lettres classiques

511 views • 23 slides
A Common Terminology Services (CTS) Back-end to Protg Harold R Solbrig Christopher G Chute,

A Common Terminology Services (CTS) Back-end to Protg Harold R Solbrig Christopher G Chute,

A Common Terminology Services (CTS) Back-end to Protg Harold R Solbrig Christopher G Chute, M.D., Dr. P.H. Division of Biomedical Informatics Mayo Clinic Outline Purpose of the project Characteristics of Terminology Links The

995 views • 68 slides
ASIC Physical Design Standard-Cell Design Flow Using the Cadence Innovus Digital Implementation

ASIC Physical Design Standard-Cell Design Flow Using the Cadence Innovus Digital Implementation

ASIC Physical Design Standard-Cell Design Flow Using the Cadence Innovus Digital Implementation System ASIC Physical Design (Standard Cell) (can also do full custom layout) Component-Level Verilog Netlist Cadence Innovus Std. Cell

878 views • 45 slides
CSC2547 Presentation: Curiosity-driven exploration Count-based VS Info gain-based Sheng Jia,

CSC2547 Presentation: Curiosity-driven exploration Count-based VS Info gain-based Sheng Jia,

CSC2547 Presentation: Curiosity-driven exploration Count-based VS Info gain-based Sheng Jia, Tinglin Duan (First year master students) 1. PLAN (2011) 1. VIME (NeurIPS2016) 1. CTS (NeurIPS2016) Outline Motivation, Related Works and

770 views • 40 slides
Functions of multiple variables We say that f is a real, scalarvalued function of n variables

Functions of multiple variables We say that f is a real, scalarvalued function of n variables

Functions of multiple variables We say that f is a real, scalarvalued function of n variables 1 2 we often need to restrict the domain 3 Usually, the domain may be determined by inspection. 4 5 We have seen before that traces may be

499 views • 29 slides
Random Access MAC for Efficient Broadcast Support in Ad Hoc Networks Ken Tang, Mario Gerla

Random Access MAC for Efficient Broadcast Support in Ad Hoc Networks Ken Tang, Mario Gerla

Random Access MAC for Efficient Broadcast Support in Ad Hoc Networks Ken Tang, Mario Gerla Computer Science Department University of California, Los Angeles Los Angeles, CA 90095 http://www.cs.ucla.edu/NRL/wireless Overview Introduction

383 views • 17 slides
Canonical Text Service Jochen Tiepmar BigData Competence Center ScaDS Naural Language Processing

Canonical Text Service Jochen Tiepmar BigData Competence Center ScaDS Naural Language Processing

Canonical Text Service Jochen Tiepmar BigData Competence Center ScaDS Naural Language Processing Leipzig University Canonical Text Service - Jochen Tiepmar 2015 www.urncts.de/survey Survey From 20.06.2015 to 30.08.2015 Anonym, no tracking,

372 views • 24 slides
What Radiology Test Disclosure Should I Order? David M. Naeger, MD Consultant to CMEinfo/Ebix

What Radiology Test Disclosure Should I Order? David M. Naeger, MD Consultant to CMEinfo/Ebix

5/23/19 What Radiology Test Disclosure Should I Order? David M. Naeger, MD Consultant to CMEinfo/Ebix Associate Professor and Associate Chair for Education david.naeger@ucsf.edu University of California, San Francisco Dept of Radiology and

452 views • 23 slides

More recommend