How AI Is Transforming Software Testing Presented by: - - PDF document

▶
how ai is transforming software testing
SMART_READER_LITE
LIVE PREVIEW

How AI Is Transforming Software Testing Presented by: - - PDF document

Feb 17, 2023 168 likes •600 views

W6 AI and Data Analytics Wednesday, October 17th, 2018 11:30 AM How AI Is Transforming Software Testing Presented by:

slide-1
SLIDE 1

¡ ¡ W6 ¡

AI ¡and ¡Data ¡Analytics ¡ Wednesday, ¡October ¡17th, ¡2018 ¡11:30 ¡AM ¡ ¡ ¡ ¡ ¡ ¡ ¡

How ¡AI ¡Is ¡Transforming ¡Software ¡ Testing ¡ ¡

Presented ¡by: ¡ ¡ ¡

Raj ¡Subramanian ¡

¡ ¡ ¡ ¡

Brought ¡to ¡you ¡by: ¡ ¡ ¡ ¡

¡

¡

¡ ¡

350 ¡Corporate ¡Way, ¡Suite ¡400, ¡Orange ¡Park, ¡FL ¡32073 ¡ ¡ 888-­‑-­‑-­‑268-­‑-­‑-­‑8770 ¡·√·√ ¡904-­‑-­‑-­‑278-­‑-­‑-­‑0524 ¡-­‑ ¡info@techwell.com ¡-­‑ ¡http://www.starwest.techwell.com/ ¡ ¡ ¡

¡

¡ ¡ ¡

slide-2
SLIDE 2

¡ ¡ ¡

¡

Raj ¡Subramanian ¡

¡ Raj ¡Subramanian ¡is ¡a ¡former ¡developer ¡who ¡moved ¡to ¡testing ¡to ¡focus ¡on ¡his ¡

  • passion. ¡Raj ¡currently ¡works ¡as ¡a ¡Developer ¡Evangelist ¡for ¡Testim.io, ¡that ¡provides ¡

stable ¡self-­‑healing ¡AI ¡based ¡test ¡automation ¡to ¡enterprises ¡such ¡as ¡Netapp, ¡ Swisscom,Wix ¡and ¡Autodesk. ¡He ¡also ¡provides ¡mobile ¡training ¡and ¡consulting ¡for ¡ different ¡clients. ¡He ¡actively ¡contributes ¡to ¡the ¡testing ¡community ¡by ¡speaking ¡at ¡ conferences, ¡writing ¡articles, ¡blogging, ¡making ¡videos ¡on ¡his ¡youtube ¡channel ¡and ¡ being ¡directly ¡involved ¡in ¡various ¡testing-­‑related ¡activities. ¡He ¡currently ¡resides ¡in ¡ Chicago ¡and ¡can ¡be ¡reached ¡at ¡raj@testim.io ¡and ¡on ¡twitter ¡at ¡@epsilon11. ¡He ¡ actively ¡blogs ¡on ¡www.testim.io ¡and ¡his ¡website ¡www.rajsubra.com. ¡His ¡videos ¡on ¡ testing, ¡leadership ¡and ¡productivity ¡can ¡be ¡found ¡here ¡ http://www.rajsubra.com/my-­‑youtube-­‑channel/ ¡ ¡

slide-3
SLIDE 3

HOW AI IS

www.testim.io

Raj Subramanian raj@testim.io

TRANSFORMING SOFTWARE TESTING

slide-4
SLIDE 4

www.testim.io

ABOUT ME

  • Developer Evangelist at Testim.io
  • Blogger, Speaker and Writer

○ http://www.rajsubra.com/blog/ ○ https://blog.testim.io/ ○ http://www.rajsubra.com/publications/ ○ http://www.rajsubra.com/presentations/ ○

  • 11+ Years of Development and Testing

Twitter: @epsilon11 E-mail: raj@testim.io

slide-5
SLIDE 5

AGENDA AGENDA

LEARN THE BASICS OF AI CURRENT CHALLENGES HOW AI CAN SOLVE THESE CHALLENGES FUTURE OF AI

slide-6
SLIDE 6

CONTEXT MATTERS CONTEXT MATTERS

slide-7
SLIDE 7

Manual Testing Waterfall methodology

1980 - 1990

Bulky Automation Tools Experimentation with Different Development Approaches

1990 – 2003

More Robust Automation tools + Open Source Frameworks Agile Approaches, faster release cycles

2003 - 2010

More about Scale DevOps, Continuous Testing, CI/CD

2010 - CURRENT

Autonomous Testing, Machine Learning and AI Collaborative, Smart testing

THE FUTURE

EVOLUTION OF TESTING EVOLUTION OF TESTING

www.testim.io

slide-8
SLIDE 8

Manual Testing Waterfall methodology

1980 - 1990

Bulky Automation Tools Experimentation with Different Development Approaches

1990 – 2003

More Robust Automation tools + Open Source Frameworks Agile Approaches, faster release cycles

2003 - 2010

More about Scale DevOps, Continuous Testing, CI/CD

2010 - CURRENT

Autonomous Testing, Machine Learning and AI Collaborative, Smart testing

THE FUTURE

EVOLUTION OF TESTING EVOLUTION OF TESTING

www.testim.io

slide-9
SLIDE 9

Manual Testing Waterfall methodology

1980 - 1990

Bulky Automation Tools Experimentation with Different Development Approaches

1990 – 2003

More Robust Automation tools + Open Source Frameworks Agile Approaches, faster release cycles

2003 - 2010

More about Scale DevOps, Continuous Testing, CI/CD

2010 - CURRENT

Autonomous Testing, Machine Learning and AI Collaborative, Smart testing

THE FUTURE

EVOLUTION OF TESTING EVOLUTION OF TESTING

www.testim.io

slide-10
SLIDE 10

Manual Testing Waterfall methodology

1980 - 1990

Bulky Automation Tools Experimentation with Different Development Approaches

1990 – 2003

More Robust Automation tools + Open Source Frameworks Agile Approaches, faster release cycles

2003 - 2010

More about Scale DevOps, Continuous Testing, CI/CD

2010 - CURRENT

Autonomous Testing, Machine Learning and AI Collaborative, Smart testing

THE FUTURE

EVOLUTION OF TESTING EVOLUTION OF TESTING

www.testim.io

slide-11
SLIDE 11

Manual Testing Waterfall methodology

1980 - 1990

Bulky Automation Tools Experimentation with Different Development Approaches

1990 – 2003

More Robust Automation tools + Open Source Frameworks Agile Approaches, faster release cycles

2003 - 2010

More about Scale DevOps, Continuous Testing, CI/CD

2010 - CURRENT

Autonomous Testing, Machine Learning and AI Collaborative, Smart testing

THE FUTURE

EVOLUTION OF TESTING EVOLUTION OF TESTING

www.testim.io

slide-12
SLIDE 12

Manual Testing Waterfall methodology

1980 - 1990

Bulky Automation Tools Experimentation with Different Development Approaches

1990 – 2003

More Robust Automation tools + Open Source Frameworks Agile Approaches, faster release cycles

2003 - 2010

More about Scale DevOps, Continuous Testing, CI/CD

2010 - CURRENT

Autonomous Testing, Machine Learning and AI Collaborative, Smart testing

THE FUTURE

EVOLUTION OF TESTING EVOLUTION OF TESTING

www.testim.io

slide-13
SLIDE 13

BASICS OF AI BASICS OF AI

slide-14
SLIDE 14

Artificial Intelligence (AI)

www.testim.io

DE-MYSTIFYING AI DE-MYSTIFYING AI

Machine Learning (ML) Deep Learning (DL)

slide-15
SLIDE 15

Size Price Size Bedrooms Family Size Zip code Wealth Walkable School Quality Price

www.testim.io

DEEP LEARNING EXAMPLE DEEP LEARNING EXAMPLE

slide-16
SLIDE 16

www.testim.io

slide-17
SLIDE 17

DEMO: QUICK DRAW DEMO: QUICK DRAW

slide-18
SLIDE 18

CURRENT CHALLENGES CURRENT CHALLENGES

slide-19
SLIDE 19

Not at all, we test last, we test quality into the product Testing involved early, but still have an extended test phase before release We could probably release after every sprint We could ship daily if we wanted to We could deploy any minute of any day 13% 57% 19% 10% 2%

YOUR RELEASES? HOW FREQUENT ARE HOW FREQUENT ARE YOUR RELEASES?

FROM A RECENT SURVEY TESTIM COMPLETED

www.testim.io

slide-20
SLIDE 20

Not at all, we test last, we test quality into the product Testing involved early, but still have an extended test phase before release We could probably release after every sprint We could ship daily if we wanted to We could deploy any minute of any day 13% 57% 19% 10% 2%

YOUR RELEASES? HOW FREQUENT ARE HOW FREQUENT ARE YOUR RELEASES?

FROM A RECENT SURVEY TESTIM COMPLETED

www.testim.io

slide-21
SLIDE 21

Not at all, we test last, we test quality into the product Testing involved early, but still have an extended test phase before release We could probably release after every sprint We could ship daily if we wanted to We could deploy any minute of any day 13% 57% 19% 10% 2%

YOUR RELEASES? HOW FREQUENT ARE HOW FREQUENT ARE YOUR RELEASES?

FROM A RECENT SURVEY TESTIM COMPLETED

www.testim.io

slide-22
SLIDE 22

CURRENT CHALLENGES CURRENT CHALLENGES RAPID RELEASE CYCLES RAPID RELEASE CYCLES

Innovators 2.5% Early Adopters 13.5% Chasm Early Majority 34% Late Majority 34% Laggards 34% Every two weeks Every day 5 times a day

www.testim.io

Every 6 month Every month

slide-23
SLIDE 23

THE CHALLENGE: THE CHALLENGE: #1 SKILL SET #1 SKILL SET

www.testim.io

Innovators 2.5% Early Adopters 13.5% Early Majority 34% Late Majority 34% Laggards 34% Every two weeks Every day 5 times a day Every 6 month Every month

www.testim.io

Chasm

slide-24
SLIDE 24

THE CHALLENGE: THE CHALLENGE: #1 SKILL SET #1 SKILL SET

www.testim.io www.testim.io

Source: http://dilbert.com/

slide-25
SLIDE 25

THE CHALLENGE: THE CHALLENGE: #1 SKILL SET #1 SKILL SET

www.testim.io www.testim.io

slide-26
SLIDE 26

THE CHALLENGE: THE CHALLENGE:

Innovators 2.5% Early Adopters 13.5% Early Majority 34% Late Majority 34% Laggards 34% Every two weeks Every day 5 times a day Every 6 month Every month Chasm

#2 AUTHORING #2 AUTHORING

Chasm

www.testim.io

slide-27
SLIDE 27

#3 INITIALIZATION

Innovators 2.5% Early Adopters 13.5% Early Majority 34% Late Majority 34% Laggards 34% Every two weeks Every day 5 times a day Every 6 month Every month Chasm

#3 INITIALIZATION THE CHALLENGE: THE CHALLENGE:

Chasm

www.testim.io

slide-28
SLIDE 28

www.testim.io

slide-29
SLIDE 29

www.testim.io

slide-30
SLIDE 30

#4 MAINTENANCE #4 MAINTENANCE

Chasm Innovators 2.5% Early Adopters 13.5% Early Majority 34% Late Majority 34% Laggards 34% Every two weeks Every day 5 times a day Every 6 month Every month

THE CHALLENGE: THE CHALLENGE:

Chasm

www.testim.io

slide-31
SLIDE 31

Innovators 2.5% Early Adopters 13.5% Early Majority 34% Late Majority 34% Laggards 34% Every two weeks Every day 5 times a day Every 6 month Every month Chasm

THE CHALLENGE: THE CHALLENGE: #5 SCALE #5 SCALE

Chasm

www.testim.io

slide-32
SLIDE 32

HOW AI CAN SOLVE THESE CHALLENGES HOW AI CAN SOLVE THESE CHALLENGES

slide-33
SLIDE 33

www.testim.io

slide-34
SLIDE 34

TESTIM.IO TESTIM.IO

www.testim.io

AI Based Locators

MULTIPLE ATTRIBUTES VS. SINGLE ONE Hundreds of attributes used to identify elements A few changes don’t break the test LOW MAINTENANCE No need to (tediously) update locators MORE EXECUTIONS => MORE STABLE With each execution we learn the changes to the application and optimize scores

slide-35
SLIDE 35

Source: Jason Arbon

www.testim.io

FEATURES Complexity increases exponentially as new features and states interact with existing features TESTS Test coverage grows linearly because they can only be added

  • ne at at time

COVERAGE GAP

THE COVERAGE GAP THE COVERAGE GAP IN SOFTWARE TESTING IN SOFTWARE TESTING

Time Complexity / Coverage

slide-36
SLIDE 36

FUTURE OF AI FUTURE OF AI

slide-37
SLIDE 37

TESTING AUTONOMOUS AUTONOMOUS TESTING

www.testim.io

LEARN BY OBSERVATION (PRODUCTION) AGGREGATE USER ACTIONS INTO FLOWS TEST PRODUCED FROM FLOWS

slide-38
SLIDE 38

www.testim.io

VIA UI TDD AUTHOR FASTER AUTHOR FASTER VIA UI TDD

RUN DURING DEVELOPMENT UPDATE ON FIRST GREEN TEST (MULTIPLE LOCATORS) USE DESIGN MOCKS TO AUTHOR (IMAGE BASED)

slide-39
SLIDE 39

AUTOMATIC RESPONSE AUTOMATIC RESPONSE

www.testim.io SERVERS TESTS’ UI INTERACTION RECORD SERVER RESPONSE

CALL TO SERVER ARE RECORDED

1

RESPOND WITH MOCKS

2

SERVERS TESTS’ UI INTERACTION PLAY SERVER RESPONSE

slide-40
SLIDE 40

www.testim.io www.testim.io

HOW TESTIM IS HOW TESTIM IS PAVING THE FUTURE PAVING THE FUTURE

slide-41
SLIDE 41

QUALITY VELOCITY

Create much more scenarios in a shorter period of time Deploy resources much more effectively create the tests proactively Dev/QA friction is minimized root cause analysis Easier for QA to maximize user coverage Take a risk based approach with true data driven decisions Prevent bugs as opposed to fixing them

THE QUALITY/VELOCITY DILEMMA AUTONOMOUS TESTING ELIMINATES AUTONOMOUS TESTING ELIMINATES THE QUALITY/VELOCITY DILEMMA

www.testim.io

slide-42
SLIDE 42

WRAP IT’S A IT’S A WRAP

Email: raj@testim.io | Website: www.testim.io | Twitter: @epsilon11 | Youtube: Testim Channel

slide-43
SLIDE 43

www.testim.io

https://blog.testim.io/summary-blog-how-ai-is-transforming-softw are-testing/

SUMMARY TALK TALK SUMMARY

Email: raj@testim.io | Website: www.testim.io | Twitter: @epsilon11 | Youtube: Testim Channel