Smart Testing with AI Using Data Mining Presented by: - - PDF document
T16 Analytics in Testing Thursday, October 3rd, 2019 1:30 PM Smart Testing with AI Using Data Mining Presented by: Lorna
¡ ¡ T16 ¡
Analytics ¡in ¡Testing ¡ Thursday, ¡October ¡3rd, ¡2019 ¡1:30 ¡PM ¡ ¡ ¡ ¡ ¡
Presented ¡by: ¡ ¡ ¡
¡ Smartbear ¡ ¡
Brought ¡to ¡you ¡by: ¡ ¡ ¡ ¡
¡
¡
¡ ¡
888-‑-‑-‑268-‑-‑-‑8770 ¡·√·√ ¡904-‑-‑-‑278-‑-‑-‑0524 ¡-‑ ¡info@techwell.com ¡-‑ ¡http://www.starwest.techwell.com/ ¡ ¡ ¡
¡
¡ ¡ ¡
¡
¡ Lorna ¡Smyth ¡is ¡from ¡Galway ¡on ¡the ¡west ¡coast ¡of ¡Ireland. ¡She ¡is ¡currently ¡a ¡technical ¡ sales ¡engineer ¡for ¡SmartBear's ¡test ¡automation ¡tools. ¡She ¡has ¡worked ¡in ¡various ¡IT ¡ roles ¡over ¡the ¡past ¡eight ¡years, ¡including ¡technical ¡analyst, ¡data ¡migration ¡test ¡lead, ¡ and ¡software ¡licensing ¡specialist. ¡She ¡specializes ¡in ¡test ¡automation ¡and ¡delivers ¡ webinars, ¡technical ¡demonstrations, ¡and ¡deep-‑dive ¡troubleshooting ¡calls ¡on ¡test ¡ automation ¡for ¡SmartBear’s ¡customers. ¡Lorna ¡is ¡an ¡advocate ¡for ¡women ¡in ¡IT ¡and ¡is ¡ heavily ¡involved ¡in ¡hosting ¡local ¡networking ¡events. ¡She ¡also ¡participates ¡in ¡ community ¡outreach ¡to ¡drive ¡awareness ¡of ¡IT ¡roles ¡in ¡the ¡West ¡of ¡Ireland. ¡Lorna ¡is ¡ an ¡adventure ¡and ¡sports ¡enthusiast, ¡which ¡has ¡led ¡her ¡to ¡living ¡in ¡Canada, ¡Norway, ¡ and ¡Austria, ¡following ¡her ¡love ¡of ¡the ¡mountains. ¡Lorna ¡has ¡recently ¡returned ¡to ¡ Ireland ¡and ¡is ¡loving ¡life ¡working ¡in ¡test ¡automation. ¡ ¡
8/19/2019 1
8/19/2019 2 ✓ What is Artificial Intelligence(AI)? ✓ How is AI transforming the world? ✓ What is Data Mining? ✓ How can AI address Software Testing Challenges? ✓ Benefits and Challenges of AI in Testing ✓ How can you Embrace AI?
“Your biggest fear?” “ Intelligence demonstrated by machines, in contrast to the natural intelligence displayed by humans.” “Computers that mimic cognitive functions that humans associate with the human mind, such as learning and problem solving” “Emphasizes the creation of intelligent machines that work and react like humans” “a conglomeration of concepts and technologies that mean different things to different people” “Something that is going to change the world…”
8/19/2019 3
Neural networks Machine learning Deep learning
the networks of hardware and software that approximate the web of neurons in the human brain. a technique using algorithms to teach machines to learn. helps machines learn to go deeper into data to recognize patterns.
AI is the broad category of methodologies that teach a computer to perform tasks as an “intelligent” person would.
8/19/2019 4
(Source: Google)
the jobs in the UK, 21% in Japan, and 35% in Germany. (Source: PwC)
8/19/2019 5
8/19/2019 6
Ever wondered how the Wimbledon editorial team focus their content?
8/19/2019 7
Virtual Nursing Assistants Robot Assisted Surgery Administrative Workflow Fraud Detection Dosage Error Reduction Connected Machines Preliminary Diagnosis Image Based Diagnosis Clinical Trial Participation Patient Engagement Bumrungrad International Hospital, Thailand: 83% concordance
83%
211 patients with breast, colorectal, gastric and lung cancer
Manipal Comprehensive Hospital, India: 73% concordance
638 patients with breast cancer
Gachon University, Gil Medical Center, South Korea: 49% concordance
656 patients with colon cancer
73% 49%
8/19/2019 8
Low; recommend relevant financial products.
which can be delegated to a machine.
seconds.
8/19/2019 9
Image source: AIAnalyticsHub
8/19/2019 10
Data Mining Techniques:
Classification Clustering Regression Outer Sequential Patterns Association Rules Prediction
Classification: Retrieve information and classify into different classes. Clustering: Identify data that is like each other. Regression: Identify and analyze relationship between variables.
Data Mining Techniques:
class as ‘likely to fail’.
Functional, System).
8/19/2019 11
Outer Detection: Identify items that do not match an expected pattern/behavior.
Data Mining Techniques:
Association Rules: Identify associations between two or more items – discovers hidden patterns..
likely to need a loan between $10,000-$15,000.
before. Prediction: combination of data mining techniques, analyzes past events or instances in a sequence for predicting a future event. Sequential Patterns: Identify patterns in transaction data for a certain time period.
Data Mining Techniques:
8/19/2019 12
Understand Business Problem
Verify & Deploy Evaluate, Train & Test Model Model Data Understand Data Prepare & Process Data
8/19/2019 13
8/19/2019 14
Automation Capability High Smart Bulky Automation Tools More time for Business Goals Speed Quality Cost Culture Innovation SDLC Time Taken Weeks Hours Low Robust Automation Tools Continuous Testing Autonomous Testing Manual Testing Waterfall Early Agile Agile Devops CI/CD
8/19/2019 15
make training machine learning systems easier and more accurate.
testing. ➢ Testing is a good use case as it’s built on a logical foundation ➢ Companies are struggling to keep up with user expectations.
TEST DESIGN TEST MAINTENANCE AND EXECUTION
8/19/2019 16
INTELLIGENT TEST DESIGN
Advanced Object Recognition Framework Generation Responsive Web Design Risk Profiling
Risk Profiling Prioritize tests based on business risk by environment, configuration, and different aspects of your application Object Recognition Automatically recognize new objects and updates to add them to the DOM and structure without manual effort Framework Generation Automatically scan your application to recommend a test framework
8/19/2019 17
Neural Network for Testing
Each application update and a new test case can act as inputs to help maintain an application.
Example Outputs:
systems
learning
Example Inputs:
controls
controls
INTELLIGENT TEST MAINTENANCE & EXECUTION
Predictive Self- Healing Intelligent Bug Hunting Application Resilience Process Automation
8/19/2019 18 Predictive Self-healing Dynamically update your test suite when your application changes or evolves Application Resilience Netflix Simian Army (Chaos Monkeys) Achieving resilience by applying predictive auto-scaling and continuous fitness functions Intelligent Bug Hunting Discover bugs in the application through AI- powered exploratory testing Process Automation Automate business workflows for end-to-end testing
quicker than before.
defect of a new release.
Source: Oleksii Kharkovyna
8/19/2019 19
Meaningful tests require varied and large test data sets.
Google DeepMind created an AI program that utilizes deep reinforcement learning to play video games by itself, thus, producing a huge amount of test data that they can learn from.
8/19/2019 20 Testim uses AI to speed up the authoring, execution, and maintenance of automated tests.
bots will start testing the application. ➢Finds bugs, crashes and error dialogs ➢Builds reports on this data.
the tool will adjust and identify it without any user intervention.
8/19/2019 21
going backwards and forwards in between screens).
actions, and data.
been executed by actual users.
Uses Computer Vision algorithms to perform automated visual testing.
8/19/2019 22
Appium Classifier Plugin: Allows you to automatically find an icon that’s visible in the application under test. Pre-requisite is that the icon must look like one of the pre-trained labels like “cart”, “edit”, and “twitter”. Appvance makes use of AI to generate test cases based on user behavior.
8/19/2019 23
AnalyseTest Results Automatically
everyone Algorithms are using the historical data in the dashboard database
hard-to-find application components such as charts, mainframes, PDFs and SAP applications.
In SmartBear’s automated functional testing tool, they’re leveraging AI to improve
8/19/2019 24
Notify you of any unused objects and changes made to your application.
8/19/2019 25
✓ Quicker releases
✓ Improved quality – less human error, increased test coverage ✓ Eliminates flaky tests – can automatically update ✓ Find issues quickly – less impact to customers ✓ Accelerate results analysis – make smart decisions quicker ✓ Increased traceability – defects and test case coverage
8/19/2019 26
having the right data infrastructure
scientist or mathematician.
success of your first project, that success needs to be measurable.
existing solutions and tools, or a mix?
8/19/2019 27
8/19/2019 28
It’s here, embrace it – AI is changing our world, are you willing to change with it? Resources:
AI may be executing our tests in the future, but testers and test automation engineers will still be needed as the subject matter experts to teach the AI.
8/19/2019 29
8/19/2019 30