Software Engineering I (02161) Testing Assoc. Prof. Hubert - - PowerPoint PPT Presentation

Software Engineering I (02161)

Testing

• Assoc. Prof. Hubert Baumeister

DTU Compute Technical University of Denmark

Spring 2020

Types of Tests

Technology Facing Business Facing Supports Programming Critique Project Unit Tests Acceptance Tests Property Testing (Performance, Scalability) Explorative Testing (trying to break the system)

Acceptance Tests

◮ Tests defined by / with the help of the user

◮ based on the requirements

◮ Traditionally

◮ manual tests ◮ after the software is delivered

◮ Agile software development

◮ automatic tests: JUnit, Cucumber, . . . ◮ created before the user story / use case scenario is implemented ◮ developed with the customer → XP practice: customer on-site

Example of acceptance tests

◮ Login feature

actor: Admin success

• 1. Start the application
• 2. Admin enters correct password
• 3. System responds true

fail

• 1. Start the application
• 2. Admin enters wrong password

3 The system reports that the password is wrong and asks to enter the correct passowrd

Manual tests

Successful login

Prerequisit: the password for the administrator is “adminadmin” Input Step Expected Output Fail OK Startup system “0) Exit” “1) Login as administrator” “1” Enter choice “password” “adminadmin” Enter string “logged in”

Failed login

Prerequisit: the password for the administrator is “adminadmin” Input Step Expected Output Fail OK Startup system “0) Exit” “1) Login as administrator” “1” Enter choice “password” “admin” Enter string “Password incorrect” “0) Exit” “1) Login as administrator”

Manual vs. automated tests

◮ Manual tests should be avoided

◮ Are expensive; can’t be run often

◮ Automated tests

◮ Are cheap; can be run often

◮ Robert Martin (Uncle Bob) in http://www.youtube.com/watch?v=hG4LH6P8Syk

◮ manual tests are immoral from 36:35 ◮ how to test applications having a UI from 40:00

◮ How to do UI tests?

→ Solution: Test under the UI

Acceptance Tests and Agile Software Development

◮ Automated tests: JUnit, Cucumber, . . . ◮ Created before the user story / use case scenario is implemented ◮ Developed together with the customer

→ tests need to be readable by the customer → Cucumber

◮ How to do UI tests?

→ Solution: Test under the UI

Test under the UI

Tests Business Logic Domain Layer e.g. User, Book, ... Business logic Persistency Layer User Application Layer e.g. LibraryApp Business logic Thin Presentation Layer No Business Logic

Acceptance test

◮ Framework for Integrated Tests (Fit) → use tables ◮ Cucumber

Feature: Admin login Description: The administrator logs into the library system Actor: Administrator Scenario: Administrator can login Given that the administrator is not logged in And the password is "adminadmin" When the administrator logs in Then the adminstrator is logged in Scenario: Administrator has the wrong password Given that the administrator is not logged in And the password is "wrong password" When the administrator logs in Then the login fails

Cucumber

◮ Behaviour-Driven Development: User Stories

Feature: Name of the feature Description ... Scenario: Name Description ... Given an initial state And ... When an action happens And ... Then an assertion is true And ...

◮ Gherkin: for scenarios (supports different languages, e.g. Danish and Pirate :-) ◮ Programming language (Java): Glue code

Example: Add book

Business experts write Cucumber features

Feature: Add book Description: A book is added to the library Actors: Administrator Scenario: Add a book successfully Given that the administrator is logged in And a book with title "Extreme Programming", author "Kent Beck", and signature "Beck99" When the book is added to the library Then the book is contained in the library

Programmers implement ”meaning” of each step

@Given("that the administrator is logged in") public void thatTheAdministratorIsLoggedIn() throws Exception { libraryApp.adminLogin("adminadmin"); }

How does Cucumber work?

◮ The Cucumber test runner parses the feature file

@RunWith(Cucumber.class) @CucumberOptions(features = "use_cases", glue = { "dtu.library.acceptance_tests"}, strict = true) public class AcceptanceTest {}

◮ For each scenario: go through all step definitions ◮ Find the method whose annotation matches

◮ class does not matter (→ no duplicated step definitions) → put steps in classes corresponding to concepts

◮ e.g. users → UserSteps; books → BookSteps

→ ubiquitous language

◮ Create objects of step classes

◮ create all objects used in the constructors ◮ E.g. create LibraryApp and UserHelper object for BookSteps(LibraryApp l, UserHelper h) and UserHelper(LibraryApp l, UserHelper h)

◮ Run the methods according to the scenario

Step definitions: AdministratorSteps

public class AdministratorSteps { private LibraryApp libraryApp; public AdministratorSteps(LibraryApp l) { libraryApp = l; } @Given("that the administrator is logged in") public void thatTheAdministratorIsLoggedIn() throws Exception { libraryApp.adminLogin("adminadmin"); } }

Step definitions: BookSteps

public class BookSteps { private LibraryApp libraryApp; private BookHelper bookHelper; public BookSteps(LibraryApp l, BookHelper h) { libraryApp = l; bookHelper = h; } @Given("a book with title {string}, author {string}, and signature {string}") public void aBookWithTitleAuthorAndSignature(String title, String author, String signature) throws Exception { bookHelper.createBook(title,author,signature); } @When("the book is added to the library") public void theBookIsAddedToTheLibrary() throws Exception { try { libraryApp.addBook(bookHelper.getBook()); } catch (OperationNotAllowedException e) { errorMessage = e.getMessage(); } } @Then("the library contains the book") public void theLibraryContainsTheBook() throws Exception { assertTrue(libraryApp.containsBookWithSignature( bookHelper.getBook().getSignature())); } }

BookHelper

public class BookHelper { private Book book; public void createBook(String title, String author, String signature) { book = new Book(title, author, signature); } public void getBook() { if (book == null) { createDefaultBook(); } return book; } private void createDefaultBook() { createBook("some title", "some author", "sig001"); } }

Meaning of Given, Then, and When

◮ For Cucumber, Given, Then, and When have no meaning.

◮ They are all treated the same → Given, Then, and When cannot have the same text ◮ They have only meaning for us

◮ Given <something>

◮ either checkes, using assertions, that a given <something> is true ◮ e.g. assertTrue(<something>) ◮ or runs actions which make <something> true

◮ When <action>

◮ execute <action>

◮ Then <condition>

◮ Check, using assertions, that <condition> is true ◮ e.g. assertTrue(<something>)

Assertions (inherited from JUnit)

◮ General assertions

import static org.junit.Assert.*; assertTrue(bexp) assertTrue(msg,bexp)

◮ Failing assertions throw an AssertionError exception ◮ Specialised assertions for readability

import static org.hamcrest.MatcherAssert.assertThat; import static org.hamcrest.CoreMatchers.*;

• 1. assertFalse(bexp) / assertThat(bexp,is(false))
• 2. fail()
• 3. assertEquals(exp,act) / assertThat(act,is(equalTo(exp)))
• 4. assertNull(act) / assertThat(act,is(nullValue()))
• 5. assertNotNull(act) / assertThat(act,is(not(nullValue())))

...

How to deal with checked exceptions?

◮ In LibraryApp

public addBook(Book b) throws OperationNotAllowedException {..}

◮ Step definition does not compile

@When("the book is added to the library") public theBookIsAddedToTheLibrary() { libraryApp.addBook(bookHolder.getBook()); }

◮ ”Standard” solution

try { .. } catch (MyException e) { e.printStackTrace(); }

◮ Better:

1 Should never happen → fail fast

@When("the book is added to the library") public theBookIsAddedToTheLibrary() throws Exception { libraryApp.addBook(bookHolder.getBook()); }

2 testing for the exception

@When("the book is added to the library") public theBookIsAddedToTheLibrary() { try {libraryApp.addBook(bookHolder.getBook());} catch (OperationNotAllowedException e) { errorMessageHolder.setErrorMessage(e.getMessage()); } }

3 In production code, handle, add throws clause, or convert to Error

try {....} catch (MyException e) { throw new Error(e); }