SLIDE 1
6 Years of Test Automation
- @mikeb2701
Assumptions
- Testing is important
- Automating testing is important
6 Years of Test Automation @mikeb2701 Assumptions Testing is - - PowerPoint PPT Presentation
6 Years of Test Automation @mikeb2701 Assumptions Testing is - - PowerPoint PPT Presentation
6 Years of Test Automation @mikeb2701 Assumptions Testing is important Automating testing is important Types of tests Static Analysis Unit Testing Integration Testing Acceptance Testing Performance Testing
SLIDE 2
SLIDE 3
Types of tests
- Static Analysis
- Unit Testing
- Integration Testing
- Acceptance Testing
- Performance Testing
- Testing in Live
- Exploratory Testing
SLIDE 4
An acceptance test may only drive and assert behaviour in the system through interfaces available to the system’s users.
SLIDE 5
Writing Acceptance Tests
SLIDE 6
@RunWith(AcceptanceTestRunner.class) @Multithreaded @AcceptanceTest(id = 29, flavour = {Flavour.PLACE_ORDER, Flavour.API}) public class PlaceOrderAcceptanceTest extends DslTestCase {
- @Before
public void beforeEveryTest() {
- adminAPI.createInstrument("name: instrument”,
"orderQuantityIncrement: 0.1”,…); registrationAPI.createUser("user", "balance: 100000"); mtfMemberAPI.createMtfMemberAndFixSession("marketMaker"); publicAPI.login("user"); mtfFixAPI.login(“marketMaker"); } // end
- @Test
public void shouldRejectOrderIfNotLoggedIn() { publicAPI.logout(); publicAPI.placeOrder(“instrument", "side: buy", "quantity: 1.2”, "timeInForce: ImmediateOrCancel”, "unauthorised: true");
SLIDE 7
@RunWith(AcceptanceTestRunner.class) @Multithreaded @AcceptanceTest(id = 29, flavour = {Flavour.PLACE_ORDER, Flavour.API}) public class PlaceOrderAcceptanceTest extends DslTestCase {
- @Before
public void beforeEveryTest() {
- adminAPI.createInstrument("name: instrument”,
"orderQuantityIncrement: 0.1”,…); registrationAPI.createUser("user", "balance: 100000"); mtfMemberAPI.createMtfMemberAndFixSession("marketMaker"); publicAPI.login("user"); mtfFixAPI.login(“marketMaker"); } // end
- @Test
public void shouldRejectOrderIfNotLoggedIn() { publicAPI.logout(); publicAPI.placeOrder(“instrument", "side: buy", "quantity: 1.2”, "timeInForce: ImmediateOrCancel”, "unauthorised: true");
SLIDE 8
@RunWith(AcceptanceTestRunner.class) @Multithreaded @AcceptanceTest(id = 29, flavour = {Flavour.PLACE_ORDER, Flavour.API}) public class PlaceOrderAcceptanceTest extends DslTestCase {
- @Before
public void beforeEveryTest() {
- adminAPI.createInstrument("name: instrument”,
"orderQuantityIncrement: 0.1”,…); registrationAPI.createUser("user", "balance: 100000"); mtfMemberAPI.createMtfMemberAndFixSession("marketMaker"); publicAPI.login("user"); mtfFixAPI.login(“marketMaker"); } // end
- @Test
public void shouldRejectOrderIfNotLoggedIn() { publicAPI.logout(); publicAPI.placeOrder(“instrument", "side: buy", "quantity: 1.2”, "timeInForce: ImmediateOrCancel”, "unauthorised: true");
SLIDE 9
@RunWith(AcceptanceTestRunner.class) @Multithreaded @AcceptanceTest(id = 29, flavour = {Flavour.PLACE_ORDER, Flavour.API}) public class PlaceOrderAcceptanceTest extends DslTestCase {
- @Before
public void beforeEveryTest() {
- adminAPI.createInstrument("name: instrument”,
"orderQuantityIncrement: 0.1”,…); registrationAPI.createUser("user", "balance: 100000"); mtfMemberAPI.createMtfMemberAndFixSession("marketMaker"); publicAPI.login("user"); mtfFixAPI.login(“marketMaker"); } // end
- @Test
public void shouldRejectOrderIfNotLoggedIn() { publicAPI.logout(); publicAPI.placeOrder(“instrument", "side: buy", "quantity: 1.2”, "timeInForce: ImmediateOrCancel”, "unauthorised: true");
SLIDE 10
@RunWith(AcceptanceTestRunner.class) @Multithreaded @AcceptanceTest(id = 29, flavour = {Flavour.PLACE_ORDER, Flavour.API}) public class PlaceOrderAcceptanceTest extends DslTestCase {
- @Before
public void beforeEveryTest() {
- adminAPI.createInstrument("name: instrument”,
"orderQuantityIncrement: 0.1”,…); registrationAPI.createUser("user", "balance: 100000"); mtfMemberAPI.createMtfMemberAndFixSession("marketMaker"); publicAPI.login("user"); mtfFixAPI.login(“marketMaker"); } // end
- @Test
public void shouldRejectOrderIfNotLoggedIn() { publicAPI.logout(); publicAPI.placeOrder(“instrument", "side: buy", "quantity: 1.2”, "timeInForce: ImmediateOrCancel”, "unauthorised: true");
SLIDE 11
DSL
PublicAPI TradingUI AdminAPI Test MtfMemberAPI LoginPanel Driver Selenium BalancesPanel Driver InstrumentPanel Driver JavaAPI Driver AdminAPI Driver MtfFixDriver XML/HTTP QuickFIX
Drivers
SLIDE 12
Time Travel
SLIDE 13
Service Service Service Service Service Time Machine Test Time Travel To Time Delta Get Current Time
SLIDE 14
@RunWith(AcceptanceTestRunner.class) @AcceptanceTest(id = 5194, flavour = {Flavour.PLACE_ORDER, Flavour.API}) @Multithreaded(tardis = true) public class CancelAndReplaceAcceptanceTest extends DslTestCase { @Before public void setUp() { dsl.forceToUtcTimeZone(); dsl.enableTimeMachine();
- dsl.createTimePoint("name: origin”,
"value: <next weekday>"); dsl.createTimePoint("name: marketOpen", "value: origin plus 1 weekdayCalendarOpenOffset”);
- dsl.waitUntil("marketOpen");
SLIDE 15
Pro-Tip:
Isolate as much as possible
SLIDE 16
Simple DSL
https://github.com/LMAX-Exchange/Simple-DSL
SLIDE 17
Acceptance Testing (Running)
SLIDE 18
RTA Parallel Sequential Sequential Sequential RTA RTA RTA Scotty Romero Server Jenkins
SLIDE 19
SLIDE 20
SLIDE 21
Intermittency
avp.wikia.com
SLIDE 22
Sources of Intermittency
- Bad code (under test)
- Bad test
- Environmental
SLIDE 23
Just Fix It!
SLIDE 24
SLIDE 25
SLIDE 26
SLIDE 27
SLIDE 28
SLIDE 29
Acceptance Test DB
SLIDE 30
SLIDE 31
SLIDE 32
Wrap-Up
- Write them well
- Run them often
- Keep them green
SLIDE 33