6 Years of Test Automation @mikeb2701 Assumptions Testing is - PowerPoint PPT Presentation

6 Years of Test Automation � @mikeb2701

Assumptions • Testing is important • Automating testing is important

Types of tests • Static Analysis • Unit Testing • Integration Testing • Acceptance Testing � • Performance Testing • Testing in Live • Exploratory Testing

An acceptance test may only drive and assert behaviour in the system through interfaces available to the system’s users.

Writing Acceptance Tests

@RunWith(AcceptanceTestRunner.class) @Multithreaded @AcceptanceTest(id = 29, flavour = {Flavour.PLACE_ORDER, Flavour.API}) public class PlaceOrderAcceptanceTest extends DslTestCase { � @Before public void beforeEveryTest() { � adminAPI.createInstrument("name: instrument”, "orderQuantityIncrement: 0.1”,…); registrationAPI.createUser("user", "balance: 100000"); mtfMemberAPI.createMtfMemberAndFixSession("marketMaker"); publicAPI.login("user"); mtfFixAPI.login(“marketMaker"); } // end � @Test public void shouldRejectOrderIfNotLoggedIn() { publicAPI.logout(); publicAPI.placeOrder(“instrument", "side: buy", "quantity: 1.2”, "timeInForce: ImmediateOrCancel”, "unauthorised: true"); �

@RunWith(AcceptanceTestRunner.class) @Multithreaded @AcceptanceTest(id = 29, flavour = {Flavour.PLACE_ORDER, Flavour.API}) public class PlaceOrderAcceptanceTest extends DslTestCase { � @Before public void beforeEveryTest() { � adminAPI.createInstrument("name: instrument”, "orderQuantityIncrement: 0.1”,…); registrationAPI.createUser("user", "balance: 100000"); mtfMemberAPI.createMtfMemberAndFixSession("marketMaker"); publicAPI.login("user"); mtfFixAPI.login(“marketMaker"); } // end � @Test public void shouldRejectOrderIfNotLoggedIn() { publicAPI.logout(); publicAPI.placeOrder(“instrument", "side: buy", "quantity: 1.2”, "timeInForce: ImmediateOrCancel”, "unauthorised: true"); �

@RunWith(AcceptanceTestRunner.class) @Multithreaded @AcceptanceTest(id = 29, flavour = {Flavour.PLACE_ORDER, Flavour.API}) public class PlaceOrderAcceptanceTest extends DslTestCase { � @Before public void beforeEveryTest() { � adminAPI.createInstrument("name: instrument”, "orderQuantityIncrement: 0.1”,…); registrationAPI.createUser("user", "balance: 100000"); mtfMemberAPI.createMtfMemberAndFixSession("marketMaker"); publicAPI.login("user"); mtfFixAPI.login(“marketMaker"); } // end � @Test public void shouldRejectOrderIfNotLoggedIn() { publicAPI.logout(); publicAPI.placeOrder(“instrument", "side: buy", "quantity: 1.2”, "timeInForce: ImmediateOrCancel”, "unauthorised: true"); �

@RunWith(AcceptanceTestRunner.class) @Multithreaded @AcceptanceTest(id = 29, flavour = {Flavour.PLACE_ORDER, Flavour.API}) public class PlaceOrderAcceptanceTest extends DslTestCase { � @Before public void beforeEveryTest() { � adminAPI.createInstrument("name: instrument”, "orderQuantityIncrement: 0.1”,…); registrationAPI.createUser("user", "balance: 100000"); mtfMemberAPI.createMtfMemberAndFixSession("marketMaker"); publicAPI.login("user"); mtfFixAPI.login(“marketMaker"); } // end � @Test public void shouldRejectOrderIfNotLoggedIn() { publicAPI.logout(); publicAPI.placeOrder(“instrument", "side: buy", "quantity: 1.2”, "timeInForce: ImmediateOrCancel”, "unauthorised: true"); �

@RunWith(AcceptanceTestRunner.class) @Multithreaded @AcceptanceTest(id = 29, flavour = {Flavour.PLACE_ORDER, Flavour.API}) public class PlaceOrderAcceptanceTest extends DslTestCase { � @Before public void beforeEveryTest() { � adminAPI.createInstrument("name: instrument”, "orderQuantityIncrement: 0.1”,…); registrationAPI.createUser("user", "balance: 100000"); mtfMemberAPI.createMtfMemberAndFixSession("marketMaker"); publicAPI.login("user"); mtfFixAPI.login(“marketMaker"); } // end � @Test public void shouldRejectOrderIfNotLoggedIn() { publicAPI.logout(); publicAPI.placeOrder(“instrument", "side: buy", "quantity: 1.2”, "timeInForce: ImmediateOrCancel”, "unauthorised: true"); �

DSL Drivers LoginPanel Driver BalancesPanel Driver Selenium Test TradingUI InstrumentPanel Driver PublicAPI JavaAPI Driver XML/HTTP AdminAPI AdminAPI Driver QuickFIX MtfFixDriver MtfMemberAPI

Time Travel

Time Travel To Time Test Machine Get Current Time Delta Time Service Service Service Service Service

@RunWith(AcceptanceTestRunner.class) @AcceptanceTest(id = 5194, flavour = {Flavour.PLACE_ORDER, Flavour.API}) @Multithreaded(tardis = true) public class CancelAndReplaceAcceptanceTest extends DslTestCase { @Before public void setUp() { dsl.forceToUtcTimeZone(); dsl.enableTimeMachine(); � dsl.createTimePoint("name: origin”, "value: <next weekday>"); dsl.createTimePoint("name: marketOpen", "value: origin plus 1 weekdayCalendarOpenOffset”); � dsl.waitUntil("marketOpen");

Pro-Tip: Isolate as much as possible

Simple DSL https://github.com/LMAX-Exchange/Simple-DSL

Acceptance Testing (Running)

Jenkins Romero Scotty Server Parallel RTA Sequential RTA Sequential RTA Sequential RTA

Intermittency avp.wikia.com

Sources of Intermittency • Bad code (under test) • Bad test • Environmental

Just Fix It!

Acceptance Test DB

Wrap-Up • Write them well • Run them often • Keep them green

Thank you Q&A