Beyond Unit Testing Steve Loughran Julio Guijarro HP Laboratories, - - PowerPoint PPT Presentation

▶
beyond unit testing
SMART_READER_LITE
LIVE PREVIEW

Beyond Unit Testing Steve Loughran Julio Guijarro HP Laboratories, - - PowerPoint PPT Presentation

Feb 25, 2023 824 likes •1.15k views

Beyond Unit Testing Steve Loughran Julio Guijarro HP Laboratories, Bristol, UK steve.loughran at hpl.hp.com julio.guijarro at hpl.hp.com About Us Julio Guij arro Research scientist at HP Laboratories on Grid-S cale Deployment Leads t

slide-1
SLIDE 1

Steve Loughran Julio Guijarro HP Laboratories, Bristol, UK

Beyond Unit Testing

steve.loughran at hpl.hp.com julio.guijarro at hpl.hp.com

slide-2
SLIDE 2

2

About Us

Julio Guij arro

Research scientist at HP Laboratories on

Grid-S cale Deployment Leads t he S mart Frog open source effort

S t eve Loughran

Research scientist at HP Laboratories on

Grid-S cale Deployment

Apache Ant & Axis committer Co-author of

Java Development wit h Ant

Behind schedule on the 2nd edition

slide-3
SLIDE 3

two different distributed systems

Multi-tier webapp CERN Large Hadron Collider

slide-4
SLIDE 4

How do you show it works?

  • Europe's high-end server farms
  • Years of simulations
  • Nobel Prize winners, Computer

Scientists and physics PhD students

  • An old laptop nobody wants
  • Any spare time before you ship
  • You
slide-5
SLIDE 5

5

Classic unit tests

  • Run in a test harness
  • Don’t stress the system
  • Don’t run on real

servers

  • Don't run with real data
slide-6
SLIDE 6

6

Write less Unit Tests!

A modest proposal

slide-7
SLIDE 7

7

Apply Formal Methods!

  • Integrating Formal Met hods with XP development.
  • How to use axiomat ic t heorem proofs to verify

correctness in a large-scale distributed system.

  • How Milner's π-calculus is the underpinnings for the

BPEL workflow language.

  • Cont inuat ions vs. bisimilar st at e machines -which is

better for correctness proofs?

  • How relaxing your concurrency const raint s results in

higher throughput.

slide-8
SLIDE 8

8

Or: System Testing

slide-9
SLIDE 9

9

System Tests

  • Deploy the app
  • Add a real dataset
  • Use the app server
  • Remotely test from
  • ther sites/hosts
  • Test in the client
  • Are big, complex and

distributed

slide-10
SLIDE 10

10

How to test big systems

  • Simulate the production system.
  • Automate deployment
  • Write functional tests
  • Remote test from clients
slide-11
SLIDE 11

11

Embrace Virtualization

  • VMWare player free; workstation for $£€
  • Create VM images that resemble

production configurations.

  • Deploy and test into virtual machines
  • Host continuous integration server in VMs
  • Simulate complex/broken networks

S imulat e t he product ion syst em Automate deployment Write functional tests Remote test from clients

slide-12
SLIDE 12

12

…and become a cluster admin

  • PXE System Installers: linuxcoe.sf.net
  • Auto-rollback images during test and production
  • Isolate insecure platforms on virtual network
slide-13
SLIDE 13

13

Automate app deployment

  • RPM/APT/.msi packages pushed out to hosts
  • S

mart Frog: http://smartfrog.org/

  • Cargo: http://cargo.codehaus.org
  • Shell Scripts
  • Ant build files using scp, ssh

Simulate the production system Aut omat e deployment Write functional tests Remote test from clients

slide-14
SLIDE 14

14

Database setup

  • Data setup is too time consuming to do every test
  • Use the same DB that production will have.
  • Automated set up of the database
  • keep this DB snapshot and revert to it after a run.

(or the entire virtual machine image)

<mysql-admin> CREATE DATABASE diary; GRANT ALL PRIVILEGES ON diary.* TO 'diary'@'localhost'; SET PASSWORD FOR 'diary'@'localhost' = PASSWORD('${mysql.diary.pass}'); </mysql-admin>

slide-15
SLIDE 15

15

What to test?

  • system health tests
  • In-container unit tests
  • Remote web service/HTML tests
  • In-browser GUI testing
  • Load tests
  • Network failure simulations

…

Simulate the production system Automate deployment Writ e funct ional t est s Remote test from clients

slide-16
SLIDE 16

16

Health Test: “happy pages”

<waitfor maxwait="30" maxwaitunit="second" timeoutproperty="unhappy"> <http url="http://server/happyaxis.jsp"/> </waitfor> <fail if="unhappy"/> <%@ taglib uri="/WEB-INF/diary.tld" prefix="h" %> <body> <ha:happy classMustExist="org.jdom.JDOMException" errorText="JDom missing"/> We are happy </body> </html>

Delegate to machines:

slide-17
SLIDE 17

17

Test in-container with cactus

http://jakarta.apache.org/cactus/

Simulate the production system Automate deployment Writ e funct ional t est s Remote test from clients

slide-18
SLIDE 18

18

Cactus Test Case

http://jakarta.apache.org/cactus/

public class CactusPersistTest extends ServletTestCase { private static int counter = 0; private SessionFactory factory; public void testPersist() throws Exception { Event event = createTestEvent(); Session session = factory.openSession(); try { session.persist(event); } finally { session.close(); } assertEventIsInDB(event); } }

Simulate the production system Automate deployment Writ e funct ional t est s Remote test from clients

slide-19
SLIDE 19

19

<cactus> task choreographs

http://jakarta.apache.org/cactus/

<cactus:cactus warfile="${cactus.war}" errorProperty="cactus.failed" failureProperty="cactus.failed"> <containerset> <generic name="server" port="8080"> <startup> <copy file="${cactus.war}" tofile="${cactus.destfile}"

  • verwrite="true"/>

</startup> <shutdown> <delete file="${cactus.destfile}"/> </shutdown> </generic> </containerset> <classpath><path refid="test.classpath"/></classpath> <formatter type="xml"/> <batchtest todir="${test.data.dir}"> <fileset dir="test" includes="**/*Test.java"> </batchtest> </cactus:cactus>

Simulate the production system Automate deployment Writ e funct ional t est s Remote test from clients

slide-20
SLIDE 20

20

Cactus Demo

− Needs classpath right for client and server − cactus servlet is possible security risk

http://jakarta.apache.org/cactus/

slide-21
SLIDE 21

21

GUI testing hurts

  • Static HTML is the easiest (HttpUnit)
  • Swing, DHTML, SWT, Flash hard.
  • Most people stop at the “model”
  • Whoever does a new GUI -fix this!

Simulate the production system Automate deployment Writ e funct ional t est s Remote test from clients

slide-22
SLIDE 22

22

jsUnit is JUnit for JavaScript

http://www.jsunit.net/

function test3() { var buffer = top.testManager.documentLoader.buffer(); var emps = buffer.document.getElementsByTagName('employee'); assert('expected 5 employees, not ' + emps.length, emps.length == 5); var empid = emps[0].getElementsByTagName('employeeId'); assert('employeeId[0] was ' + empid[0].firstChild.data, empid[0].firstChild.data == 'EMP0001'); }

slide-23
SLIDE 23

23

Selenium: tests in a table

http://www.openqa.org/selenium/

<tr> <td>verifyTitle</td> <td>Click Page Target</td> <td>&nbsp;</td> </tr>

slide-24
SLIDE 24

24

WS Interop Testing

  • Use the real client API/classes
  • Pass down URLs via system properties
  • Test different endpoints in parallel processes
  • Include timeouts; proxy support
  • Log for blame assignment

protected String getOption(String property, boolean required) { String option = System.getProperty(property); if (required && option== null) { fail("No property " + property); } return option; }

ex: http://deployapi.iseran.com:8080/logs/

Simulate the production system Automate deployment Write functional tests Remot e t est from client s

slide-25
SLIDE 25

25

Distributed Testing

  • Allocate & configure test systems
  • Deploy application across nodes
  • Deploy tests on other nodes
  • Collect and correlate results
  • Try to understand what went wrong

Simulate the production system Automate deployment Write functional tests Remot e t est from client s

slide-26
SLIDE 26

26

SmartFrog

A framework for describing, deploying and managing distributed service components.

HttpUnitTests extends JUnitTestSuite { package "d1.webapp.test"; name "HttpUnitTests"; server.url TBD; sfProcessHost "client"; properties [ ["server.url",server.url], ["cactus.contextURL",server.url] ]; classes [ "EventFeedTest", "HappyTagTest", "IndexTest" ]; }

http://smartfrog.org/

  • SmartFrog daemon
slide-27
SLIDE 27

27

Distributed Deployment of App & JUnit

  • SmartFrog daemon
slide-28
SLIDE 28

28

XHTML output of test results

− no x-system summary − no merging of logs from different systems − no notification + ~live output + log capture

slide-29
SLIDE 29

29

Future GUI? GridUnit

http://gridunit.sourceforge.net/

  • Swing GUI for testing on

OurGrid

  • Unit test across many

different machines

  • But not (yet) distributed

applications

  • Aggregate view of results
  • “partial” success
  • Common JUnit wire format
slide-30
SLIDE 30

30

Call to Action

  • Focus on system tests
  • Embrace Virtualization: VMWare, Xen
  • Use Cactus for in-container testing
  • Use Selenium/jsUnit for browser tests
  • Join us in distributed system testing
slide-31
SLIDE 31

31

Junit4?

  • Java5 only
  • Extension tools not there yet
  • Integration with Ant, Maven coming

along.

  • Ant 1.7 <junit> will work with junit4.jar
  • JUnit team plan their own task

(Ant team are working with them)