quickcheck
play

QuickCheck John Hughes DEMO Registry tests on multiple nodes - PowerPoint PPT Presentation

Jan 06, 2023 •989 likes •1.29k views

QuickCheck John Hughes DEMO Registry tests on multiple nodes 40000 35000 30000 Tests per minute 25000 20000 15000 10000 5000 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 Nodes Four core/8HT SkyLake i7

  1. QuickCheck John Hughes

  2. DEMO

  3. Registry tests on multiple nodes 40000 35000 30000 Tests per minute 25000 20000 15000 10000 5000 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 Nodes Four core/8HT SkyLake i7 Dual core/4HT Ivy Bridge i7

  4. Architecture Count tests, print dots, start and stop Master Worker Worker Worker Worker Generate and run tests

  5. How can the Master count the tests? Master Worker Worker Worker Worker

  6. How can the Master stop the workers? Master Worker Worker Worker Worker

  7. Benchmark Bouncer Mirror 1.2 million/second

  8. Multiple bouncers Bouncer Mirror Bouncer Mirror Bouncer Mirror

  9. Message round-trips per second Four core/8 thread i7 4500000 Round trips per second 4000000 3500000 3000000 2500000 2000000 1500000 1000000 500000 0 1 2 3 4 5 6 7 8 9 10 P--number of bouncers P bouncers, P mirrors

  10. Multiple bouncers, one mirror Bouncer Bouncer Mirror Bouncer

  11. Message round-trips per second Four core/8 thread i7 4500000 4000000 Round trips per second 3500000 3000000 2500000 2000000 1500000 1000000 500000 0 1 2 3 4 5 6 7 8 9 10 P--number of bouncers P bouncers, P mirrors P bouncers, 1 mirror

  12. Batch permissions? Master Worker Worker Worker Worker BUT this may delay termination!

  13. Alternative Architecture Master Manager Manager Manager Manager Yes Another? Worker Worker Worker Worker

  14. • Every worker • Stopping can be communicates with its slightly delayed own manager— scalable!

  15. > eqc:quickcheck(examples:prop_reverse()). .................................................................................................... OK, passed 100 tests true > eqc:quickcheck(examples:prop_reverse()). ....................................................................................................(x10).(x1)......... OK, passed 119 tests true > eqc:quickcheck(eqc:in_parallel(examples:prop_reverse())). ....................................................................................................(x10)..(x1)...... OK, passed 126 tests true > eqc:quickcheck(eqc:on_nodes(examples:prop_reverse())). ....................................................................................................(x10)................................ ........................(x1)......... OK, passed 669 tests But how bad is it to run a few extra tests?

  16. What about node placement? Master Manager Manager Manager Manager Yes Another? Worker Worker Worker Worker

  17. What about node placement? Master Manager Manager Manager Manager Worker Worker Worker Worker

  18. What about node placement? Master Manager Manager Manager Manager Yes Another? Worker Worker Worker Worker

  19. P bouncers, 1 mirror, different nodes Four core/8 HT i7 160000 140000 120000 100000 80000 60000 40000 20000 0 1 2 3 4 5 6 7 8 9 10

  20. P bouncers, 1 mirror Four core/8HT i7 1600000 1400000 1200000 1000000 10-30x slower from a 800000 different node! 600000 400000 200000 0 1 2 3 4 5 6 7 8 9 10 Different nodes Same node

  21. P bouncers, 1 mirror Bouncers on dual core laptop, mirror on quad core 30000 25000 20000 15000 10000 5000 0 0 5 10 15 20 25 30 35

  22. Bounces per second (single bouncer) 1400000 1200000 1000000 800000 600000 400000 200000 0 Same LAN Same Host Same Node

  23. Bounces per second (single bouncer) 7 6 5 30x Log scale 4 50x 3 2 1 0 Same LAN Same Host Same Node

  24. What about success messages? Master Manager Manager Manager Manager Success Worker Worker Worker Worker

  25. Two-way vs one-way Bouncer Mirror 1.2 million/second 5.4 million/second

  26. Bounces/Messages per second 8 4x 10x 7 100x 6 Log scale 5 4 3 2 1 0 Same LAN Same Host Same Node Two way One way

  27. One more optimization… Send the total every There are a LOT of Master 100ms success messages Count them! Manager Manager Manager Manager Success Worker Worker Worker Worker

  28. • Tripled the speed of • Stopping can be even quickcheck(true)! more delayed

  29. Lessons • There is at least an order of magnitude difference between communication costs • Within a node • Between nodes • Between hosts • Latency is much worse affected than bandwidth • This affects design for performance • Favours asynchronous over synchronous communication between nodes • Optimising performance may require changes to observable behaviour • … and we didn’t even consider fault tolerance

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

QuickCheck John Hughes Chalmers University/Quviq AB What is QuickCheck? A library for

QuickCheck John Hughes Chalmers University/Quviq AB What is QuickCheck? A library for

Specification Based Testing with QuickCheck John Hughes Chalmers University/Quviq AB What is QuickCheck? A library for writing and testing properties of program code Some code: A property: Properties as Code A test data A

1.08k views • 72 slides
Be#er Tes(ng with Less Work: QuickCheck Tes(ng in Prac(ce

Be#er Tes(ng with Less Work: QuickCheck Tes(ng in Prac(ce

Be#er Tes(ng with Less Work: QuickCheck Tes(ng in Prac(ce John Hughes QuickCheck in a Nutshell Test case Test case Test case Proper(es Test

820 views • 49 slides
QuickCheck: Beyond the Basics Dave Laing May 13, 2014 Basic QuickCheck Some properties

QuickCheck: Beyond the Basics Dave Laing May 13, 2014 Basic QuickCheck Some properties

QuickCheck: Beyond the Basics Dave Laing May 13, 2014 Basic QuickCheck Some properties reverse_involutive :: [Int] -> Bool reverse_involutive xs = xs == (reverse . reverse) xs sort_idempotent :: [Int] -> Bool sort_idempotent xs =

807 views • 38 slides
QuickCheck 11.11 The two boxes are on a frictionless surface. They had been sitting together at

QuickCheck 11.11 The two boxes are on a frictionless surface. They had been sitting together at

QuickCheck 11.11 The two boxes are on a frictionless surface. They had been sitting together at rest, but an explosion between them has just pushed them apart. How fast is the 2 kg box going? A. 1 m/s B. 2 m/s C. 4 m/s D. 8 m/s

834 views • 4 slides
QuickCheck 11.2 A 2.0 kg object moving to the right with speed 0.50 m/s experiences the force

QuickCheck 11.2 A 2.0 kg object moving to the right with speed 0.50 m/s experiences the force

QuickCheck 11.2 A 2.0 kg object moving to the right with speed 0.50 m/s experiences the force shown. What are the objects speed and direction after the force ends? A. 0.50 m/s left B. At rest C. 0.50 m/s right D. 1.0 m/s right E. 2.0 m/s

178 views • 4 slides
QuickCheck 24.12 A cylindrical Gaussian surface surrounds an infinite line of charge. The flux

QuickCheck 24.12 A cylindrical Gaussian surface surrounds an infinite line of charge. The flux

QuickCheck 24.12 A cylindrical Gaussian surface surrounds an infinite line of charge. The flux e through the two flat ends of the cylinder is A. 0 2 2 rE B. 2 r 2 E C. 2 rLE D. E. It will require an integration to find

697 views • 6 slides
Ch 22 HW due Friday, January 17 @ 23:59 QuickCheck 23.2 What is the direction of the

Ch 22 HW due Friday, January 17 @ 23:59 QuickCheck 23.2 What is the direction of the

Ch 22 HW due Friday, January 17 @ 23:59 QuickCheck 23.2 What is the direction of the electric field at the dot? E. The field is zero. QuickCheck 23.2 What is the direction of the electric field at the dot? E. The field is zero.

246 views • 7 slides
QuickCheck 4.2 A heavy red ball is released from rest 2.0 m above a flat, horizontal surface. At

QuickCheck 4.2 A heavy red ball is released from rest 2.0 m above a flat, horizontal surface. At

QuickCheck 4.2 A heavy red ball is released from rest 2.0 m above a flat, horizontal surface. At exactly the same instant, a yellow ball with the same mass is fired horizontally at 3.0 m/s. Which ball hits the ground first? A. The red ball

1.01k views • 4 slides
Quiz #2 Friday, March 20 QuickCheck 28.14 The three bulbs are identical and the two

Quiz #2 Friday, March 20 QuickCheck 28.14 The three bulbs are identical and the two

Ch 26 HW due Friday, March 6 (today) @ 23:59 Quiz #2 Friday, March 20 QuickCheck 28.14 The three bulbs are identical and the two batteries are identical. Compare the brightnesses of the bulbs. A B C A. A C B B. C.

338 views • 7 slides
Graphical editing support for QuickCheck models Thomas Arts, Kirill Bogdanov, Alex Gerdes, John

Graphical editing support for QuickCheck models Thomas Arts, Kirill Bogdanov, Alex Gerdes, John

Graphical editing support for QuickCheck models Thomas Arts, Kirill Bogdanov, Alex Gerdes, John Hughes This project has received funding from the EU FP7 Collaborative project PROWESS , grant number 317820, http://www.prowessproject.eu

127 views • 12 slides
Testing monadic programs using QuickCheck and state machine based models Stevan Andjelkovic

Testing monadic programs using QuickCheck and state machine based models Stevan Andjelkovic

Testing monadic programs using QuickCheck and state machine based models Stevan Andjelkovic 2018.2.23, BOBKonf (Berlin) Introduction Property based testing in general How to apply property based testing to stateful and concurrent

474 views • 18 slides
due Friday, January 31 @ 23:59 Quiz #1 Friday, January 31 QuickCheck 24.8 Which spherical

due Friday, January 31 @ 23:59 Quiz #1 Friday, January 31 QuickCheck 24.8 Which spherical

Ch 24 HW due Friday, January 31 @ 23:59 Quiz #1 Friday, January 31 QuickCheck 24.8 Which spherical Gaussian surface has the larger electric flux? A. Surface A B. Surface B C. They have the same flux. D. Not enough information

379 views • 8 slides
QuickCheck 10.2 Starting from rest, a marble first rolls down a steeper hill, then down a less

QuickCheck 10.2 Starting from rest, a marble first rolls down a steeper hill, then down a less

QuickCheck 10.2 Starting from rest, a marble first rolls down a steeper hill, then down a less steep hill of the same height. For which is it going faster at the bottom? A. Faster at the bottom of the steeper hill. B. Faster at the bottom

352 views • 4 slides
Suppose 1. we wanted to design a machine architecture

Suppose 1. we wanted to design a machine architecture

Test-Driven Development of an Informa3on-Flow ISA A QuickCheck Adventure Arthur Azevedo de Amorim, Catalin Hritcu, John Hughes, Leonidas Lampropoulos, Ulf

684 views • 37 slides
5 7 pm in ACS 130 Midterm Friday, February 14 No tutorials this week No

5 7 pm in ACS 130 Midterm Friday, February 14 No tutorials this week No

QSCU Midterm Review session Wednesday, February 12 (today) 5 7 pm in ACS 130 Midterm Friday, February 14 No tutorials this week No homework this week QuickCheck 26.1 This is a graph of the x -component of the

201 views • 7 slides
Optional Ch. 25 28 review problems on Mastering Physics Quiz #2 Friday, March 20

Optional Ch. 25 28 review problems on Mastering Physics Quiz #2 Friday, March 20

Ch 27 HW due Friday, March 13 (today) @ 23:59 Optional Ch. 25 28 review problems on Mastering Physics Quiz #2 Friday, March 20 QuickCheck 28.19 The capacitor is initially unchanged. Immediately after the switch closes, the

305 views • 7 slides
Unifying Mirror Symmetry Constructions David Favero favero@ualberta.ca University of Alberta

Unifying Mirror Symmetry Constructions David Favero favero@ualberta.ca University of Alberta

Unifying Mirror Symmetry Constructions Unifying Mirror Symmetry Constructions David Favero favero@ualberta.ca University of Alberta May 2016 Korean Institute for Advanced Study Slides available at: www.ualberta.ca/ favero Unifying Mirror

587 views • 41 slides
Linear Sigma Models and (0 , 2) Mirror Symmetry Ilarion Melnikov University of Chicago String

Linear Sigma Models and (0 , 2) Mirror Symmetry Ilarion Melnikov University of Chicago String

Linear Sigma Models and (0 , 2) Mirror Symmetry Ilarion Melnikov University of Chicago String Phenomenology 2008, University of Pennsylvania Based on work in progress with Jock McOrist. Summary Mirror Symmetry is a powerful tool in the study

234 views • 10 slides
Evidence for (Infinitely Diverse) Non-Convex Mirrors Tristan Hbsch @ Southeastern

Evidence for (Infinitely Diverse) Non-Convex Mirrors Tristan Hbsch @ Southeastern

Evidence for (Infinitely Diverse) Non-Convex Mirrors Tristan Hbsch @ Southeastern Regional Mathematical String Theory Meeting V-Tech University, Blacksburg VA; 2017.10.07 Departments of Physics & Astronomy and Mathematics,

864 views • 30 slides
D i s a s t e r R e c o v e r y a n d C e p h B l o c k S t o r a

D i s a s t e r R e c o v e r y a n d C e p h B l o c k S t o r a

D i s a s t e r R e c o v e r y a n d C e p h B l o c k S t o r a g e I n t r o d u c i n g M u l t i - S i t e M i r r o r i n g J a s o n D i l l a m a n R B D P r

344 views • 22 slides
Lecture 12- ECE 240a Threshold Mirror Loss Ver Chap. 8-9 Threshold Conditions Homogeneous

Lecture 12- ECE 240a Threshold Mirror Loss Ver Chap. 8-9 Threshold Conditions Homogeneous

Lecture 12- ECE 240a Review Small Signal Gain Resonant Frequencies Lasing Conditions Lasing Lecture 12- ECE 240a Threshold Mirror Loss Ver Chap. 8-9 Threshold Conditions Homogeneous Gain Media Gain Saturation 1 ECE 240a Lasers -

425 views • 40 slides
Reflection-based Word Attribute Transfer Background Analogy Analogy in the embedding space

Reflection-based Word Attribute Transfer Background Analogy Analogy in the embedding space

Nara Institute of Science and Technology (NAIST), Japan Yoichi Ishibashi, Katsuhito Sudoh, Koichiro Yoshino, Satoshi Nakamura Reflection-based Word Attribute Transfer Background Analogy Analogy in the embedding space is a operation that

857 views • 23 slides
AFS at Intel AFS at Intel Travis Broughton Travis Broughton Agenda Agenda Intels

AFS at Intel AFS at Intel Travis Broughton Travis Broughton Agenda Agenda Intels

AFS at Intel AFS at Intel Travis Broughton Travis Broughton Agenda Agenda Intels Engineering Environment Intels Engineering Environment Things AFS Does well Things AFS Does well How Intel uses AFS How Intel uses AFS How not to use

393 views • 17 slides
NAT64/DNS64 real life experiments and one useful tool NAT64Check ...from go6lab.si and

NAT64/DNS64 real life experiments and one useful tool NAT64Check ...from go6lab.si and

NAT64/DNS64 real life experiments and one useful tool NAT64Check ...from go6lab.si and IPv6-lab.net Brought to you by: Jan or Sander Steffann zorz@isoc.org sander@steffann.nl Problem statement and real world status (Six Degrees of

560 views • 42 slides

More recommend