Crowdsourced Conformance Testing via Remote Sandboxing Joe Gibbs - - PowerPoint PPT Presentation

crowdsourced conformance testing via remote sandboxing
SMART_READER_LITE
LIVE PREVIEW

Crowdsourced Conformance Testing via Remote Sandboxing Joe Gibbs - - PowerPoint PPT Presentation

Crowdsourced Conformance Testing via Remote Sandboxing Joe Gibbs Politz http://www.batchgeo.com Remote Sandboxing? Students' interpreters g r a d e . r k t 1 6 0 0 t i m e s ? Students' interpreters g r a d e


slide-1
SLIDE 1

Crowdsourced Conformance Testing via Remote Sandboxing

Joe Gibbs Politz

slide-2
SLIDE 2
slide-3
SLIDE 3

http://www.batchgeo.com

slide-4
SLIDE 4

“Remote Sandboxing?”

slide-5
SLIDE 5

1 6 t i m e s ?

g r a d e . r k t

Students' interpreters

slide-6
SLIDE 6

Students' interpreters

g r a d e . r k t

slide-7
SLIDE 7

# l a n g r a c k e t / b a s e ( r e q u i r e ' i n t e r p . r k t ' ) ( c

  • m

m a n d

  • l

i n e “

  • t

e s t ” t e s t

  • d

i r ( r u n

  • t

e s t s i n t e r p t e s t

  • d

i r ) )

slide-8
SLIDE 8

# l a n g r a c k e t / b a s e ( r e q u i r e ' i n t e r p . r k t ' ) ( c

  • m

m a n d

  • l

i n e “

  • t

e s t ” t e s t

  • d

i r ( r u n

  • t

e s t s i n t e r p t e s t

  • d

i r ) “

  • r

e p

  • r

t ” ( s i g n ( r u n

  • t

e s t s i n t e r p

  • u

r

  • t

e s t s ) “ s

  • p

e r

  • s

e e k r e t ” ) )

slide-9
SLIDE 9

# l a n g r a c k e t / b a s e ( d e f i n e i n t e r p ( d y n a m i c

  • r

e q u i r e “ i n t e r p . r k t ” ' i n t e r p ) ) ( c

  • m

m a n d

  • l

i n e “

  • t

e s t ” t e s t

  • d

i r ( r u n

  • t

e s t s i n t e r p t e s t

  • d

i r ) “

  • r

e p

  • r

t ” ( s i g n ( r u n

  • t

e s t s i n t e r p

  • u

r

  • t

e s t s ) “ s

  • p

e r

  • s

e e k r e t ” ) )

slide-10
SLIDE 10

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

Students' interpreters

g r a d e . b i n

Remote Sandboxing

slide-11
SLIDE 11

“Crowdsouced Conformance Testing”?

slide-12
SLIDE 12

correct-interp for-update-after-test plus-coerces-strings arity-before-evaluate

. . .

  • tests/
  • loop-tests/
  • for-test1.psl
  • for-test2.psl
  • operator-tests/
  • subtract-test1.psl
  • subtract-test2.psl
  • add-test1.psl
  • function-tests/
  • apply1.psl
  • apply2.psl
  • closure1.psl
  • closure2.psl

discard-if-effects

✓ ✓ ✓

ParselTongue Testing Assignment

slide-13
SLIDE 13
slide-14
SLIDE 14

> 10,000 tests from 250 people That's as many tests as JavaScript's conformance suite.

Crowdsourced Conformance Testing

slide-15
SLIDE 15

Lessons

  • Racket's dynamic code support is awesome,

and Racket's build tools for distribution just work

  • Value proposition for teaching PL online – get

crowdsourced tools and tests