SLIDE 1
Crowdsourced Conformance Testing via Remote Sandboxing
Joe Gibbs Politz
Crowdsourced Conformance Testing via Remote Sandboxing Joe Gibbs - - PowerPoint PPT Presentation
Crowdsourced Conformance Testing via Remote Sandboxing Joe Gibbs - - PowerPoint PPT Presentation
Crowdsourced Conformance Testing via Remote Sandboxing Joe Gibbs Politz http://www.batchgeo.com Remote Sandboxing? Students' interpreters g r a d e . r k t 1 6 0 0 t i m e s ? Students' interpreters g r a d e
SLIDE 2
SLIDE 3
http://www.batchgeo.com
SLIDE 4
“Remote Sandboxing?”
SLIDE 5
1 6 t i m e s ?
g r a d e . r k t
Students' interpreters
SLIDE 6
Students' interpreters
g r a d e . r k t
SLIDE 7
# l a n g r a c k e t / b a s e ( r e q u i r e ' i n t e r p . r k t ' ) ( c
- m
m a n d
- l
i n e “
- t
e s t ” t e s t
- d
i r ( r u n
- t
e s t s i n t e r p t e s t
- d
i r ) )
SLIDE 8
# l a n g r a c k e t / b a s e ( r e q u i r e ' i n t e r p . r k t ' ) ( c
- m
m a n d
- l
i n e “
- t
e s t ” t e s t
- d
i r ( r u n
- t
e s t s i n t e r p t e s t
- d
i r ) “
- r
e p
- r
t ” ( s i g n ( r u n
- t
e s t s i n t e r p
- u
r
- t
e s t s ) “ s
- p
e r
- s
e e k r e t ” ) )
SLIDE 9
# l a n g r a c k e t / b a s e ( d e f i n e i n t e r p ( d y n a m i c
- r
e q u i r e “ i n t e r p . r k t ” ' i n t e r p ) ) ( c
- m
m a n d
- l
i n e “
- t
e s t ” t e s t
- d
i r ( r u n
- t
e s t s i n t e r p t e s t
- d
i r ) “
- r
e p
- r
t ” ( s i g n ( r u n
- t
e s t s i n t e r p
- u
r
- t
e s t s ) “ s
- p
e r
- s
e e k r e t ” ) )
SLIDE 10
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
Students' interpreters
g r a d e . b i n
Remote Sandboxing
SLIDE 11
“Crowdsouced Conformance Testing”?
SLIDE 12
correct-interp for-update-after-test plus-coerces-strings arity-before-evaluate
. . .
- tests/
- loop-tests/
- for-test1.psl
- for-test2.psl
- operator-tests/
- subtract-test1.psl
- subtract-test2.psl
- add-test1.psl
- function-tests/
- apply1.psl
- apply2.psl
- closure1.psl
- closure2.psl
discard-if-effects
✓ ✓ ✓
ParselTongue Testing Assignment
SLIDE 13
SLIDE 14
> 10,000 tests from 250 people That's as many tests as JavaScript's conformance suite.
Crowdsourced Conformance Testing
SLIDE 15
Lessons
- Racket's dynamic code support is awesome,
and Racket's build tools for distribution just work
- Value proposition for teaching PL online – get
crowdsourced tools and tests