Design Lessons From Binary Fission A Crowd Sourced Game for - - PowerPoint PPT Presentation

▶
design lessons from binary fission
SMART_READER_LITE
LIVE PREVIEW

Design Lessons From Binary Fission A Crowd Sourced Game for - - PowerPoint PPT Presentation

Jan 21, 2023 38 likes •322 views

Design Lessons From Binary Fission A Crowd Sourced Game for Precondition Discovery Kate Compton, Heather Logas, Joseph C. Osborn, Chandranil Chakrabortti, Kelsey Coffman, Daniel Fava, Dylan Lederle-Ensign, Zhongpeng Lin, Jo Mazeika, Afshin

slide-1
SLIDE 1

Design Lessons From Binary Fission

A Crowd Sourced Game for Precondition Discovery

Augmented Design Laboratory // Center for Games and Playable Media / http://games.soe.ucsc.edu/adl // http://games.soe.ucsc.edu

Kate Compton, Heather Logas, Joseph C. Osborn, Chandranil Chakrabortti, Kelsey Coffman, Daniel Fava, Dylan Lederle-Ensign, Zhongpeng Lin, Jo Mazeika, Afshin Mobramaein, Johnathan Pagnutti, Husacar Sanchez, Jim Whitehead, Brenda Laurel University of California, Santa Cruz John Murray SRI International 


slide-2
SLIDE 2
  • Designing for citizen science
  • Formal program verification
  • Turning players into computers (but happy ones!)
  • Helping the computer help the player
  • Players supporting players
  • Results and conclusions

This material is based upon work supported by the United States Air Force Research Lab- oratory (AFRL) and the Defense Advanced Research Projects Agency under Contract No. FA8750-12-C-0225. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of AFRL or DARPA.

Design Lessons From Binary Fission

slide-3
SLIDE 3

Designing for citizen science

Khatib, Firas, Seth Cooper, Michael D Tyka, Kefan Xu, Ilya Makedon, Zoran Popović, David Baker, and Foldit Players. 2011. “Algorithm discovery by protein folding game players.” Proceedings of the National Academy of Sciences 108 (47): 18949–18953.

Design Lessons From Binary Fission

  • Players are good at being better

“computers” than computers

  • Better pattern-induction, better 3D

spatial mapping, better “intuition”

slide-4
SLIDE 4

Designing for citizen science

Reed, Jason, et al. "An exploratory factor analysis of motivations for participating in Zooniverse, a collection of virtual citizen science projects." System Sciences (HICSS), 2013 46th Hawaii International Conference on. IEEE, 2013.

Design Lessons From Binary Fission

  • Players are good at being better

“computers” than computers

  • Better pattern-induction, better 3D

spatial mapping, better “intuition”

slide-5
SLIDE 5

Designing for citizen science

  • Players are good at being better

“computers” than computers

  • Better pattern-induction, better 3D

spatial mapping, better “intuition”

Design Lessons From Binary Fission

Iacovides, Ioanna, et al. "Do games attract or sustain engagement in citizen science?: a study of volunteer motivations." CHI'13 Extended Abstracts on Human Factors in Computing Systems. ACM, 2013.

slide-6
SLIDE 6

Designing for citizen science

  • DARPA Verigames Project
  • Crowd Sourced Formal Verification
  • Formal program verification is slow and expensive
  • …but it’s very useful! (the promise of secures and bug-free programs)
  • Can we repeat the success of Foldit, Zooniverse and EyeWire?
  • Dean, Drew, Sean Gaurino, Leonard Eusebi, Andrew Keplinger, Tim Pavlik, Ronald Watro,

Aaron Cammarata, et al. 2015. “Lessons Learned in Game Development for Crowd- sourced Software Formal Verification.” In 2015 USENIX Summit on Gaming, Games, and Gamification in Security Education (3GSE 15). Washington, D.C.: USENIX Association, August.

Design Lessons From Binary Fission

slide-7
SLIDE 7

Formal program verification

  • Is x always even?
  • Will t = y(init) at the end of the

loop?

  • Is x always less than y? Is z?
  • What about for different starting

values of x, y and z?

  • What can we say about this loop?

Design Lessons From Binary Fission

var x = 0; var y = 20; var z = 1; var t = 0; while (x < y) { z += x%(y%6 + 1); x += z%3; y—; t++; }

slide-8
SLIDE 8

Formal program verification

  • Fava, Daniel, et al. "Gamifying program

analysis." Logic for Programming, Artificial Intelligence, and Reasoning. Springer Berlin Heidelberg, 2015.

  • Fava, Daniel, et al. "Crowdsourcing program

preconditions via a classification game." Proceedings of the 38th International Conference on Software Engineering. ACM, 2016.

  • Require function invariants, so lets get some

from the players

Design Lessons From Binary Fission

var x = 0; var y = 20; var z = 1; var t = 0; while (x < y) { z += x%(y%6 + 1); x += z%3; y—; t++; }

slide-9
SLIDE 9

The Verigames project

Design Lessons From Binary Fission

http://www.verigames.com/

slide-10
SLIDE 10

Design Lessons From Binary Fission

The Verigames project (Xylem)

slide-11
SLIDE 11

Design Lessons From Binary Fission

Don’t make your player feel like a bad computer, Make them feel like an awesome cyborg!

The Verigames project (Xylem)

slide-12
SLIDE 12

Basic design of Binary Fission

Design Lessons From Binary Fission

  • Hiding the data, hide the

math

  • The “Dragonbox”

technique

  • Dealing with many sets of

data at a time

  • (generating sample sets of data to test

complexity early on!)

slide-13
SLIDE 13

Basic design of Binary Fission

Design Lessons From Binary Fission

Brief Demo!

slide-14
SLIDE 14

Basic design of Binary Fission

Design Lessons From Binary Fission

  • There are data that were generated

by the function (“true” data)

  • and data that weren’t (“false” data)
  • We represent these in blue and

gold “quarks”

  • Each “filter” splits the quarks in a

different way

  • Can you build a tree of filters to split

the “good” data from the “bad” data, effectively?

slide-15
SLIDE 15

Optimizing for the right results

Design Lessons From Binary Fission

  • Scoring: optimize for

sorting well, but efficiently

  • In”wild” data, not every

problem may be perfectly sortable!

  • (avoid “100%” UI, this game may be

frustrating to completionists)

slide-16
SLIDE 16

Turning players into computers

  • Don’t waste human-time doing

what algorithms are good at

  • If we can brute-force a solution, do that!

(unless you have a grant that says you need to use crowd-sourcing)

  • What aren’t algorithms good at?
  • “intuiting” patterns?
  • from what data?

Design Lessons From Binary Fission

Hypothesis: give players the “right” data, at the right rate, and they will come up with intuitive understandings of underlying patterns (magic intuition)

slide-17
SLIDE 17

Turning players into computers

  • What we can do with algorithms
  • Make potential “basic” filters
  • x > y
  • x*19 - 4 < y%10
  • Calculate their efficiency at sorting any

given set of data

  • What we can’t do*
  • build an optimal tree

Design Lessons From Binary Fission

slide-18
SLIDE 18

Helping the computer help the player

Design Lessons From Binary Fission

  • Given filters, how good are they for

the current set of quarks?

  • Shopping in “filter space”
  • Players as the decider
slide-19
SLIDE 19

Helping the computer help the player

Design Lessons From Binary Fission

  • Automate more information!
  • What *can* we calculate? What *can* we show?

(magic intuition)

slide-20
SLIDE 20

Helping the computer help the player

  • * …. so is this actually better than brute-force or AI?
  • Can AI build a better tree with an AI than human

users can?

  • FANTASTIC.
  • Give them a “robot arm” autosolver, and let them

figure out when to override it.

  • Maybe it’ll be useful for more complex problems

that can make use of human+bot power

Design Lessons From Binary Fission

awesome cyborg!

slide-21
SLIDE 21

Helping the computer help the player

  • ….by helping humans help

each other

  • Build a system where helping
  • thers is easy, obvious, and

supported with automation?

Design Lessons From Binary Fission

slide-22
SLIDE 22

Players supporting players

  • Players like helping other players
  • Players like seeing that other players are playing

(empty restaurant syndrome)

  • Players like competing with each other
  • Sometimes competing to be the most helpful!
  • Scale matters.
  • Feedback loops happen.

Design Lessons From Binary Fission

slide-23
SLIDE 23

Results and conclusions

  • The crowd managed to find

some novel, legible and useful program invariants.

  • Social features never took off

(we were an empty restaurant)

Design Lessons From Binary Fission

slide-24
SLIDE 24

Results and conclusions

  • Was there enough

information to create the “magic intuition”?

  • Power users wanted more

views of the math, more tools

Design Lessons From Binary Fission

slide-25
SLIDE 25

Results and conclusions

  • Was there enough

information to create the “magic intuition”?

  • Power users wanted more

views of the math, more tools

Design Lessons From Binary Fission

slide-26
SLIDE 26

Results and conclusions

  • Did we have the right theme?
  • What is the role of story and

theme in science games?

  • if it gives you an actionable

“hypothesis” (SimCity effect*)

  • if it gives you the right

audience for your gameplay

Design Lessons From Binary Fission

*Noah Wardrip-Fruin, Expressive Processing

slide-27
SLIDE 27

In conclusion

Automate Augment (awesome cyborg)

Iterate!!!

slide-28
SLIDE 28

Questions?

  • Kate Compton, kcompton@soe.ucsc.edu
  • Jim Whitehead, ejw@cs.ucsc.edu
  • John Murray, SRI jxm@sri.com

Design Lessons From Binary Fission

In conclusion

Automate Augment (awesome cyborg)

Iterate!!!