running probabilistic running probabilistic running
play

Running Probabilistic Running Probabilistic Running Probabilistic - PowerPoint PPT Presentation

Running Probabilistic Running Probabilistic Running Probabilistic Programs Backwards Programs Backwards Programs Backwards Neil Toronto * Jay McCarthy David Van Horn * * University of Maryland Vassar College ESOP 2015 2015/04/14


  1. Simpler Example Simpler Example Simpler Example • Assume (random) returns a value uniformly in for value of (max 0.5 (random)) : Density function 1.25 1.25 1.25 1.25 1.25 1.25 1.25 1.25 1.25 p ₘ (x) = ??? p ₘ (x) = ??? p ₘ (x) = ??? p ₘ (x) = ??? p ₘ (x) = ??? p ₘ (x) = ??? p ₘ (x) = ??? p ₘ (x) = ??? p ₘ (x) = ??? 1 1 1 1 1 1 1 1 1 .75 .75 .75 .75 .75 .75 .75 .75 .75 p ₘ (x) p ₘ (x) p ₘ (x) p ₘ (x) p ₘ (x) p ₘ (x) p ₘ (x) p ₘ (x) p ₘ (x) .5 .5 .5 .5 .5 .5 .5 .5 .5 .25 .25 .25 .25 .25 .25 .25 .25 .25 0 0 0 0 0 0 0 0 0 0 0 0 .25 .25 .25 .5 .5 .5 .75 .75 .75 1 1 1 1.25 1.25 1.25 0 0 0 .25 .25 .25 .5 .5 .5 .75 .75 .75 1 1 1 1.25 1.25 1.25 0 0 0 .25 .25 .25 .5 .5 .5 .75 .75 .75 1 1 1 1.25 1.25 1.25 x x x x x x x x x 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3

  2. Simpler Example Simpler Example Simpler Example • Assume (random) returns a value uniformly in for value of (max 0.5 (random)) : Density function 1.25 1.25 1.25 1.25 1.25 1.25 1.25 1.25 1.25 p ₘ (x) = ??? p ₘ (x) = ??? p ₘ (x) = ??? p ₘ (x) = ??? p ₘ (x) = ??? p ₘ (x) = ??? p ₘ (x) = ??? p ₘ (x) = ??? p ₘ (x) = ??? 1 1 1 1 1 1 1 1 1 .75 .75 .75 .75 .75 .75 .75 .75 .75 p ₘ (x) p ₘ (x) p ₘ (x) p ₘ (x) p ₘ (x) p ₘ (x) p ₘ (x) p ₘ (x) p ₘ (x) .5 .5 .5 .5 .5 .5 .5 .5 .5 .25 .25 .25 .25 .25 .25 .25 .25 .25 0 0 0 0 0 0 0 0 0 0 0 0 .25 .25 .25 .5 .5 .5 .75 .75 .75 1 1 1 1.25 1.25 1.25 0 0 0 .25 .25 .25 .5 .5 .5 .75 .75 .75 1 1 1 1.25 1.25 1.25 0 0 0 .25 .25 .25 .5 .5 .5 .75 .75 .75 1 1 1 1.25 1.25 1.25 x x x x x x x x x 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3

  3. Simpler Example Simpler Example Simpler Example • Assume (random) returns a value uniformly in for value of (max 0.5 (random)) : Density function 1.25 1.25 1.25 1.25 1.25 1.25 1.25 1.25 1.25 p ₘ (x) = ??? p ₘ (x) = ??? p ₘ (x) = ??? p ₘ (x) = ??? p ₘ (x) = ??? p ₘ (x) = ??? p ₘ (x) = ??? p ₘ (x) = ??? p ₘ (x) = ??? 1 1 1 1 1 1 1 1 1 .75 .75 .75 .75 .75 .75 .75 .75 .75 p ₘ (x) p ₘ (x) p ₘ (x) p ₘ (x) p ₘ (x) p ₘ (x) p ₘ (x) p ₘ (x) p ₘ (x) .5 .5 .5 .5 .5 .5 .5 .5 .5 .25 .25 .25 .25 .25 .25 .25 .25 .25 0 0 0 0 0 0 0 0 0 0 0 0 .25 .25 .25 .5 .5 .5 .75 .75 .75 1 1 1 1.25 1.25 1.25 0 0 0 .25 .25 .25 .5 .5 .5 .75 .75 .75 1 1 1 1.25 1.25 1.25 0 0 0 .25 .25 .25 .5 .5 .5 .75 .75 .75 1 1 1 1.25 1.25 1.25 x x x x x x x x x 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3

  4. Simpler Example Simpler Example Simpler Example • Assume (random) returns a value uniformly in for value of (max 0.5 (random)) : Density function 1.25 1.25 1.25 1.25 1.25 1.25 1.25 1.25 1.25 p ₘ (x) doesn't exist p ₘ (x) doesn't exist p ₘ (x) doesn't exist p ₘ (x) doesn't exist p ₘ (x) doesn't exist p ₘ (x) doesn't exist p ₘ (x) doesn't exist p ₘ (x) doesn't exist p ₘ (x) doesn't exist 1 1 1 1 1 1 1 1 1 .75 .75 .75 .75 .75 .75 .75 .75 .75 p ₘ (x) p ₘ (x) p ₘ (x) p ₘ (x) p ₘ (x) p ₘ (x) p ₘ (x) p ₘ (x) p ₘ (x) .5 .5 .5 .5 .5 .5 .5 .5 .5 .25 .25 .25 .25 .25 .25 .25 .25 .25 0 0 0 0 0 0 0 0 0 0 0 0 .25 .25 .25 .5 .5 .5 .75 .75 .75 1 1 1 1.25 1.25 1.25 0 0 0 .25 .25 .25 .5 .5 .5 .75 .75 .75 1 1 1 1.25 1.25 1.25 0 0 0 .25 .25 .25 .5 .5 .5 .75 .75 .75 1 1 1 1.25 1.25 1.25 x x x x x x x x x 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3

  5. What Can’t Densities Model? What Can’t Densities Model? What Can’t Densities Model? 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4

  6. What Can’t Densities Model? What Can’t Densities Model? What Can’t Densities Model? • Results of discontinuous functions (bounded measuring devices) (let ([temperature (normal 99 1)]) (min 100 temperature)) 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4

  7. What Can’t Densities Model? What Can’t Densities Model? What Can’t Densities Model? • Results of discontinuous functions (bounded measuring devices) (let ([temperature (normal 99 1)]) (min 100 temperature)) • Variable-dimensional things (union types) (if test? none (just x)) 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4

  8. What Can’t Densities Model? What Can’t Densities Model? What Can’t Densities Model? • Results of discontinuous functions (bounded measuring devices) (let ([temperature (normal 99 1)]) (min 100 temperature)) • Variable-dimensional things (union types) (if test? none (just x)) • In fi nite-dimensional things (recursion) 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4

  9. What Can’t Densities Model? What Can’t Densities Model? What Can’t Densities Model? • Results of discontinuous functions (bounded measuring devices) (let ([temperature (normal 99 1)]) (min 100 temperature)) • Variable-dimensional things (union types) (if test? none (just x)) • In fi nite-dimensional things (recursion) • In general: the distributions of program values 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4

  10. Probability Measures Probability Measures Probability Measures • Like already-integrated densities, but a primitive concept 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5

  11. Probability Measures Probability Measures Probability Measures • Like already-integrated densities, but a primitive concept • Measure of (random) is , de fi ned by 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5

  12. Probability Measures Probability Measures Probability Measures • Like already-integrated densities, but a primitive concept • Measure of (random) is , de fi ned by 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5

  13. Probability Measures Probability Measures Probability Measures • Like already-integrated densities, but a primitive concept • Measure of (random) is , de fi ned by • Measure of (max 0.5 (random)) de fi ned by 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5

  14. Probability Measures Probability Measures Probability Measures • Like already-integrated densities, but a primitive concept • Measure of (random) is , de fi ned by • Measure of (max 0.5 (random)) de fi ned by This term assigns probability 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5

  15. Probability Measures Probability Measures Probability Measures • Like already-integrated densities, but a primitive concept • Measure of (random) is , de fi ned by • Measure of (max 0.5 (random)) de fi ned by This term assigns probability • Need a way to derive measures from code 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5

  16. Probability Measures Via Preimages Probability Measures Via Preimages Probability Measures Via Preimages • Interpret (max 0.5 (random)) as , de fi ned 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6

  17. Probability Measures Via Preimages Probability Measures Via Preimages Probability Measures Via Preimages • Interpret (max 0.5 (random)) as , de fi ned • Derive measure of (max 0.5 (random)) as 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6

  18. Probability Measures Via Preimages Probability Measures Via Preimages Probability Measures Via Preimages • Interpret (max 0.5 (random)) as , de fi ned • Derive measure of (max 0.5 (random)) as where 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6

  19. Probability Measures Via Preimages Probability Measures Via Preimages Probability Measures Via Preimages • Interpret (max 0.5 (random)) as , de fi ned • Derive measure of (max 0.5 (random)) as where • Factored into random and deterministic parts: 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6

  20. Probability Measures Via Preimages Probability Measures Via Preimages Probability Measures Via Preimages • Interpret (max 0.5 (random)) as , de fi ned • Derive measure of (max 0.5 (random)) as where • Factored into random and deterministic parts: • In other words, compute measures of expressions by running them backwards 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6

  21. Crazy Idea is Feasible If... Crazy Idea is Feasible If... Crazy Idea is Feasible If... • Seems like we need: 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7

  22. Crazy Idea is Feasible If... Crazy Idea is Feasible If... Crazy Idea is Feasible If... • Seems like we need: Standard interpretation of programs as pure functions from a random source 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7

  23. Crazy Idea is Feasible If... Crazy Idea is Feasible If... Crazy Idea is Feasible If... • Seems like we need: Standard interpretation of programs as pure functions from a random source E ffi cient way to compute preimage sets 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7

  24. Crazy Idea is Feasible If... Crazy Idea is Feasible If... Crazy Idea is Feasible If... • Seems like we need: Standard interpretation of programs as pure functions from a random source E ffi cient way to compute preimage sets E ffi cient representation of arbitrary sets 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7

  25. Crazy Idea is Feasible If... Crazy Idea is Feasible If... Crazy Idea is Feasible If... • Seems like we need: Standard interpretation of programs as pure functions from a random source E ffi cient way to compute preimage sets E ffi cient representation of arbitrary sets E ffi cient way to compute areas of preimage sets 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7

  26. Crazy Idea is Feasible If... Crazy Idea is Feasible If... Crazy Idea is Feasible If... • Seems like we need: Standard interpretation of programs as pure functions from a random source E ffi cient way to compute preimage sets E ffi cient representation of arbitrary sets E ffi cient way to compute areas of preimage sets Proof of correctness w.r.t. standard interpretation 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7

  27. Crazy Idea is Feasible If... Crazy Idea is Feasible If... Crazy Idea is Feasible If... • Seems like we need: Standard interpretation of programs as pure functions from a random source E ffi cient way to compute preimage sets E ffi cient representation of arbitrary sets E ffi cient way to compute areas of preimage sets Proof of correctness w.r.t. standard interpretation • WAT 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7

  28. What About Approximating? What About Approximating? What About Approximating? Conservative approximation with rectangles: 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8

  29. What About Approximating? What About Approximating? What About Approximating? Conservative approximation with rectangles: 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8

  30. What About Approximating? What About Approximating? What About Approximating? Restricting preimages to rectangular subdomains: 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9

  31. What About Approximating? What About Approximating? What About Approximating? Restricting preimages to rectangular subdomains: 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9

  32. What About Approximating? What About Approximating? What About Approximating? Restricting preimages to rectangular subdomains: 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9

  33. What About Approximating? What About Approximating? What About Approximating? Restricting preimages to rectangular subdomains: 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9

  34. What About Approximating? What About Approximating? What About Approximating? Restricting preimages to rectangular subdomains: 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9

  35. What About Approximating? What About Approximating? What About Approximating? Restricting preimages to rectangular subdomains: 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9

  36. What About Approximating? What About Approximating? What About Approximating? Restricting preimages to rectangular subdomains: 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9

  37. What About Approximating? What About Approximating? What About Approximating? Restricting preimages to rectangular subdomains: 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9

  38. What About Approximating? What About Approximating? What About Approximating? Restricting preimages to rectangular subdomains: 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9

  39. What About Approximating? What About Approximating? What About Approximating? Restricting preimages to rectangular subdomains: 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9

  40. What About Approximating? What About Approximating? What About Approximating? Restricting preimages to rectangular subdomains: 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9

  41. What About Approximating? What About Approximating? What About Approximating? Restricting preimages to rectangular subdomains: 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9

  42. What About Approximating? What About Approximating? What About Approximating? Sampling: exponential to quadratic (e.g. days to minutes) 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10

  43. What About Approximating? What About Approximating? What About Approximating? Sampling: exponential to quadratic (e.g. days to minutes) 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10

  44. Contribution: Making This Crazy Idea Feasible Contribution: Making This Crazy Idea Feasible Contribution: Making This Crazy Idea Feasible • Standard interpretation of programs as pure functions from a random source • E ffi cient way to compute preimage sets • E ffi cient representation of arbitrary sets • E ffi cient way to compute volumes of preimage sets • Proof of correctness w.r.t. standard interpretation 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11

  45. Contribution: Making This Crazy Idea Feasible Contribution: Making This Crazy Idea Feasible Contribution: Making This Crazy Idea Feasible • Standard interpretation of programs as pure functions from a random source • E ffi cient way to compute abstract preimage subsets • E ffi cient representation of arbitrary sets • E ffi cient way to compute volumes of preimage sets • Proof of correctness w.r.t. standard interpretation 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11

  46. Contribution: Making This Crazy Idea Feasible Contribution: Making This Crazy Idea Feasible Contribution: Making This Crazy Idea Feasible • Standard interpretation of programs as pure functions from a random source • E ffi cient way to compute abstract preimage subsets • E ffi cient representation of abstract sets • E ffi cient way to compute volumes of preimage sets • Proof of correctness w.r.t. standard interpretation 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11

  47. Contribution: Making This Crazy Idea Feasible Contribution: Making This Crazy Idea Feasible Contribution: Making This Crazy Idea Feasible • Standard interpretation of programs as pure functions from a random source • E ffi cient way to compute abstract preimage subsets • E ffi cient representation of abstract sets • E ffi cient way to sample uniformly in preimage sets • Proof of correctness w.r.t. standard interpretation 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11

  48. Contribution: Making This Crazy Idea Feasible Contribution: Making This Crazy Idea Feasible Contribution: Making This Crazy Idea Feasible • Standard interpretation of programs as pure functions from a random source • E ffi cient way to compute abstract preimage subsets • E ffi cient representation of abstract sets • E ffi cient way to sample uniformly in preimage sets E ffi cient domain partition sampling • Proof of correctness w.r.t. standard interpretation 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11

  49. Contribution: Making This Crazy Idea Feasible Contribution: Making This Crazy Idea Feasible Contribution: Making This Crazy Idea Feasible • Standard interpretation of programs as pure functions from a random source • E ffi cient way to compute abstract preimage subsets • E ffi cient representation of abstract sets • E ffi cient way to sample uniformly in preimage sets E ffi cient domain partition sampling E ffi cient way to determine whether a domain sample is actually in the preimage (just use standard interpretation) • Proof of correctness w.r.t. standard interpretation 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11

  50. How Many Meanings? How Many Meanings? How Many Meanings? • Start with pure programs, then lift by threading a random store 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11

  51. How Many Meanings? How Many Meanings? How Many Meanings? • Start with pure programs, then lift by threading a random store • Nonrecursive, nonprobabilistic programs: , , 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11

  52. How Many Meanings? How Many Meanings? How Many Meanings? • Start with pure programs, then lift by threading a random store • Nonrecursive, nonprobabilistic programs: , , • Add 3 semantic functions for recursion and probabilistic choice 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11

  53. How Many Meanings? How Many Meanings? How Many Meanings? • Start with pure programs, then lift by threading a random store • Nonrecursive, nonprobabilistic programs: , , • Add 3 semantic functions for recursion and probabilistic choice • Full development needs 2 more to transfer theorems from measure theory... 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11

  54. How Many Meanings? How Many Meanings? How Many Meanings? • Start with pure programs, then lift by threading a random store • Nonrecursive, nonprobabilistic programs: , , • Add 3 semantic functions for recursion and probabilistic choice • Full development needs 2 more to transfer theorems from measure theory... • ... oh, and 1 more to collect information for Monte Carlo integration 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11

  55. How Many Meanings? How Many Meanings? How Many Meanings? • Start with pure programs, then lift by threading a random store • Nonrecursive, nonprobabilistic programs: , , • Add 3 semantic functions for recursion and probabilistic choice • Full development needs 2 more to transfer theorems from measure theory... • ... oh, and 1 more to collect information for Monte Carlo integration Tally: 3+3+2+1 = 9 semantic functions, 11 or 12 rules each 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11

  56. Enter Category Theory Enter Category Theory Enter Category Theory • Moggi (1989): Introduces monads for interpreting e ff ects 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11

  57. Enter Category Theory Enter Category Theory Enter Category Theory • Moggi (1989): Introduces monads for interpreting e ff ects • Other kinds of categories: idioms, arrows 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11

  58. Enter Category Theory Enter Category Theory Enter Category Theory • Moggi (1989): Introduces monads for interpreting e ff ects • Other kinds of categories: idioms, arrows • Arrow de fi ned by type constructor and these combinators: 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11

  59. Enter Category Theory Enter Category Theory Enter Category Theory • Moggi (1989): Introduces monads for interpreting e ff ects • Other kinds of categories: idioms, arrows • Arrow de fi ned by type constructor and these combinators: • Arrows are always function-like 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11

  60. Reducing Complexity Reducing Complexity Reducing Complexity Function arrow: is just 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11

  61. Reducing Complexity Reducing Complexity Reducing Complexity Function arrow: is just 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11

  62. Reducing Complexity Reducing Complexity Reducing Complexity Function arrow: is just 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11

  63. Reducing Complexity Reducing Complexity Reducing Complexity Function arrow: is just 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11

  64. Reducing Complexity Reducing Complexity Reducing Complexity Function arrow: is just 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11

  65. Reducing Complexity Reducing Complexity Reducing Complexity Function arrow: is just 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11

  66. Reducing Complexity Reducing Complexity Reducing Complexity Function arrow: is just 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11

  67. Reducing Complexity Reducing Complexity Reducing Complexity Function arrow: is just 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11

  68. Reducing Complexity Reducing Complexity Reducing Complexity Function arrow: is just 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend


More recommend