a universal identifier for computational results

A Universal Identifier for Computational Results Matan Gavish and - PowerPoint PPT Presentation

A Universal Identifier for Computational Results Matan Gavish and David Donoho Statistics Department, Stanford University gavish@stanford.edu July 14, 2011 (gavish@stanford.edu) VCR July 14, 2011 1 / 46 35:00 (gavish@stanford.edu) VCR


  1. RR efforts since 1990: Re-execute computation Not { realistic, easy to adopt, real benefits } Ignores the scientific tradition of chronicling experiments Ignores the hard truth: for acadmics, publication is the endgame The all-importnat publication is informal description of work we do in private Reproducibility involves cleaning up post-facto. It’s a pretentious waste of time . (–DLD, 2010) Alternative approach: Verifiable Computational Results Automatically record, validate run-time steps leading to publishable results (gavish@stanford.edu) VCR July 14, 2011 13 / 46

  2. RR efforts since 1990: Re-execute computation Not { realistic, easy to adopt, real benefits } Ignores the scientific tradition of chronicling experiments Ignores the hard truth: for acadmics, publication is the endgame The all-importnat publication is informal description of work we do in private Reproducibility involves cleaning up post-facto. It’s a pretentious waste of time . (–DLD, 2010) Alternative approach: Verifiable Computational Results Automatically record, validate run-time steps leading to publishable results (gavish@stanford.edu) VCR July 14, 2011 13 / 46

  3. RR efforts since 1990: Re-execute computation Not { realistic, easy to adopt, real benefits } Ignores the scientific tradition of chronicling experiments Ignores the hard truth: for acadmics, publication is the endgame The all-importnat publication is informal description of work we do in private Reproducibility involves cleaning up post-facto. It’s a pretentious waste of time . (–DLD, 2010) Alternative approach: Verifiable Computational Results Automatically record, validate run-time steps leading to publishable results (gavish@stanford.edu) VCR July 14, 2011 13 / 46

  4. RR efforts since 1990: Re-execute computation Not { realistic, easy to adopt, real benefits } Ignores the scientific tradition of chronicling experiments Ignores the hard truth: for acadmics, publication is the endgame The all-importnat publication is informal description of work we do in private Reproducibility involves cleaning up post-facto. It’s a pretentious waste of time . (–DLD, 2010) Alternative approach: Verifiable Computational Results Automatically record, validate run-time steps leading to publishable results (gavish@stanford.edu) VCR July 14, 2011 13 / 46

  5. RR efforts since 1990: Re-execute computation Not { realistic, easy to adopt, real benefits } Ignores the scientific tradition of chronicling experiments Ignores the hard truth: for acadmics, publication is the endgame The all-importnat publication is informal description of work we do in private Reproducibility involves cleaning up post-facto. It’s a pretentious waste of time . (–DLD, 2010) Alternative approach: Verifiable Computational Results Automatically record, validate run-time steps leading to publishable results (gavish@stanford.edu) VCR July 14, 2011 13 / 46

  6. RR efforts since 1990: Re-execute computation Not { realistic, easy to adopt, real benefits } Ignores the scientific tradition of chronicling experiments Ignores the hard truth: for acadmics, publication is the endgame The all-importnat publication is informal description of work we do in private Reproducibility involves cleaning up post-facto. It’s a pretentious waste of time . (–DLD, 2010) Alternative approach: Verifiable Computational Results Automatically record, validate run-time steps leading to publishable results (gavish@stanford.edu) VCR July 14, 2011 13 / 46

  7. (gavish@stanford.edu) VCR July 14, 2011 14 / 46

  8. (gavish@stanford.edu) VCR July 14, 2011 14 / 46

  9. (gavish@stanford.edu) VCR July 14, 2011 14 / 46

  10. (gavish@stanford.edu) VCR July 14, 2011 14 / 46

  11. (gavish@stanford.edu) VCR July 14, 2011 14 / 46

  12. (gavish@stanford.edu) VCR July 14, 2011 14 / 46

  13. (gavish@stanford.edu) VCR July 14, 2011 14 / 46

  14. VCR record of a computational experiment Includes – Code Original data (standard format) Parameters used in run-time Publishable & intermediate results (standard format) Dependencies VCR records are permanent and citable. Peers reference it in articles, computer programs, databases. (gavish@stanford.edu) VCR July 14, 2011 15 / 46

  15. VCR record of a computational experiment Includes – Code Original data (standard format) Parameters used in run-time Publishable & intermediate results (standard format) Dependencies VCR records are permanent and citable. Peers reference it in articles, computer programs, databases. (gavish@stanford.edu) VCR July 14, 2011 15 / 46

  16. VCR record of a computational experiment Includes – Code Original data (standard format) Parameters used in run-time Publishable & intermediate results (standard format) Dependencies VCR records are permanent and citable. Peers reference it in articles, computer programs, databases. (gavish@stanford.edu) VCR July 14, 2011 15 / 46

  17. VCR record of a computational experiment Includes – Code Original data (standard format) Parameters used in run-time Publishable & intermediate results (standard format) Dependencies VCR records are permanent and citable. Peers reference it in articles, computer programs, databases. (gavish@stanford.edu) VCR July 14, 2011 15 / 46

  18. VCR record of a computational experiment Includes – Code Original data (standard format) Parameters used in run-time Publishable & intermediate results (standard format) Dependencies VCR records are permanent and citable. Peers reference it in articles, computer programs, databases. (gavish@stanford.edu) VCR July 14, 2011 15 / 46

  19. VCR record of a computational experiment Includes – Code Original data (standard format) Parameters used in run-time Publishable & intermediate results (standard format) Dependencies VCR records are permanent and citable. Peers reference it in articles, computer programs, databases. (gavish@stanford.edu) VCR July 14, 2011 15 / 46

  20. VCR record of a computational experiment Includes – Code Original data (standard format) Parameters used in run-time Publishable & intermediate results (standard format) Dependencies VCR records are permanent and citable. Peers reference it in articles, computer programs, databases. (gavish@stanford.edu) VCR July 14, 2011 15 / 46

  21. VCR Foundation Record and transmit, in run-time, computation steps leading to results Automatically validate and store on publisher repository server Universal identifier connects it to article figure Anyone can develop applications that mine knowledge in chronicles (gavish@stanford.edu) VCR July 14, 2011 16 / 46

  22. VCR Foundation Record and transmit, in run-time, computation steps leading to results Automatically validate and store on publisher repository server Universal identifier connects it to article figure Anyone can develop applications that mine knowledge in chronicles (gavish@stanford.edu) VCR July 14, 2011 16 / 46

  23. VCR Foundation Record and transmit, in run-time, computation steps leading to results Automatically validate and store on publisher repository server Universal identifier connects it to article figure Anyone can develop applications that mine knowledge in chronicles (gavish@stanford.edu) VCR July 14, 2011 16 / 46

  24. VCR Foundation Record and transmit, in run-time, computation steps leading to results Automatically validate and store on publisher repository server Universal identifier connects it to article figure Anyone can develop applications that mine knowledge in chronicles (gavish@stanford.edu) VCR July 14, 2011 16 / 46

  25. VCR Foundation Record and transmit, in run-time, computation steps leading to results Automatically validate and store on publisher repository server Universal identifier connects it to article figure Anyone can develop applications that mine knowledge in chronicles (gavish@stanford.edu) VCR July 14, 2011 16 / 46

  26. Outline 1 The VCR Foundation and API 2 Chronicing computations 3 Basic VCR apps (gavish@stanford.edu) VCR July 14, 2011 17 / 46

  27. Outline 1 The VCR Foundation and API 2 Chronicing computations 3 Basic VCR apps (gavish@stanford.edu) VCR July 14, 2011 17 / 46

  28. Outline 1 The VCR Foundation and API 2 Chronicing computations 3 Basic VCR apps (gavish@stanford.edu) VCR July 14, 2011 17 / 46

  29. Outline 1 The VCR Foundation and API 2 Chronicing computations 3 Basic VCR apps (gavish@stanford.edu) VCR July 14, 2011 17 / 46

  30. VCR Foundation system (e.g vcr-stat.stanford.edu) VCR repository server Verifiable Result Identifier (VRI) Addressing standard: URL, secure digital signature and QR barcode vcr.cell.com/ffaaffb148d7 VCR Application Programmer Interface (API) RESTful interface for depositing, searching and retrieving content on repositories GET https://vcr.cell.com/ffaaffb148d7/graphics?format=eps (gavish@stanford.edu) VCR July 14, 2011 18 / 46

  31. VCR Foundation system (e.g vcr-stat.stanford.edu) VCR repository server Verifiable Result Identifier (VRI) Addressing standard: URL, secure digital signature and QR barcode vcr.cell.com/ffaaffb148d7 VCR Application Programmer Interface (API) RESTful interface for depositing, searching and retrieving content on repositories GET https://vcr.cell.com/ffaaffb148d7/graphics?format=eps (gavish@stanford.edu) VCR July 14, 2011 18 / 46

  32. VCR Foundation system (e.g vcr-stat.stanford.edu) VCR repository server Verifiable Result Identifier (VRI) Addressing standard: URL, secure digital signature and QR barcode vcr.cell.com/ffaaffb148d7 VCR Application Programmer Interface (API) RESTful interface for depositing, searching and retrieving content on repositories GET https://vcr.cell.com/ffaaffb148d7/graphics?format=eps (gavish@stanford.edu) VCR July 14, 2011 18 / 46

  33. VCR Foundation system (e.g vcr-stat.stanford.edu) VCR repository server Verifiable Result Identifier (VRI) Addressing standard: URL, secure digital signature and QR barcode vcr.cell.com/ffaaffb148d7 VCR Application Programmer Interface (API) RESTful interface for depositing, searching and retrieving content on repositories GET https://vcr.cell.com/ffaaffb148d7/graphics?format=eps (gavish@stanford.edu) VCR July 14, 2011 18 / 46

  34. Outline 1 The VCR Foundation and API 2 Chronicing computations 3 Basic VCR apps (gavish@stanford.edu) VCR July 14, 2011 19 / 46

  35. Outline 1 The VCR Foundation and API 2 Chronicing computations 3 Basic VCR apps (gavish@stanford.edu) VCR July 14, 2011 19 / 46

  36. Chronicing computations in real-time VCR computation platform Plugin = Computation recorder Regular program code figure1 = plot(x) save(figure1,’figure1.eps’) > file /home/figure1.eps saved > (gavish@stanford.edu) VCR July 14, 2011 20 / 46

  37. Chronicing computations in real-time VCR computation platform Plugin = Computation recorder Regular program code figure1 = plot(x) save(figure1,’figure1.eps’) > file /home/figure1.eps saved > (gavish@stanford.edu) VCR July 14, 2011 20 / 46

  38. Chronicing computations in real-time VCR computation platform Plugin = Computation recorder Regular program code figure1 = plot(x) save(figure1,’figure1.eps’) > file /home/figure1.eps saved > (gavish@stanford.edu) VCR July 14, 2011 20 / 46

  39. Chronicing computations in real-time VCR computation platform Plugin = Computation recorder Regular program code figure1 = plot(x) save(figure1,’figure1.eps’) > file /home/figure1.eps saved > (gavish@stanford.edu) VCR July 14, 2011 20 / 46

  40. Chronicing computations in real-time VCR computation platform Plugin = Computation recorder Regular program code figure1 = plot(x) save(figure1,’figure1.eps’) > file /home/figure1.eps saved > (gavish@stanford.edu) VCR July 14, 2011 20 / 46

  41. Chronicing computations in real-time VCR computation platform Plugin = Computation recorder Program code with VCR plugin repository vcr.nature.com verifiable figure1 = plot(x) > vcr.nature.com approved: > access figure1 at https://vcr.nature.com/ffaaffb148d7 (gavish@stanford.edu) VCR July 14, 2011 20 / 46

  42. Chronicing computations in real-time VCR computation platform Plugin = Computation recorder Program code with VCR plugin repository vcr.nature.com verifiable figure1 = plot(x) > vcr.nature.com approved: > access figure1 at https://vcr.nature.com/ffaaffb148d7 (gavish@stanford.edu) VCR July 14, 2011 20 / 46

  43. matlab plugin (gavish@stanford.edu) VCR July 14, 2011 21 / 46

  44. (gavish@stanford.edu) VCR July 14, 2011 22 / 46

  45. (gavish@stanford.edu) VCR July 14, 2011 22 / 46

  46. (gavish@stanford.edu) VCR July 14, 2011 22 / 46

  47. Converting computation result into content in real-time Data/code published at run-time, no intervention possible or required Researcher controls granularity, can choose“private VRI” Repository automatically validates, signs computation result content (gavish@stanford.edu) VCR July 14, 2011 23 / 46

  48. Converting computation result into content in real-time Data/code published at run-time, no intervention possible or required Researcher controls granularity, can choose“private VRI” Repository automatically validates, signs computation result content (gavish@stanford.edu) VCR July 14, 2011 23 / 46

  49. Converting computation result into content in real-time Data/code published at run-time, no intervention possible or required Researcher controls granularity, can choose“private VRI” Repository automatically validates, signs computation result content (gavish@stanford.edu) VCR July 14, 2011 23 / 46

  50. Converting computation result into content in real-time Data/code published at run-time, no intervention possible or required Researcher controls granularity, can choose“private VRI” Repository automatically validates, signs computation result content (gavish@stanford.edu) VCR July 14, 2011 23 / 46

  51. VCR foundation software (gavish@stanford.edu) VCR July 14, 2011 24 / 46

  52. VCR foundation software (gavish@stanford.edu) VCR July 14, 2011 24 / 46

  53. VCR foundation software (gavish@stanford.edu) VCR July 14, 2011 24 / 46

  54. Outline 1 The VCR Foundation and API 2 Chronicing computations 3 Basic VCR apps (gavish@stanford.edu) VCR July 14, 2011 25 / 46

  55. Outline 1 The VCR Foundation and API 2 Chronicing computations 3 Basic VCR apps (gavish@stanford.edu) VCR July 14, 2011 25 / 46

  56. Basic VCR apps Direct data import Word-processor plugin Content browser Search (gavish@stanford.edu) VCR July 14, 2011 26 / 46

  57. Direct data import App (gavish@stanford.edu) VCR July 14, 2011 27 / 46

  58. Direct data import App (gavish@stanford.edu) VCR July 14, 2011 27 / 46

  59. Direct data import App (gavish@stanford.edu) VCR July 14, 2011 27 / 46

  60. Direct data import App (gavish@stanford.edu) VCR July 14, 2011 27 / 46

  61. Direct data import App Program Code x = load(’genome.csv’) run_experiment(x) Program Code with VCR plugin run_experiment([vcr.cell.com/ffaaffb148d7]) A natural evolution of scholarly citation (gavish@stanford.edu) VCR July 14, 2011 28 / 46

  62. Direct data import App Program Code x = load(’genome.csv’) run_experiment(x) Program Code with VCR plugin run_experiment([vcr.cell.com/ffaaffb148d7]) A natural evolution of scholarly citation (gavish@stanford.edu) VCR July 14, 2011 28 / 46

  63. Direct data import App Program Code x = load(’genome.csv’) run_experiment(x) Program Code with VCR plugin run_experiment([vcr.cell.com/ffaaffb148d7]) A natural evolution of scholarly citation (gavish@stanford.edu) VCR July 14, 2011 28 / 46

  64. Direct data import App Program Code x = load(’genome.csv’) run_experiment(x) Program Code with VCR plugin run_experiment([vcr.cell.com/ffaaffb148d7]) A natural evolution of scholarly citation (gavish@stanford.edu) VCR July 14, 2011 28 / 46

Recommend


More recommend