a universal identifier for computational results
play

A Universal Identifier for Computational Results Matan Gavish and - PowerPoint PPT Presentation

A Universal Identifier for Computational Results Matan Gavish and David Donoho Statistics Department, Stanford University gavish@stanford.edu July 14, 2011 (gavish@stanford.edu) VCR July 14, 2011 1 / 46 35:00 (gavish@stanford.edu) VCR


  1. RR efforts since 1990: Re-execute computation Not { realistic, easy to adopt, real benefits } Ignores the scientific tradition of chronicling experiments Ignores the hard truth: for acadmics, publication is the endgame The all-importnat publication is informal description of work we do in private Reproducibility involves cleaning up post-facto. It’s a pretentious waste of time . (–DLD, 2010) Alternative approach: Verifiable Computational Results Automatically record, validate run-time steps leading to publishable results (gavish@stanford.edu) VCR July 14, 2011 13 / 46

  2. RR efforts since 1990: Re-execute computation Not { realistic, easy to adopt, real benefits } Ignores the scientific tradition of chronicling experiments Ignores the hard truth: for acadmics, publication is the endgame The all-importnat publication is informal description of work we do in private Reproducibility involves cleaning up post-facto. It’s a pretentious waste of time . (–DLD, 2010) Alternative approach: Verifiable Computational Results Automatically record, validate run-time steps leading to publishable results (gavish@stanford.edu) VCR July 14, 2011 13 / 46

  3. RR efforts since 1990: Re-execute computation Not { realistic, easy to adopt, real benefits } Ignores the scientific tradition of chronicling experiments Ignores the hard truth: for acadmics, publication is the endgame The all-importnat publication is informal description of work we do in private Reproducibility involves cleaning up post-facto. It’s a pretentious waste of time . (–DLD, 2010) Alternative approach: Verifiable Computational Results Automatically record, validate run-time steps leading to publishable results (gavish@stanford.edu) VCR July 14, 2011 13 / 46

  4. RR efforts since 1990: Re-execute computation Not { realistic, easy to adopt, real benefits } Ignores the scientific tradition of chronicling experiments Ignores the hard truth: for acadmics, publication is the endgame The all-importnat publication is informal description of work we do in private Reproducibility involves cleaning up post-facto. It’s a pretentious waste of time . (–DLD, 2010) Alternative approach: Verifiable Computational Results Automatically record, validate run-time steps leading to publishable results (gavish@stanford.edu) VCR July 14, 2011 13 / 46

  5. RR efforts since 1990: Re-execute computation Not { realistic, easy to adopt, real benefits } Ignores the scientific tradition of chronicling experiments Ignores the hard truth: for acadmics, publication is the endgame The all-importnat publication is informal description of work we do in private Reproducibility involves cleaning up post-facto. It’s a pretentious waste of time . (–DLD, 2010) Alternative approach: Verifiable Computational Results Automatically record, validate run-time steps leading to publishable results (gavish@stanford.edu) VCR July 14, 2011 13 / 46

  6. RR efforts since 1990: Re-execute computation Not { realistic, easy to adopt, real benefits } Ignores the scientific tradition of chronicling experiments Ignores the hard truth: for acadmics, publication is the endgame The all-importnat publication is informal description of work we do in private Reproducibility involves cleaning up post-facto. It’s a pretentious waste of time . (–DLD, 2010) Alternative approach: Verifiable Computational Results Automatically record, validate run-time steps leading to publishable results (gavish@stanford.edu) VCR July 14, 2011 13 / 46

  7. (gavish@stanford.edu) VCR July 14, 2011 14 / 46

  8. (gavish@stanford.edu) VCR July 14, 2011 14 / 46

  9. (gavish@stanford.edu) VCR July 14, 2011 14 / 46

  10. (gavish@stanford.edu) VCR July 14, 2011 14 / 46

  11. (gavish@stanford.edu) VCR July 14, 2011 14 / 46

  12. (gavish@stanford.edu) VCR July 14, 2011 14 / 46

  13. (gavish@stanford.edu) VCR July 14, 2011 14 / 46

  14. VCR record of a computational experiment Includes – Code Original data (standard format) Parameters used in run-time Publishable & intermediate results (standard format) Dependencies VCR records are permanent and citable. Peers reference it in articles, computer programs, databases. (gavish@stanford.edu) VCR July 14, 2011 15 / 46

  15. VCR record of a computational experiment Includes – Code Original data (standard format) Parameters used in run-time Publishable & intermediate results (standard format) Dependencies VCR records are permanent and citable. Peers reference it in articles, computer programs, databases. (gavish@stanford.edu) VCR July 14, 2011 15 / 46

  16. VCR record of a computational experiment Includes – Code Original data (standard format) Parameters used in run-time Publishable & intermediate results (standard format) Dependencies VCR records are permanent and citable. Peers reference it in articles, computer programs, databases. (gavish@stanford.edu) VCR July 14, 2011 15 / 46

  17. VCR record of a computational experiment Includes – Code Original data (standard format) Parameters used in run-time Publishable & intermediate results (standard format) Dependencies VCR records are permanent and citable. Peers reference it in articles, computer programs, databases. (gavish@stanford.edu) VCR July 14, 2011 15 / 46

  18. VCR record of a computational experiment Includes – Code Original data (standard format) Parameters used in run-time Publishable & intermediate results (standard format) Dependencies VCR records are permanent and citable. Peers reference it in articles, computer programs, databases. (gavish@stanford.edu) VCR July 14, 2011 15 / 46

  19. VCR record of a computational experiment Includes – Code Original data (standard format) Parameters used in run-time Publishable & intermediate results (standard format) Dependencies VCR records are permanent and citable. Peers reference it in articles, computer programs, databases. (gavish@stanford.edu) VCR July 14, 2011 15 / 46

  20. VCR record of a computational experiment Includes – Code Original data (standard format) Parameters used in run-time Publishable & intermediate results (standard format) Dependencies VCR records are permanent and citable. Peers reference it in articles, computer programs, databases. (gavish@stanford.edu) VCR July 14, 2011 15 / 46

  21. VCR Foundation Record and transmit, in run-time, computation steps leading to results Automatically validate and store on publisher repository server Universal identifier connects it to article figure Anyone can develop applications that mine knowledge in chronicles (gavish@stanford.edu) VCR July 14, 2011 16 / 46

  22. VCR Foundation Record and transmit, in run-time, computation steps leading to results Automatically validate and store on publisher repository server Universal identifier connects it to article figure Anyone can develop applications that mine knowledge in chronicles (gavish@stanford.edu) VCR July 14, 2011 16 / 46

  23. VCR Foundation Record and transmit, in run-time, computation steps leading to results Automatically validate and store on publisher repository server Universal identifier connects it to article figure Anyone can develop applications that mine knowledge in chronicles (gavish@stanford.edu) VCR July 14, 2011 16 / 46

  24. VCR Foundation Record and transmit, in run-time, computation steps leading to results Automatically validate and store on publisher repository server Universal identifier connects it to article figure Anyone can develop applications that mine knowledge in chronicles (gavish@stanford.edu) VCR July 14, 2011 16 / 46

  25. VCR Foundation Record and transmit, in run-time, computation steps leading to results Automatically validate and store on publisher repository server Universal identifier connects it to article figure Anyone can develop applications that mine knowledge in chronicles (gavish@stanford.edu) VCR July 14, 2011 16 / 46

  26. Outline 1 The VCR Foundation and API 2 Chronicing computations 3 Basic VCR apps (gavish@stanford.edu) VCR July 14, 2011 17 / 46

  27. Outline 1 The VCR Foundation and API 2 Chronicing computations 3 Basic VCR apps (gavish@stanford.edu) VCR July 14, 2011 17 / 46

  28. Outline 1 The VCR Foundation and API 2 Chronicing computations 3 Basic VCR apps (gavish@stanford.edu) VCR July 14, 2011 17 / 46

  29. Outline 1 The VCR Foundation and API 2 Chronicing computations 3 Basic VCR apps (gavish@stanford.edu) VCR July 14, 2011 17 / 46

  30. VCR Foundation system (e.g vcr-stat.stanford.edu) VCR repository server Verifiable Result Identifier (VRI) Addressing standard: URL, secure digital signature and QR barcode vcr.cell.com/ffaaffb148d7 VCR Application Programmer Interface (API) RESTful interface for depositing, searching and retrieving content on repositories GET https://vcr.cell.com/ffaaffb148d7/graphics?format=eps (gavish@stanford.edu) VCR July 14, 2011 18 / 46

  31. VCR Foundation system (e.g vcr-stat.stanford.edu) VCR repository server Verifiable Result Identifier (VRI) Addressing standard: URL, secure digital signature and QR barcode vcr.cell.com/ffaaffb148d7 VCR Application Programmer Interface (API) RESTful interface for depositing, searching and retrieving content on repositories GET https://vcr.cell.com/ffaaffb148d7/graphics?format=eps (gavish@stanford.edu) VCR July 14, 2011 18 / 46

  32. VCR Foundation system (e.g vcr-stat.stanford.edu) VCR repository server Verifiable Result Identifier (VRI) Addressing standard: URL, secure digital signature and QR barcode vcr.cell.com/ffaaffb148d7 VCR Application Programmer Interface (API) RESTful interface for depositing, searching and retrieving content on repositories GET https://vcr.cell.com/ffaaffb148d7/graphics?format=eps (gavish@stanford.edu) VCR July 14, 2011 18 / 46

  33. VCR Foundation system (e.g vcr-stat.stanford.edu) VCR repository server Verifiable Result Identifier (VRI) Addressing standard: URL, secure digital signature and QR barcode vcr.cell.com/ffaaffb148d7 VCR Application Programmer Interface (API) RESTful interface for depositing, searching and retrieving content on repositories GET https://vcr.cell.com/ffaaffb148d7/graphics?format=eps (gavish@stanford.edu) VCR July 14, 2011 18 / 46

  34. Outline 1 The VCR Foundation and API 2 Chronicing computations 3 Basic VCR apps (gavish@stanford.edu) VCR July 14, 2011 19 / 46

  35. Outline 1 The VCR Foundation and API 2 Chronicing computations 3 Basic VCR apps (gavish@stanford.edu) VCR July 14, 2011 19 / 46

  36. Chronicing computations in real-time VCR computation platform Plugin = Computation recorder Regular program code figure1 = plot(x) save(figure1,’figure1.eps’) > file /home/figure1.eps saved > (gavish@stanford.edu) VCR July 14, 2011 20 / 46

  37. Chronicing computations in real-time VCR computation platform Plugin = Computation recorder Regular program code figure1 = plot(x) save(figure1,’figure1.eps’) > file /home/figure1.eps saved > (gavish@stanford.edu) VCR July 14, 2011 20 / 46

  38. Chronicing computations in real-time VCR computation platform Plugin = Computation recorder Regular program code figure1 = plot(x) save(figure1,’figure1.eps’) > file /home/figure1.eps saved > (gavish@stanford.edu) VCR July 14, 2011 20 / 46

  39. Chronicing computations in real-time VCR computation platform Plugin = Computation recorder Regular program code figure1 = plot(x) save(figure1,’figure1.eps’) > file /home/figure1.eps saved > (gavish@stanford.edu) VCR July 14, 2011 20 / 46

  40. Chronicing computations in real-time VCR computation platform Plugin = Computation recorder Regular program code figure1 = plot(x) save(figure1,’figure1.eps’) > file /home/figure1.eps saved > (gavish@stanford.edu) VCR July 14, 2011 20 / 46

  41. Chronicing computations in real-time VCR computation platform Plugin = Computation recorder Program code with VCR plugin repository vcr.nature.com verifiable figure1 = plot(x) > vcr.nature.com approved: > access figure1 at https://vcr.nature.com/ffaaffb148d7 (gavish@stanford.edu) VCR July 14, 2011 20 / 46

  42. Chronicing computations in real-time VCR computation platform Plugin = Computation recorder Program code with VCR plugin repository vcr.nature.com verifiable figure1 = plot(x) > vcr.nature.com approved: > access figure1 at https://vcr.nature.com/ffaaffb148d7 (gavish@stanford.edu) VCR July 14, 2011 20 / 46

  43. matlab plugin (gavish@stanford.edu) VCR July 14, 2011 21 / 46

  44. (gavish@stanford.edu) VCR July 14, 2011 22 / 46

  45. (gavish@stanford.edu) VCR July 14, 2011 22 / 46

  46. (gavish@stanford.edu) VCR July 14, 2011 22 / 46

  47. Converting computation result into content in real-time Data/code published at run-time, no intervention possible or required Researcher controls granularity, can choose“private VRI” Repository automatically validates, signs computation result content (gavish@stanford.edu) VCR July 14, 2011 23 / 46

  48. Converting computation result into content in real-time Data/code published at run-time, no intervention possible or required Researcher controls granularity, can choose“private VRI” Repository automatically validates, signs computation result content (gavish@stanford.edu) VCR July 14, 2011 23 / 46

  49. Converting computation result into content in real-time Data/code published at run-time, no intervention possible or required Researcher controls granularity, can choose“private VRI” Repository automatically validates, signs computation result content (gavish@stanford.edu) VCR July 14, 2011 23 / 46

  50. Converting computation result into content in real-time Data/code published at run-time, no intervention possible or required Researcher controls granularity, can choose“private VRI” Repository automatically validates, signs computation result content (gavish@stanford.edu) VCR July 14, 2011 23 / 46

  51. VCR foundation software (gavish@stanford.edu) VCR July 14, 2011 24 / 46

  52. VCR foundation software (gavish@stanford.edu) VCR July 14, 2011 24 / 46

  53. VCR foundation software (gavish@stanford.edu) VCR July 14, 2011 24 / 46

  54. Outline 1 The VCR Foundation and API 2 Chronicing computations 3 Basic VCR apps (gavish@stanford.edu) VCR July 14, 2011 25 / 46

  55. Outline 1 The VCR Foundation and API 2 Chronicing computations 3 Basic VCR apps (gavish@stanford.edu) VCR July 14, 2011 25 / 46

  56. Basic VCR apps Direct data import Word-processor plugin Content browser Search (gavish@stanford.edu) VCR July 14, 2011 26 / 46

  57. Direct data import App (gavish@stanford.edu) VCR July 14, 2011 27 / 46

  58. Direct data import App (gavish@stanford.edu) VCR July 14, 2011 27 / 46

  59. Direct data import App (gavish@stanford.edu) VCR July 14, 2011 27 / 46

  60. Direct data import App (gavish@stanford.edu) VCR July 14, 2011 27 / 46

  61. Direct data import App Program Code x = load(’genome.csv’) run_experiment(x) Program Code with VCR plugin run_experiment([vcr.cell.com/ffaaffb148d7]) A natural evolution of scholarly citation (gavish@stanford.edu) VCR July 14, 2011 28 / 46

  62. Direct data import App Program Code x = load(’genome.csv’) run_experiment(x) Program Code with VCR plugin run_experiment([vcr.cell.com/ffaaffb148d7]) A natural evolution of scholarly citation (gavish@stanford.edu) VCR July 14, 2011 28 / 46

  63. Direct data import App Program Code x = load(’genome.csv’) run_experiment(x) Program Code with VCR plugin run_experiment([vcr.cell.com/ffaaffb148d7]) A natural evolution of scholarly citation (gavish@stanford.edu) VCR July 14, 2011 28 / 46

  64. Direct data import App Program Code x = load(’genome.csv’) run_experiment(x) Program Code with VCR plugin run_experiment([vcr.cell.com/ffaaffb148d7]) A natural evolution of scholarly citation (gavish@stanford.edu) VCR July 14, 2011 28 / 46

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend


More recommend