Some advice from a reproducible researcher about how some advice - - PowerPoint PPT Presentation

▶

some advice from a reproducible researcher about how some

Some advice from a reproducible researcher about how some advice - - PowerPoint PPT Presentation

Feb 23, 2023 378 likes •840 views

Some advice from a reproducible researcher about how some advice from research data repositories to irreproducible researchers about reproducibility and repositories might help researchers, repositories, and reproducibility Thomas J. Leeper

slide-1

SLIDE 1

Some advice from a reproducible researcher about how some advice from research data repositories to irreproducible researchers about reproducibility and repositories might help researchers, repositories, and reproducibility

Thomas J. Leeper

Department of Government London School of Economics and Political Science

16 June 2017

slide-2

SLIDE 2

Why reproducibility?

Journal requirements Funding agency requirements Institutional requirements The coming revolution

slide-3

SLIDE 3

Why reproducibility?

Journal requirements Funding agency requirements Institutional requirements The coming revolution How can we shift thinking from extrinsic motivations to intrinsic motivations?

slide-4

SLIDE 4

Getting intrinsic

slide-5

SLIDE 5

What makes up the ideal reproducible research product?

slide-6

SLIDE 6

What makes up the ideal reproducible research product?

Nobody seems to agree!

slide-7

SLIDE 7

slide-8

SLIDE 8

Confession: This is my PhD dissertation.

slide-9

SLIDE 9

What makes up the ideal reproducible research product?

Gandrud’s template rOpenSci’s “Research Compendium” Project TIER

slide-10

SLIDE 10

Root Rep-Res-ExampleProject1 Paper.Rnw Slideshow.Rnw Website.Rnw Main.bib Data MainData.csv Makefile MergeData.R Gather1.R MainData_VariableDescriptions.md README.Rmd Analysis GoogleVisMap.R ScatterUDSFert.R README.md

slide-11

SLIDE 11

project |- DESCRIPTION # project metadata and dependencies |- README.md # top-level description of content | |- data/ # raw data, not changed once created | +- my_data.csv # data files in open formats | |- analysis/ # any programmatic code | +- my_scripts.R # R code used to analyse data

slide-12

SLIDE 12

slide-13

SLIDE 13

What makes up the ideal reproducible research product?

Gandrud’s template rOpenSci’s “Research Compendium” Project TIER

slide-14

SLIDE 14

What makes up the ideal reproducible research product?

Gandrud’s template rOpenSci’s “Research Compendium” Project TIER Docker container?

slide-15

SLIDE 15

What makes up the ideal reproducible research product?

Gandrud’s template rOpenSci’s “Research Compendium” Project TIER Docker container? Virtual machine?

slide-16

SLIDE 16

What makes up the ideal reproducible research product?

Gandrud’s template rOpenSci’s “Research Compendium” Project TIER Docker container? Virtual machine? ???

slide-17

SLIDE 17

What makes up the ideal reproducible research product?

Gandrud’s template rOpenSci’s “Research Compendium” Project TIER Docker container? Virtual machine? ??? More is probably better for reproducibility, but declining marginal returns for the researcher.

slide-18

SLIDE 18

What makes up the ideal reproducible research product?

Big disagreements

What exactly is being reproduced? What is being assumed about software, hardware, data formats, etc.? What tools are best? packages? make? docker?

slide-19

SLIDE 19

What makes up the ideal reproducible research product?

Big disagreements

What exactly is being reproduced? What is being assumed about software, hardware, data formats, etc.? What tools are best? packages? make? docker?

Consensus is not possible!

slide-20

SLIDE 20

slide-21

SLIDE 21

What makes up the ideal reproducible research product?

Big disagreements

What exactly is being reproduced? What is being assumed about software, hardware, data formats, etc.? What tools are best? packages? make? docker?

Consensus is not possible!

slide-22

SLIDE 22

What makes up the ideal reproducible research product?

Big disagreements

What exactly is being reproduced? What is being assumed about software, hardware, data formats, etc.? What tools are best? packages? make? docker?

Consensus is not possible! Nudge instead.

slide-23

SLIDE 23

What makes up the ideal reproducible research product?

Big disagreements

What exactly is being reproduced? What is being assumed about software, hardware, data formats, etc.? What tools are best? packages? make? docker?

Consensus is not possible! Nudge instead.

Provide templates to use when starting a project

slide-24

SLIDE 24

What makes up the ideal reproducible research product?

Big disagreements

What exactly is being reproduced? What is being assumed about software, hardware, data formats, etc.? What tools are best? packages? make? docker?

Consensus is not possible! Nudge instead.

Provide templates to use when starting a project Provide exemplars to show how to conceptualize the organization of a project

slide-25

SLIDE 25

slide-26

SLIDE 26

slide-27

SLIDE 27

slide-28

SLIDE 28

The Advice to Researchers

1 Reproducibility isn’t just one more burden 2 It’s about helping your (future) yourself first 3 Be reproducible for science second

slide-29

SLIDE 29

The Advice to Researchers

1 Reproducibility isn’t just one more burden 2 It’s about helping your (future) yourself first 3 Be reproducible for science second

slide-30

SLIDE 30

slide-31

SLIDE 31

slide-32

SLIDE 32

slide-33

SLIDE 33

Irreproducibility

slide-34

SLIDE 34

Irreproducibility

Fabrication

slide-35

SLIDE 35

Irreproducibility

Fabrication Human error

slide-36

SLIDE 36

Irreproducibility

Fabrication Human error Lack of methodological transparency

slide-37

SLIDE 37

Irreproducibility

Fabrication Human error Lack of methodological transparency Proprietary data and file formats

slide-38

SLIDE 38

Irreproducibility

Fabrication Human error Lack of methodological transparency Proprietary data and file formats Unavailable data

slide-39

SLIDE 39

Irreproducibility

Fabrication Human error Lack of methodological transparency Proprietary data and file formats Unavailable data Analysis uses proprietary software/hardware

slide-40

SLIDE 40

Irreproducibility

Fabrication Human error Lack of methodological transparency Proprietary data and file formats Unavailable data Analysis uses proprietary software/hardware Analysis unavailable

slide-41

SLIDE 41

Irreproducibility

Fabrication Human error Lack of methodological transparency Proprietary data and file formats Unavailable data Analysis uses proprietary software/hardware Analysis unavailable “Available from the author”

slide-42

SLIDE 42

Irreproducibility

Fabrication Human error Lack of methodological transparency Proprietary data and file formats Unavailable data Analysis uses proprietary software/hardware Analysis unavailable “Available from the author”

slide-43

SLIDE 43

Irreproducibility

Fabrication Human error Lack of methodological transparency Proprietary data and file formats Unavailable data Analysis uses proprietary software/hardware Analysis unavailable “Available from the author (now deceased)”

slide-44

SLIDE 44