Open notebooks in cognitive neuroscience Jrn Alexander Quent MRC - - PowerPoint PPT Presentation

MRC Cognition and Brain Sciences Unit

Open notebooks in cognitive neuroscience

Jörn Alexander Quent MRC Cognition and Brain Sciences Unit

20 November 2018

MRC Cognition and Brain Sciences Unit

Keeping notes of your work

MRC Cognition and Brain Sciences Unit

Keeping notes of your work

Why should we keep notes?

• Documentation.
• It will help you with writing.
• Your most important collaborator is your future self!

What should keep notes of?

• Your Ideas.
• Reasons for parameters.
• Rational for analysis.
• Anything needed for replication.

MRC Cognition and Brain Sciences Unit

Keeping notes of your work

How should you keep notes?

• Write them for someone else.
• Organise the files and data:
• Use appropriate file/variable names.
• Write readme files.
• Link notes to projects/files.
• Order chronologically and use timestamps.
• Think about permanent data storage and back ups.
• Use electronic notebook e.g. Jupyter Notebook or

RStudio/Rmarkdown to accompany your code.

MRC Cognition and Brain Sciences Unit

What are open notebooks?

• Open notebook science (ONS) is about making the entirety of your

research public.

• The term has been coined by Jean-Claude

Bradley (Chemistry) in 2006.

• A lot of scientist (esp. in wet labs) already keep

a paper version. This is just an open version.

MRC Cognition and Brain Sciences Unit

My experience so far

MRC Cognition and Brain Sciences Unit

My new workflow

Version controlled GitHub repos to work in Created with blogdown and GitHub pages

Notebook entries Project 1: Experiment Project 2: R package Project 3: Simulation Personal website Pre- registration

MRC Cognition and Brain Sciences Unit

My new workflow

MRC Cognition and Brain Sciences Unit

My new workflow

MRC Cognition and Brain Sciences Unit

My new workflow

MRC Cognition and Brain Sciences Unit

Benefits for our field and you

For our field:

• Improve reproducibility:
• Ensure that the whole process can be scrutinized.
• Gold standard is ability to continue without you.
• End of file drawer problem.
• Useful for long-term comprehensibility.

For yourself:

• Convince the skeptics of your work.
• Let other people catch your mistakes before publication.
• Make collaboration and mutual learning easier.
• Point reviewers to notebook.
• Employability (even outside academia).

MRC Cognition and Brain Sciences Unit

What are possible downsides?

• Time investment.
• Fear of scoops:
• ONS can help you to protect your work.
• You decide when to go public!
• Fear of rejection by editors:
• Same rules for pre-prints.
• Fear of reputation or misinterpretation of your work.
• Lack of recognition of your work.
• People will write you how helpful something is.
• Research integrity still requires people to cite you.
• Choose a license (e.g. CC-BY-SA).

MRC Cognition and Brain Sciences Unit

Keeping open notes of your work

• Use version control for all files including data:
• This can serve as your back up.
• Be careful not to disclose any Personal Identifiers (PID).
• Content is up to you and your discipline.

MRC Cognition and Brain Sciences Unit

Closing remarks

• The open is not the problem in ONS.
• ONS can be the gold standard in our field with maximal

reproducibility and accountability.

• ONS should not be another source of stress and pressure!
• ONS should not replace peer-review but complement it.

MRC Cognition and Brain Sciences Unit

Resources

Blogdown: https://bookdown.org/yihui/blogdown/ Introduction to Git/GitHub: https://psyarxiv.com/6tzh8/ Notebooks Chris H.J. Hartgerink (http://onsnetwork.org/chartgerink/) John Griffiths (https://johngriffiths.github.io/LabNotebook/) Open notebook science network (http://onsnetwork.org/ and @ONScience)

MRC Cognition and Brain Sciences Unit

What do you think?