What's in it for me? What's in it for me? Transparently organizing - PowerPoint PPT Presentation

What's in it for me? What's in it for me? Transparently organizing your research from start to fi nish Candice C. Morey MRC-CBU Open Science Day

Adopting open research practices · Scientists endorse openness, but most don't prioritize it · Perceived to be a lot of work · Rewards fuzzy: - Idealistic only? - Maybe important later, but not vital now? 2/30

Strategic concerns 3/30

Strategic concerns · Won't it slow me down? 4/30

Strategic concerns · Why should someone else bene fi t from my work? 5/30

Strategic concerns · What if I fail? 6/30

Science is a collaborative e ff ort You are working together to achieve something bigger. 7/30

Why PIs need in-lab transparency 8/30

Why PIs need in-lab transparency 9/30

Why PIs need in-lab transparency Each student thinks their project is theirs, but it is always part of something bigger. 10/30

Why PIs need in-lab transparency · To ensure consistency in training researchers · To know the provenance and protect the security of data · To stem information loss that comes with turn-over · To make it easy to share publicly when its time 11/30

Components of my open-lab work fl ow · Lab handbook · Open Science Framework · Scripted analysis · Pre-registration 12/30

1. My lab handbook https://ccmorey.github.io/labHandbook/ 13/30

Why have a lab handbook? · End theory-of-mind games when training new students · Articulate a standard to aspire to - So students know what I would consider "professional" (i.e., worthy of high marks) 14/30

Why make it public? · To get feedback · To be helpful · So you might in fl uence standards · Enrich descriptions of your published method 15/30

2. The Open Science Framework · OSF: A tool you can use to organize, back-up, and eventually share fi nished components · Sharing your fi nished products != working totally in public - Custom software, stimuli, protocols: they're fi nished when study is ready to be run - Pre-registrations: Before analyzing new data - Anonymized data: when the data set is closed to new members, entries - Analysis code and papers that are ready for scrutiny from someone 16/30

Nice features of OSF · It's free · Control project members, visibility · Can get DOI assigned to project · Good tool for supervision - can see quickly where project stands 17/30

How I use it · Students are introduced to OSF at start · They are added to an existing project or create their own · Finished products (materials, data, writing) are uploaded · They "register" project at important milestones - Creates a back-up - Provides a timestamp in case we need it · The OSF page supplements whatever we publish 18/30

3. Scripted Analyses · Not part of my training! · Work fl ow I learned in my training was not easy to reproduce 19/30

Principles of scripting · Processing data shouldn't lead to information loss · Script starts from anonymized raw data - Every time · Everything done to the data is recorded in the script - Is documented, can be changed e ffi ciently 20/30

Advantages of scripting · Once you have a routine, it is much faster · Changes can be made quickly and easily · You have a record of every step performed · The analysis is reproducible · Your colleagues (inside and outside the lab) can see exactly what was done 21/30

4. Pre-registration · Documenting detail of method and analysis plan before analyzing (or collecting) data 22/30

Why it is helpful · Prevents rushing into data collection · Better ensures that students collecting data understand what they are doing (and can provide useful criticism ahead of time) · Makes it clear to students how to proceed with data analysis · Prevents p-hacking 23/30

Does pre-registration slow projects down? · Probably. · But also - prevents wasting time running sub-optimal designs - helps catch confounds - improves pedagogy · Inculcates a lab culture where it is clear that speaking up about design is welcome 24/30

Why these things spec fi cally? · Adopted them because I saw they would solve speci fi c problems I experience directing a lab run largely on short-term student labor - Uncertainty about what students had done in the lab - Or with the data - Wanting to make better use of their e ff orts · Didn't adopt them wholesale, immediately - Was always gradual, picked something I could manage, built on it later 25/30

Openness in the lab makes openness outside it easier · When you are con fi dent about - the provenance of the data - the reproducibility of the analyses - the quality of the design · Less anxiety about sharing outwards · Sharing publicly has been bene fi cial - Increases citations - Reputation bene fi ts - Goodwill from others who appreciate your resources 26/30

Transparency is for your team 27/30

Transparency is also for Science 28/30

Transparency is also for Science But it's mostly for you. 29/30

Thanks for your attention! My data and materials are publicly available on Open Science Framework (https://osf.io/4xwa8) Blogging at The Mnemonic Lode, candicemorey.org Twitter: @CandiceMorey Editor-in-chief, www.journalofcognition.org 30/30