What's in it for me? What's in it for me? Transparently organizing - - PowerPoint PPT Presentation
What's in it for me? What's in it for me? Transparently organizing your research from start to fi nish Candice C. Morey MRC-CBU Open Science Day Adopting open research practices Scientists endorse openness, but most don't prioritize it
Candice C. Morey MRC-CBU Open Science Day
Scientists endorse openness, but most don't prioritize it Perceived to be a lot of work Rewards fuzzy: · · · Idealistic only? Maybe important later, but not vital now?
3/30
Won't it slow me down? ·
4/30
Why should someone else benefit from my work? ·
5/30
What if I fail? ·
6/30
You are working together to achieve something bigger.
7/30
8/30
9/30
Each student thinks their project is theirs, but it is always part of something bigger.
10/30
To ensure consistency in training researchers To know the provenance and protect the security of data To stem information loss that comes with turn-over To make it easy to share publicly when its time · · · ·
11/30
Lab handbook Open Science Framework Scripted analysis Pre-registration · · · ·
12/30
https://ccmorey.github.io/labHandbook/
13/30
End theory-of-mind games when training new students Articulate a standard to aspire to · · So students know what I would consider "professional" (i.e., worthy of high marks)
To get feedback To be helpful So you might influence standards Enrich descriptions of your published method · · · ·
15/30
OSF: A tool you can use to organize, back-up, and eventually share finished components Sharing your finished products != working totally in public · · Custom software, stimuli, protocols: they're finished when study is ready to be run Pre-registrations: Before analyzing new data Anonymized data: when the data set is closed to new members, entries Analysis code and papers that are ready for scrutiny from someone
It's free Control project members, visibility Can get DOI assigned to project Good tool for supervision - can see quickly where project stands · · · ·
17/30
Students are introduced to OSF at start They are added to an existing project or create their own Finished products (materials, data, writing) are uploaded They "register" project at important milestones The OSF page supplements whatever we publish · · · · Creates a back-up Provides a timestamp in case we need it
18/30
Not part of my training! Workflow I learned in my training was not easy to reproduce · ·
19/30
Processing data shouldn't lead to information loss Script starts from anonymized raw data Everything done to the data is recorded in the script · · Every time
Is documented, can be changed efficiently
Once you have a routine, it is much faster Changes can be made quickly and easily You have a record of every step performed The analysis is reproducible Your colleagues (inside and outside the lab) can see exactly what was done · · · · ·
21/30
Documenting detail of method and analysis plan before analyzing (or collecting) data ·
22/30
Prevents rushing into data collection Better ensures that students collecting data understand what they are doing (and can provide useful criticism ahead of time) Makes it clear to students how to proceed with data analysis Prevents p-hacking · · · ·
23/30
Probably. But also Inculcates a lab culture where it is clear that speaking up about design is welcome · · prevents wasting time running sub-optimal designs helps catch confounds improves pedagogy
24/30
Adopted them because I saw they would solve specific problems I experience directing a lab run largely on short-term student labor Didn't adopt them wholesale, immediately · Uncertainty about what students had done in the lab Or with the data Wanting to make better use of their efforts
Was always gradual, picked something I could manage, built on it later
When you are confident about Less anxiety about sharing outwards Sharing publicly has been beneficial · the provenance of the data the reproducibility of the analyses the quality of the design
· Increases citations Reputation benefits Goodwill from others who appreciate your resources
27/30
28/30
But it's mostly for you.
29/30
My data and materials are publicly available on Open Science Framework (https://osf.io/4xwa8) Blogging at The Mnemonic Lode, candicemorey.org Twitter: @CandiceMorey Editor-in-chief, www.journalofcognition.org
30/30