Databrary Advisory Board Spring Meeting April 7, 2014 NYU 1 - - PowerPoint PPT Presentation

databrary advisory board spring meeting
SMART_READER_LITE
LIVE PREVIEW

Databrary Advisory Board Spring Meeting April 7, 2014 NYU 1 - - PowerPoint PPT Presentation

Databrary Advisory Board Spring Meeting April 7, 2014 NYU 1 Meeting Agenda 1:00-1:15 Welcome and recent accomplishments 1:15-1:45 Roadmap 1:45-2:15 Policies 2:15-2:30 Break 2:30-3:30 Beta: Upload and search 3:30-4:00 Researcher asks


slide-1
SLIDE 1

Databrary Advisory Board Spring Meeting

April 7, 2014 NYU

1

slide-2
SLIDE 2

Meeting Agenda

1:00-1:15 Welcome and recent accomplishments 1:15-1:45 Roadmap 1:45-2:15 Policies 2:15-2:30 Break 2:30-3:30 Beta: Upload and search 3:30-4:00 Researcher asks and support 4:00-4:10 Break 4:10-5:00 Positioning Databrary for the future 5:00-5:30 Wrap up and discussion

2

slide-3
SLIDE 3

Announcements

Next Board Meeting: October 13, 2014 at NYU Wi-fi access: nyuguest

Guest ID: guest132 Password: keddlith

Meeting will be recorded Push-to-talk microphones Reimbursement form and receipts to Lina Wictoren Roy Dinner following at Hundred Acres

38 Macdougal Street, corner of Prince

3

slide-4
SLIDE 4

Databrary PIs

Karen Adolph NYU David Millman NYU Rick Gilmore Penn State

4

slide-5
SLIDE 5

Agency partners

Jim Griffin, NICHD, Science Officer Lisa Freund, NICHD, Program Officer Laura Namy, NSF, Director DLS

5

slide-6
SLIDE 6

Special Guests

Lynn Liben

Distinguished Professor of Psychology, Penn State, President of the Society for Research in Child Development

Daphne Maurer

Distinguished University Professor of Psychology, Neuroscience & Behaviour, McMaster University, President of the International Society on Infant Studies

Amanda Woodward

William S. Gray Professor of Psychology, University of Chicago, President of the Cognitive Development Society

6

slide-7
SLIDE 7

Special thanks to NYU

David Ackerman: IT support, storage, hiring Nancy Daneau: Authorization Alison Dewhurst and Marti Dunne:

Participant release, IRB

Pamela Morris: IHDSC, administrative support Eric Rasmussen and Mark Righter:

Authorization, university agreements, legal

7

slide-8
SLIDE 8

Very special thanks

Richard Louth

8

slide-9
SLIDE 9

Databrary team

Dylan Simon Jesse Lingeman John Franchak Vicky Foo Jon Coe Mike Continues Andrea Byrne Lisa Steiger Lina Wictoren Roy

9

slide-10
SLIDE 10

Board member updates

Welcome to new advisors

Daniel Messinger, University of Miami, analytic tools Brian Nosek, University of Virginia, COS Tal Yarkoni, University of Texas, neurosynth.org

Farewell to departing member

Sarah Morrow, PSU

10

slide-11
SLIDE 11

Publications

Submitted

International Journal of Digital Libraries

In progress

WIRE’s Cognitive Science (invited paper) Best practices in behavioral coding Call to arms Top 10 reasons to share Top 10 concerns about sharing

11

slide-12
SLIDE 12

Presentations

NFAIS/CENDI/Library of Congress (Nov. 12, 2013) CNI 2013 Fall Meeting (Dec. 9-10, 2013) Spatial DataMine Workshop (Feb. 7-9 2014) 12+ presentations to psychologists, pediatric physical/occupational therapists, movement scientists

12

slide-13
SLIDE 13

Datavyu

Released stable version of Datavyu 1.1 Published user guide Best practices in behavioral coding forthcoming

13

slide-14
SLIDE 14

Seeding the repository: Datasets acquired

Adolph Tamis-LeMonda Karasik Gordon LoBue

14

slide-15
SLIDE 15

Forthcoming datasets

Messinger lab Gernsbacher lab Maher longitudinal study CHILDES videos Gesell archives

In partnership with MIAP, Howard Besser

15

slide-16
SLIDE 16

Meeting Agenda

1:00-1:15 Welcome and recent accomplishments 1:15-1:45 Roadmap 1:45-2:15 Policies 2:15-2:30 Break 2:30-3:30 Beta: Upload and search 3:30-4:00 Researcher asks and support 4:00-4:10 Break 4:10-5:00 Positioning Databrary for the future 5:00-5:30 Wrap up and discussion

16

slide-17
SLIDE 17

Why Databrary exists

17

slide-18
SLIDE 18

Why Databrary exists

Challenge in developmental psychology

Most developmental researchers collect video data Not exploiting the richness in video Good data going to waste Impeding transparency and the pace of discovery

17

slide-19
SLIDE 19

Why Databrary exists

Challenge in developmental psychology

Most developmental researchers collect video data Not exploiting the richness in video Good data going to waste Impeding transparency and the pace of discovery

Collecting and storing video is costly

17

slide-20
SLIDE 20

Why Databrary exists

Challenge in developmental psychology

Most developmental researchers collect video data Not exploiting the richness in video Good data going to waste Impeding transparency and the pace of discovery

Collecting and storing video is costly Open up videos to be repurposed for new questions

17

slide-21
SLIDE 21

Key aims of the Databrary project

Build Databrary repository for sharing video Create appropriate policy environment Provide data management tools Enhance Datavyu tool for scoring video Transform the culture of developmental science!

18

slide-22
SLIDE 22

View & Use Videos Videos to Share

Roadmap

19

slide-23
SLIDE 23

View & Use Videos Videos to Share Upload

Roadmap

20

slide-24
SLIDE 24

View & Use Videos Videos to Share Upload

Roadmap

Data dump

Datasets are browsable but not searchable at session level

20

slide-25
SLIDE 25

View & Use Videos Videos to Share Upload

Roadmap

Data dump

Datasets are browsable but not searchable at session level

Structured datasets and studies

Assign appropriate metadata to each session

20

slide-26
SLIDE 26

View & Use Videos Videos to Share Upload

Roadmap

Videos are identifiable

So each video (session) must have a release level Dataset dump not possible because metadata tags for each session are required

21

slide-27
SLIDE 27

Videos to Share Upload

Roadmap

22

slide-28
SLIDE 28

Videos to Share Upload

Roadmap

23

slide-29
SLIDE 29

Upload After the Fact Upload As You Go Videos to Share Upload

Roadmap

24

slide-30
SLIDE 30

Upload After the Fact Upload As You Go Videos to Share Upload

Roadmap

25

slide-31
SLIDE 31

Roadmap

Upload As You Go Videos to Share Upload Upload After the Fact

Incentives: Preservation; data easily available to collaborators; mensch...? Disincentives: Time; effort; $.

26

slide-32
SLIDE 32

Upload After the Fact Upload As You Go Videos to Share Upload

Roadmap

27

slide-33
SLIDE 33

Upload After the Fact Upload As You Go Videos to Share Upload

Data Collection Coding, Analyses, Writing Paper in Press

Roadmap

Ask Ss to Share

28

slide-34
SLIDE 34

Upload After the Fact Upload As You Go Videos to Share Upload

Data Collection Coding, Analyses, Writing Paper in Press

Roadmap

Ask Ss to Share

28

slide-35
SLIDE 35

Upload After the Fact Upload As You Go Videos to Share Upload

Data Collection Coding, Analyses, Writing Paper in Press

Roadmap

Ask Ss to Share

28

slide-36
SLIDE 36

Upload After the Fact Upload As You Go Videos to Share Upload

Data Collection Coding, Analyses, Writing Paper in Press

Roadmap

Ask Ss to Share

28

slide-37
SLIDE 37

Upload After the Fact Upload As You Go Videos to Share Upload

Data Collection Coding, Analyses, Writing Paper in Press

Roadmap

Ask Ss to Share

28

slide-38
SLIDE 38

Upload After the Fact Upload As You Go Videos to Share Upload

Data Collection Coding, Analyses, Writing Paper in Press

Roadmap

Ask Ss to Share

28

slide-39
SLIDE 39

Upload After the Fact Upload As You Go Videos to Share Upload Datavyu

Data Collection Coding, Analyses, Writing Paper in Press

Roadmap

Ask Ss to Share

29

slide-40
SLIDE 40

Upload After the Fact Videos to Share Upload

Incentives: Preservation; “lab server;” easy access for students & collaborators; data organization; coding, transcoding syncing, and splitting videos Disincentives: Change current practices

Upload As You Go

Data Collection Coding, Analyses, Writing Paper in Press Ask Ss to Share

30

slide-41
SLIDE 41

Upload After the Fact Upload As You Go Videos to Share Upload

Ask Ss to Share Data Collection Coding, Analyses, Writing Paper in Press

IRB

Roadmap

31

slide-42
SLIDE 42

Roadmap

Data Collection Coding, Analyses, Writing Paper in Press

Upload After the Fact Upload As You Go Videos to Share Upload

Incentives: Use of excerpts. Disincentives: None!

IRB

Ask Ss to Share

32

slide-43
SLIDE 43

Upload After the Fact Upload As You Go Videos to Share Upload

Grants & Contracts Data Collection Coding, Analyses, Writing Paper in Press

Roadmap

Ask Ss to Share

33

slide-44
SLIDE 44

Ask Ss to Share Data Collection Coding, Analyses, Writing Paper in Press

IRB

Upload As You Go Upload Upload After the Fact

Grants & Contracts

Incentives: Full access. Disincentives: None.

Videos to Share View & Use Videos

34

slide-45
SLIDE 45

Upload As You Go Upload Videos to Share Upload After the Fact View & Use Videos

Grants & Contracts Data Collection Coding, Analyses, Writing Paper in Press

2018-04

IRB

Timeline

Ask Ss to Share

35

slide-46
SLIDE 46

Upload As You Go Upload Videos to Share Upload After the Fact View & Use Videos

Grants & Contracts Data Collection Coding, Analyses, Writing Paper in Press

IRB

Timeline

Ask Ss to Share

36

slide-47
SLIDE 47

Upload As You Go Upload Videos to Share Upload After the Fact View & Use Videos

Grants & Contracts Data Collection Coding, Analyses, Writing Paper in Press

2017-07

IRB

Timeline

Ask Ss to Share

36

slide-48
SLIDE 48

Upload As You Go Upload Videos to Share Upload After the Fact View & Use Videos

Grants & Contracts Data Collection Coding, Analyses, Writing Paper in Press

2017-07

IRB

Timeline

Ask Ss to Share

37

slide-49
SLIDE 49

Upload As You Go Upload Videos to Share Upload After the Fact View & Use Videos

Grants & Contracts Data Collection Coding, Analyses, Writing Paper in Press

2017-07

IRB

2014-04

Timeline

Ask Ss to Share

37

slide-50
SLIDE 50

Upload As You Go Upload Videos to Share Upload After the Fact View & Use Videos

Grants & Contracts Data Collection Coding, Analyses, Writing Paper in Press

IRB

2017-07 2014-04

Timeline

Ask Ss to Share

38

slide-51
SLIDE 51

Upload As You Go Upload Videos to Share Upload After the Fact View & Use Videos

Grants & Contracts Data Collection Coding, Analyses, Writing Paper in Press

IRB

2017-07 2015-04 2014-04

Timeline

Ask Ss to Share

38

slide-52
SLIDE 52

Upload As You Go Upload Videos to Share Upload After the Fact View & Use Videos

Grants & Contracts Data Collection Coding, Analyses, Writing Paper in Press

IRB

2017-07 2014-04

Timeline

Ask Ss to Share

39

slide-53
SLIDE 53

Upload As You Go Upload Videos to Share Upload After the Fact View & Use Videos

Grants & Contracts Data Collection Coding, Analyses, Writing Paper in Press

IRB

2017-07 2014-04 2016-04

Timeline

Ask Ss to Share

39

slide-54
SLIDE 54

Upload As You Go Upload Videos to Share Upload After the Fact View & Use Videos

Grants & Contracts Data Collection Coding, Analyses, Writing Paper in Press

IRB

2017-07

Timeline

Ask Ss to Share

40

slide-55
SLIDE 55

Upload As You Go Upload Videos to Share Upload After the Fact View & Use Videos

Grants & Contracts Data Collection Coding, Analyses, Writing Paper in Press

IRB

2017-07 2014-09 2015-09

Timeline

Ask Ss to Share

40

slide-56
SLIDE 56

Upload As You Go Upload Videos to Share Upload After the Fact View & Use Videos

Grants & Contracts Data Collection Coding, Analyses, Writing Paper in Press

IRB

2017-07 2014-09

Timeline

Ask Ss to Share

41

slide-57
SLIDE 57

Upload As You Go Upload Videos to Share Upload After the Fact View & Use Videos

Grants & Contracts Data Collection Coding, Analyses, Writing Paper in Press

IRB

2017-07 2014-09 2016-09

Timeline

Ask Ss to Share

41

slide-58
SLIDE 58

Upload as you go (Labnanny)

We need to get the video data into Databrary We can’t rely on researchers uploading data after their paper is submitted We need to make it easy to upload data while researchers are collecting it

42

slide-59
SLIDE 59

What we need from advisory board

Advocate for data sharing and Databrary Lead by example Identify and share already-collected data Request participants to share their data Become authorized Databrary Investigators Explore the beta site and give us feedback Provide insights for designing upload and search

43

slide-60
SLIDE 60

Meeting Agenda

1:00-1:15 Welcome and recent accomplishments 1:15-1:45 Roadmap 1:45-2:15 Policies 2:15-2:30 Break 2:30-3:30 Beta: Upload and search, Datavyu 2.0 3:30-4:00 Researcher asks and support 4:00-4:10 Break 4:10-5:00 Positioning Databrary for the future 5:00-5:30 Wrap up and discussion

44

slide-61
SLIDE 61

Policy framework

Special problem with video: Identifiable data Policies that enable open sharing of identifiable data

Informed consent to share videos Ensuring adherence to a common set of practices and ethical principles

45

slide-62
SLIDE 62

Databrary release

Template release form Informed consent: can share identifiable data if you tell participants (all depicted individuals) and they agree Standardization across contributors

Developed to correspond directly with release levels

46

slide-63
SLIDE 63

Release levels

Available restrictions on identifiable data:

Did not ask: If undocumented; functions as private Private: Restricted to data owners and editors Shared: Restricted to authorized Databrary investigators Shared + excerpts: Restricted like shared, but excerpts may be used for informational, scientific, and educational purposes Public: Available to the public

47

slide-64
SLIDE 64

What data are being released?

Videos from a session (or part of a session)

Can decline sharing for sessions or segments of sessions that contains sensitive information

Session- and participant-level metadata that are identifiable

Birthdates, faces, names, interior of homes, classrooms, disabilities, self-reported health info, etc.

Codes of behaviors

48

slide-65
SLIDE 65

Databrary release: What is unique?

Completely separates consent to participate from release to share

Consent to participate (in the study) before Release to share after—it’s clear what was recorded

49

slide-66
SLIDE 66

Our template works

Multiple iterations of release form and procedure for talking with participants

Tested on our own participants!

Adaptable and flexible

Only local IRB must approve Can collect decisions of all depicted individuals on one page

Used in many contexts

International, oral/written, diverse ethnic groups, children with disabilities

50

slide-67
SLIDE 67

Video example

databrary.org/user-guide/getting-started/release-script/example-videos.html

51

slide-68
SLIDE 68

Video example

databrary.org/user-guide/getting-started/release-script/example-videos.html

51

slide-69
SLIDE 69

Just do it!

Asking to share does not obligate you to share Can’t share without asking We’ll store everything, so long as participants are asked We need as many researchers as possible to incorporate Databrary releases

52

slide-70
SLIDE 70

Progress on releases

9 labs are currently requesting participant release 10 labs have pending IRB protocol amendments 17 other labs in process of adding Databrary to IRB

53

slide-71
SLIDE 71

Discussion re: excerpts

Most common (pre-Databrary) use case

Clips for talks, teaching Demonstration of specific procedure or method Relations between behaviors and codes

Why ask for excerpts separately?

Effectively public Needs to be clear to participants

54

slide-72
SLIDE 72

Databrary and excerpts

Should Databrary make excerpts publicly available on the site? Who decides which excerpts are public on Databrary?

Only the data owner or any PI on Databrary?

55

slide-73
SLIDE 73

Authorization

Access to shared videos Contribute and/or use data

No requirement to contribute

Institutional sign-off by Authorizing Official

Grants and contracts No IRB needed

56

slide-74
SLIDE 74

Access

PIs can authorize and manage affiliates Can also add lab members and collaborators

Regardless of authorization

Four pre-selected options

Lab and collaborators only Databrary data only Lab data and Databrary data Proxy

57

slide-75
SLIDE 75

E V E R Y B O D Y R E G I S T E R E D U S E R S A U T H O R I Z E D U S E R S C O L L A B O R A T O R S / L A B I N T E R E S T E D P A R T I E S Access publicly shared data Access and share Databrary data Upload data Access released data, selectively shared Access and edit all private data

58

slide-76
SLIDE 76

Terms of the agreement

Researcher upholds same ethical standards as with their own data Responsibility of researcher to get whatever approval is necessary when use constitutes human subjects research Institution must verify that Investigator is eligible to be a Databrary Principal Investigator

59

slide-77
SLIDE 77

Progress on authorization

7 schools signed:

NYU, PSU, Rochester, Indiana, McMaster, UVA, Rutgers

Points of clarification

Does not go to the IRB office for signing Further, IRB approval not required for authorization Only authorizing official can bind institution or enter agreement

60

slide-78
SLIDE 78

Feedback

Agreement written with NYU and PSU Piloted agreement on advisory board We want as much feedback as possible

Where are researchers getting hung up? Where are institutions getting hung up?

61

slide-79
SLIDE 79

Registration demo

62

slide-80
SLIDE 80

Next step for authorization policy

Shift to institutional agreements

Once schools are comfortable with Investigator Agreement No substantive changes to re-frame as an institutional agreement

Allows universities to manage their authorizations on Databrary

63

slide-81
SLIDE 81

Beta release

Sending email tomorrow (4/8) with private beta link

Register and request authorization

Explore the site and tell us what you think! We will authorize you temporarily through the beta if needed Please don’t download Tajik videos

64

slide-82
SLIDE 82

Outreach plan for Databrary 1.0

Community outreach

Invites to ICIS, CDS, SRCD members Conference exhibits at major academic society meetings Workshops on sharing identifiable data

Timesavers as incentives

Developing ‘boilerplate’ Databrary language for grant proposals and reports Data management plans (partner with DMPtool) Resource sharing plans

65

slide-83
SLIDE 83

What we need from advisory board

Advocate for data sharing and Databrary Lead by example Identify and share already-collected data Request participants to share their data Become authorized Databrary Investigators Explore the beta site and give us feedback Provide insights for designing upload and search

66

slide-84
SLIDE 84

Break

67

slide-85
SLIDE 85

Meeting Agenda

1:00-1:15 Welcome and recent accomplishments 1:15-1:45 Roadmap 1:45-2:15 Policies 2:15-2:30 Break 2:30-3:30 Beta: Upload and search 3:30-4:00 Researcher asks and support 4:00-4:10 Break 4:10-5:00 Positioning Databrary for the future 5:00-5:30 Wrap up and discussion

68

slide-86
SLIDE 86

Datavyu current and future

Released stable Datavyu 1.1 desktop version and user guide Addressing remaining bugs as they trickle in, but otherwise complete Next steps (to begin 2015+): Bring Datavyu into Databrary web framework

Target simple coding tasks, on-line Later possibility for off-line version

69

slide-87
SLIDE 87

Databrary site

70

slide-88
SLIDE 88

October 2013 board meeting

Lots of videos

Piled up into studies Minimal organization

“Static” views

Single presentation option Limited flexibility

71

slide-89
SLIDE 89

Development progress

Revised technical requirements

Flexible discovery, browsing options Responsive, intuitive editing, uploading High-level visualizations, summaries API, scripting interface

Built new, completely dynamic JS interface Front-end uses API

72

slide-90
SLIDE 90

Uploading after the fact

Learned a lot from manually curating and

  • rganizing existing and new datasets

Broadened types of possible, standard metadata Developed better ways to group and relate videos, metadata, studies, datasets

73

slide-91
SLIDE 91

Lessons from the data

Always exceptions, irregularities in real data

Missing, multiple videos Don’t fit cleanly in defined groups Excluded and later re-purposed data Always something non-rectangular

Different types of data (classroom, public, longitudinal) have different needs

74

slide-92
SLIDE 92

Updated model

Better understanding of constraints, required flexibility Need responsive, iterative, “data-driven” approach

Real data are always more nuanced

75

slide-93
SLIDE 93

Walkthrough of live site

76

slide-94
SLIDE 94

Next steps

Improve discovery within and across datasets

Searching: by keyword, text, descriptions, names, etc. Filtering: limiting results by age, gender, numeric values, etc. Sorting: order to present results, for scanning visually

View summarized or aggregated data (counts, distributions, visualizations, etc.) Many existing user stories and use cases

77

slide-95
SLIDE 95

User stories: Teaching and talks

Video clips for teaching Illustrate an idea Show the range of behaviors and exceptions Show an excerpt in a talk

78

slide-96
SLIDE 96

User stories: Pre-research

Browse the work in my field Decide whether a study is worth doing Preliminary data for grant proposal Ideas and inspiration Replicate, expand on, or review previous work based on the procedure or coding manual

79

slide-97
SLIDE 97

User stories: Research

Repurpose videos for new uses Replicate existing work by recoding videos Grow sample size Include participants from other contexts and populations Conduct integrative analyses Complete grant progress report

80

slide-98
SLIDE 98

Next steps

Primary focus was on browsing, searching, using (downloading, coding, commenting, tagging) New focus: Upload as you go Build browser interface that also makes sense for entering and editing data

81

slide-99
SLIDE 99

User stories: Upload as you go

Describe study design (conditions, groups, etc.) Enter (type/paste/import) session metadata Upload new video, associate it with session Keep track of which data were entered Export previously entered data for analysis Customize presentation of dataset (title, excerpts, display)

82

slide-100
SLIDE 100

Design process for upload as you go

Focus group including developers, UI/UX experts, researchers Refine user stories Identify requirements and priorities Iterative process, test and improve interface

83

slide-101
SLIDE 101

Inferring structure as you go

Initial questions about study design are useful But, things are always changing midcourse Allow users to iterate easily by finding smart ways to expand structure as they go

84

slide-102
SLIDE 102

Meeting Agenda

1:00-1:15 Welcome and recent accomplishments 1:15-1:45 Roadmap 1:45-2:15 Policies 2:15-2:30 Break 2:30-3:30 Beta: Upload and search 3:30-4:00 Researcher asks and support 4:00-4:10 Break 4:10-5:00 Positioning Databrary for the future 5:00-5:30 Wrap up and discussion

85

slide-103
SLIDE 103

Session is the basic unit

Covered by a release level Data collected at same time Contains timeline of raw data files Annotated with metadata

Session

April 7, 2014

inta ke.mp4 ba by- ca m.mp4

  • verhead-

ca m.mp4 survey.doc

86

slide-104
SLIDE 104

Session metadata

Release level Date Pilot Excluded Participants Conditions Tasks Groups Location

87

slide-105
SLIDE 105

Session release level

Not asked Private Shared Shared + excerpts Public

88

slide-106
SLIDE 106

Session date

Technically considered identifiable information Could be unknown (Gesell)

89

slide-107
SLIDE 107

Pilot

Indicates that the methods used in the session were not finalized or standard

90

slide-108
SLIDE 108

Excluded

Reason

Did not meet inclusion criteria Procedural/experimenter error Withdrew/fussy/tired Outlier ...

Indicates that session was not usable for the target study

91

slide-109
SLIDE 109

Participant

Subject ID Birthdate Gender Race (NIH) Ethnicity (NIH) Disability/typically dev Language Country (of origin) SES? ...

Represents an individual depicted or represented in this session

92

slide-110
SLIDE 110

Task

Name Summary Description Represents a particular task, activity, or phase that occurred during all or a portion of the session

93

slide-111
SLIDE 111

Condition

Name Summary Description Other variables... Represents an experimenter-determined manipulation (within or between sessions)

94

slide-112
SLIDE 112

Group

Name Summary Description Other variables... Represents a grouping of sessions determined by an aspect of the data (participant ability, age, experience, measurements used/available)

95

slide-113
SLIDE 113

Location

Setting

lab, home, classroom,

  • utdoors, clinic,

museum ...

Language (of study materials) State Country Represents a particular setting or other variable aspect of where/when/how session data were collected

96

slide-114
SLIDE 114

Datasets and studies (provenance)

Datasets: raw data collected in sessions Studies: generated data, analyses, papers Studies include sessions from datasets and “layer” files over them Allow re-purposing, re-presenting data

97

slide-115
SLIDE 115

Session is the basic unit

Covered by a release level Data collected at same time Contains timeline of raw data files Annotated with metadata Analysis files layered on

Session

April 7, 2014

inta ke.mp4 ba by- ca m.mp4 coding

  • spreadsheet.opf
  • verhead-

ca m.mp4 a na lysis.sa v survey.doc

98

slide-116
SLIDE 116

Datasets and Studies

One to one mapping likely to be rare

Excluded Ss, pilot Ss In interest of transparency and repurposing, should be no disincentive for sharing

Excerpts only, displays only

Studies, no datasets

Repurposing data

Studies and datasets

99

slide-117
SLIDE 117

Simple after-the-fact cases

Easy to upload studies with:

Publication, abstract Excerpts only Stimuli, displays

100

slide-118
SLIDE 118

What’s on the table for standardizing?

Metadata variables describing sessions, participants, studies, datasets Not interfering with the science Not standardizing tasks

Subgroups can do this (emo-group)

Not standardizing codes

Datavyu is agnostic

101

slide-119
SLIDE 119

Discussion: How much to standardize

Continuum between enforced structure and flexibility Currently, types of metadata (participant, task, location, etc.) are fixed

Is this enough? Too much? Are there others? Should we let users define their own?

Variables within metadata (birthdate, language, description) are user-extensible

Should we restrict these? Add more?

102

slide-120
SLIDE 120

Benefits and costs of standardization

More powerful searching

Video or session level searches

Aggregating across datasets But possible change in current practices More work for PIs to document measures

103

slide-121
SLIDE 121

Potential solutions for flexibility

Encourage use of existing tags (pop-up suggestions) Our curators clean up after the fact (merge conceptually identical tags) Leave the mess as-is

Semantic search (google: “crawl” finds “crawling”, “crawlers”, etc.)

104

slide-122
SLIDE 122

Already doing stuff as you go

Organizing, typing, writing data anyway after each data collection session

How is this done now? How can we make this non-invasive/simpler than current practices?

Uploading, transcoding, splitting videos anyway Exporting to SPSS/Excel/Datavyu anyway

105

slide-123
SLIDE 123

Things that change later

Exclusions, groups, entire study organization can change later Needs to be easy to do this in Databrary, drag around sessions, include/exclude, add to study, make new group Easy to create, download excerpts Last step: add publication citation, done!

106

slide-124
SLIDE 124

Taking advantage of your Databrary lab

Organization within labs requires less standardization

Each can have their own standards

Search within your own data Mine your data for subsequent analyses Easy to disseminate examples Need effective ‘overview’ and ‘monitoring’ tools

107

slide-125
SLIDE 125

What we need from advisory board

Advocate for data sharing and Databrary Lead by example Identify and share already-collected data Request participants to share their data Become authorized Databrary Investigators Explore the beta site and give us feedback Provide insights for designing upload and search

108

slide-126
SLIDE 126

Break

109

slide-127
SLIDE 127

Meeting Agenda

1:00-1:15 Welcome and recent accomplishments 1:15-1:45 Roadmap 1:45-2:15 Policies 2:15-2:30 Break 2:30-3:30 Beta: Upload and search 3:30-4:00 Researcher asks and support 4:00-4:10 Break 4:10-5:00 Positioning Databrary for the future 5:00-5:30 Wrap up and discussion

110

slide-128
SLIDE 128

Quick SWOT analysis

Strengths Weaknesses Opportunities Threats

111

slide-129
SLIDE 129

Strengths

Solution for sharing identifiable data Growing awareness, visibility within content domain Flexible data model, poised for growth University, agency support Free, open-source video coding tool

112

slide-130
SLIDE 130

Weaknesses

Data curation hard, slow Fixing the past vs. fixing the future Sustainability model undetermined Small curation, development staff Datavyu user base small

How to compete/cooperate with commercial entities?

113

slide-131
SLIDE 131

Your thoughts?

Strengths we neglected or overstated? Weaknesses we neglected or overstated?

114

slide-132
SLIDE 132

Threats

Open science space moving quickly Attracting venture capital

115

slide-133
SLIDE 133

Data-sharing

116

slide-134
SLIDE 134

117

slide-135
SLIDE 135

118

slide-136
SLIDE 136

Threats

Databrary software not yet released How can Databrary compete? Why share with Databrary vs. alternatives? What features will attract, keep users? What features will ensure sustainability, change the culture of developmental science?

119

slide-137
SLIDE 137

Your turn

More threats? Different spin on them?

120

slide-138
SLIDE 138

Opportunities

Potential for partnering

TalkBank/CHILDES Other sharing organizations

  • Research Data Alliance (RDA)
  • Center for Open Science (COS)
  • Stanford Meta-Research Center

Societies

  • Report on developmental society meeting

121

slide-139
SLIDE 139

Opportunities

Potential for partnering

Journals

  • Store supplemental data?
  • Store extended materials for all submissions?
  • Which fields?
  • Developmental science only
  • Psychology
  • Neuroscience

122

slide-140
SLIDE 140

Opportunities

Potential for partnering

Conferences (both society-affiliated and not)

  • Store, share talk recordings?
  • Store supplemental data?

Sustainability component

  • Member or attendee fees-for-service?

123

slide-141
SLIDE 141

Opportunities

Emphasize competitive advantages

Domain-focus, visibility

  • Your colleagues are here

Policies to share identifiable research data, especially recordings University & Federal support Stability and long-term sustainability Sharing is our business model

124

slide-142
SLIDE 142

Your thoughts?

Partnerships Journals Conferences Sharpen our message Other ideas?

125

slide-143
SLIDE 143

Opportunities

Integration with external services

Cloud storage/data sharing

Offer new services via APIs

Lab management Project/task management Data visualization Social media, communications Electronic CV, profile management

126

slide-144
SLIDE 144

Your thoughts?

Do you use any of these services? Is providing data sharing services enough? Does integration with other tools help or hinder our goals?

127

slide-145
SLIDE 145

Opportunities

Beyond storing, tagging video

Visualization Additional analysis streams

128

slide-146
SLIDE 146

Meeting Agenda

1:00-1:15 Welcome and recent accomplishments 1:15-1:45 Roadmap 1:45-2:15 Policies 2:15-2:30 Break 2:30-3:30 Beta: Upload and search 3:30-4:00 Researcher asks and support 4:00-4:10 Break 4:10-5:00 Positioning Databrary for the future 5:00-5:30 Wrap up and discussion

129

slide-147
SLIDE 147

Thank you

130