C+I Metrics Initiative Introducing a Crowdsourced Bottom-Up Approach - - PowerPoint PPT Presentation

▶

Apr 12, 2024 258 likes •533 views

C+I Metrics Initiative Introducing a Crowdsourced Bottom-Up Approach to Developing Transdisciplinary Scholarship Metrics Ivica Ico Bukvic, Director Creativity+Innovation ico@vt.edu Overview 1. Background 3. Demo+Experience Motivation

SLIDE 1

C+I Metrics Initiative

Introducing a Crowdsourced Bottom-Up Approach to Developing Transdisciplinary Scholarship Metrics Ivica Ico Bukvic, Director

Creativity+Innovation ico@vt.edu

SLIDE 2

http://bit.ly/ci-metrics-a2ru

Overview

1. Background
Motivation
Existing Work
2. C+I Metrics
What is It?
What It is Not...
…But May Become
60s Initiative History
How Does It Work?
3. Demo+Experience
4. Reflection
Pilot Results
Lessons Learned
Emerging Needs
5. Next Steps
Scaling the Initiative
Acknowledgments
Discussion

SLIDE 3

http://bit.ly/ci-metrics-a2ru

1. Background

Accountability increasingly important in higher ed

Motivation

SLIDE 4

http://bit.ly/ci-metrics-a2ru

1. Background

Accountability increasingly important in higher ed Existing metrics does not capture all forms of scholarship

Motivation

SLIDE 5

http://bit.ly/ci-metrics-a2ru

1. Background

Accountability increasingly important in higher ed Existing metrics does not capture all forms of scholarship No metrics is perfect

Motivation

SLIDE 6

http://bit.ly/ci-metrics-a2ru

1. Background

Accountability increasingly important in higher ed Existing metrics does not capture all forms of scholarship No metrics is perfect Important distinction between the metric and measurement

Motivation

SLIDE 7

http://bit.ly/ci-metrics-a2ru

1. Background

Accountability increasingly important in higher ed Existing metrics does not capture all forms of scholarship No metrics is perfect Important distinction between the metric and measurement Higher ed paradox

Motivation

SLIDE 8

http://bit.ly/ci-metrics-a2ru

1. Background

Challenges

Existing Work

One size does not fit all Capturing instead of conforming Defining artistic research Lack of resources

SLIDE 9

http://bit.ly/ci-metrics-a2ru

1. Background

Challenges Academic Analytics

Existing Work

SLIDE 10

http://bit.ly/ci-metrics-a2ru

1. Background

Challenges Academic Analytics vs. Alt Metrics

Existing Work

A number of projects/initiatives Focus on early impact Repositories Standardization Alt Metrics in CVs Is it already mainstream?

SLIDE 11

http://bit.ly/ci-metrics-a2ru

1. Background

Challenges Academic Analytics vs. Alt Metrics Hybrid approaches

Existing Work

Citations vs. readership Contextualized data approach Repositories

SLIDE 12

http://bit.ly/ci-metrics-a2ru

1. Background

Challenges Academic Analytics vs. Alt Metrics Hybrid approaches Grassroots efforts

Existing Work

SLIDE 13

http://bit.ly/ci-metrics-a2ru

2. C+I Metrics

A proactive, bottom-up, crowdsourced, and adaptable survey instrument designed to: Validate existing and identify new forms of scholarly output, and Offer metrics by which similar forms of scholarly output can be valued and compared. A mechanism for: Identifying and filling the gaps in the inter/trans/disciplinary scholarly metrics. Reevaluating existing metrics.

What is It?

SLIDE 14

http://bit.ly/ci-metrics-a2ru

2. C+I Metrics

A tool for facilitating academic unit’s summative scholarly output to the university for the purpose of performance-based budget. A way to empower individual faculty’s self-reflection and appropriately value their own scholarly output. A malleable tool with anecdotal data that supports its hypothesis that calls for further testing and iterative improvement. An IRB approved protocol that includes supporting stakeholder engagement mechanisms.

What is It?

SLIDE 15

http://bit.ly/ci-metrics-a2ru

2. C+I Metrics

An effort to develop a summative measurement of a potentially diverse scholarly output. A way to provide a rigid structure to a P&T process.

What It is Not...

A way to compare scholarly output from two or more scholars within the same scholarly domain.

..but May Become

SLIDE 16

http://bit.ly/ci-metrics-a2ru

2. C+I Metrics

60s Initiative History

SLIDE 17

http://bit.ly/ci-metrics-a2ru

2. C+I Metrics

Guiding principles

How Does It Work?

Granularity-Consolidation Participatory design Anonymity

SLIDE 18

http://bit.ly/ci-metrics-a2ru

2. C+I Metrics

Guiding principles VT Pilot

How Does It Work?

Short and broad Minimizing redundancy Iterative Research categories Scholarly outputs Proposed metrics Analysis Reflection Adoption

SLIDE 19

http://bit.ly/ci-metrics-a2ru

3. Demo + Experience

SLIDE 20

http://bit.ly/ci-metrics-a2ru

4. Reflection

50% response rate Interest in both quantitative and qualitative New proposed ways of evaluation as a way to define viable metrics Interest in capturing online presence (Alt Metrics)

Pilot Results

SLIDE 21

http://bit.ly/ci-metrics-a2ru

4. Reflection

C+I Metrics model affords disciplines underrepresented in the scholarly metrics discourse empowerment that moves them away from a defensive stance towards a more proactive one. The underrepresented disciplines in the scholarly metrics discourse may benefit from the well-established metrics that may at first appear not to be applicable to them. For the success and recognition of diverse forms of scholarship outside the well-established norms, including the emergent transdisciplinary fields, universities may need to seek inspiration from the industry, potentially including Google’s 80-20 model within the context of the resource allocation.

Lessons Learned

SLIDE 22

http://bit.ly/ci-metrics-a2ru

4. Reflection

Universities may want to develop capacity for treating their administrative and educational structures as an ongoing research project, rather than a tenure-track process. Universities may need to develop capacity for nimble experimentation and adaptation.

Lessons Learned

SLIDE 23

http://bit.ly/ci-metrics-a2ru

4. Reflection

Project’s full potential can be only realized through broader engagement

Emerging Needs

SLIDE 24

http://bit.ly/ci-metrics-a2ru

5. Next Steps

Scaling the Initiative

Exploring building a cross-institutional taskforce Considering a2ru as the host

SLIDE 25

http://bit.ly/ci-metrics-a2ru

5. Next Steps

Acknowledgments

VT CLAHS Kelly Scarff Lotte Mitchell Reford Thanassis Rikakis SOPA colleagues who partook in the pilot

SLIDE 26

http://bit.ly/ci-metrics-a2ru

Discussion

Thank you

C+I Metrics Initiative

Introducing a Crowdsourced Bottom-Up Approach to Developing Transdisciplinary Scholarship Metrics Ivica Ico Bukvic, Director

Virginia Tech Creativity+Innovation ico@vt.edu