C+I Metrics Initiative Introducing a Crowdsourced Bottom-Up Approach to Developing Transdisciplinary Scholarship Metrics Ivica Ico Bukvic, Director Creativity+Innovation ico@vt.edu
Overview 1. Background 3. Demo+Experience ● Motivation ● Existing Work 4. Reflection ● Pilot Results 2. C+I Metrics ● Lessons Learned ● What is It? ● Emerging Needs ● What It is Not... ● …But May Become 5. Next Steps ● 60s Initiative History ● Scaling the Initiative ● How Does It Work? ● Acknowledgments ● Discussion http://bit.ly/ci-metrics-a2ru
1. Background Motivation Accountability increasingly important in higher ed http://bit.ly/ci-metrics-a2ru
1. Background Motivation Accountability increasingly important in higher ed Existing metrics does not capture all forms of scholarship http://bit.ly/ci-metrics-a2ru
1. Background Motivation Accountability increasingly important in higher ed Existing metrics does not capture all forms of scholarship No metrics is perfect http://bit.ly/ci-metrics-a2ru
1. Background Motivation Accountability increasingly important in higher ed Existing metrics does not capture all forms of scholarship No metrics is perfect Important distinction between the metric and measurement http://bit.ly/ci-metrics-a2ru
1. Background Motivation Accountability increasingly important in higher ed Existing metrics does not capture all forms of scholarship No metrics is perfect Important distinction between the metric and measurement Higher ed paradox http://bit.ly/ci-metrics-a2ru
1. Background Existing Work Challenges One size does not fit all Capturing instead of conforming Defining artistic research Lack of resources http://bit.ly/ci-metrics-a2ru
1. Background Existing Work Challenges Academic Analytics http://bit.ly/ci-metrics-a2ru
1. Background Existing Work Challenges A number of projects/initiatives Focus on early impact Academic Analytics vs. Alt Metrics Repositories Standardization Alt Metrics in CVs Is it already mainstream? http://bit.ly/ci-metrics-a2ru
1. Background Existing Work Challenges Citations vs. readership Academic Analytics vs. Alt Metrics Contextualized data approach Hybrid approaches Repositories http://bit.ly/ci-metrics-a2ru
1. Background Existing Work Challenges Academic Analytics vs. Alt Metrics Hybrid approaches Grassroots efforts http://bit.ly/ci-metrics-a2ru
2. C+I Metrics What is It? A proactive, bottom-up, crowdsourced, and adaptable survey instrument designed to: Validate existing and identify new forms of scholarly output, and Offer metrics by which similar forms of scholarly output can be valued and compared. A mechanism for: Identifying and filling the gaps in the inter/trans/disciplinary scholarly metrics. Reevaluating existing metrics. http://bit.ly/ci-metrics-a2ru
2. C+I Metrics What is It? A tool for facilitating academic unit’s summative scholarly output to the university for the purpose of performance-based budget. A way to empower individual faculty’s self-reflection and appropriately value their own scholarly output. A malleable tool with anecdotal data that supports its hypothesis that calls for further testing and iterative improvement. An IRB approved protocol that includes supporting stakeholder engagement mechanisms. http://bit.ly/ci-metrics-a2ru
2. C+I Metrics What It is Not... An effort to develop a summative measurement of a potentially diverse scholarly output. A way to provide a rigid structure to a P&T process. ..but May Become A way to compare scholarly output from two or more scholars within the same scholarly domain. http://bit.ly/ci-metrics-a2ru
2. C+I Metrics 60s Initiative History http://bit.ly/ci-metrics-a2ru
2. C+I Metrics How Does It Work? Guiding principles Granularity-Consolidation Participatory design Anonymity http://bit.ly/ci-metrics-a2ru
2. C+I Metrics How Does It Work? Guiding principles Short and broad VT Pilot Minimizing redundancy Iterative Research categories Scholarly outputs Proposed metrics Analysis Reflection Adoption http://bit.ly/ci-metrics-a2ru
3. Demo + Experience http://bit.ly/ci-metrics-a2ru
4. Reflection Pilot Results 50% response rate Interest in both quantitative and qualitative New proposed ways of evaluation as a way to define viable metrics Interest in capturing online presence (Alt Metrics) http://bit.ly/ci-metrics-a2ru
4. Reflection Lessons Learned C+I Metrics model affords disciplines underrepresented in the scholarly metrics discourse empowerment that moves them away from a defensive stance towards a more proactive one. The underrepresented disciplines in the scholarly metrics discourse may benefit from the well-established metrics that may at first appear not to be applicable to them. For the success and recognition of diverse forms of scholarship outside the well-established norms, including the emergent transdisciplinary fields, universities may need to seek inspiration from the industry, potentially including Google’s 80-20 model within the context of the resource allocation. http://bit.ly/ci-metrics-a2ru
4. Reflection Lessons Learned Universities may want to develop capacity for treating their administrative and educational structures as an ongoing research project, rather than a tenure-track process. Universities may need to develop capacity for nimble experimentation and adaptation. http://bit.ly/ci-metrics-a2ru
4. Reflection Emerging Needs Project’s full potential can be only realized through broader engagement http://bit.ly/ci-metrics-a2ru
5. Next Steps Scaling the Initiative Exploring building a cross-institutional taskforce Considering a2ru as the host http://bit.ly/ci-metrics-a2ru
5. Next Steps Acknowledgments VT CLAHS Kelly Scarff Lotte Mitchell Reford Thanassis Rikakis SOPA colleagues who partook in the pilot http://bit.ly/ci-metrics-a2ru
Discussion Thank you C+I Metrics Initiative Introducing a Crowdsourced Bottom-Up Approach to Developing Transdisciplinary Scholarship Metrics Ivica Ico Bukvic, Director Virginia Tech Creativity+Innovation ico@vt.edu http://bit.ly/ci-metrics-a2ru
Recommend
More recommend
