is about Embracing Complexity Dragan Ga evi @dgasevic 23 rd - PowerPoint PPT Presentation

Learning Analytics in Higher Education is about Embracing Complexity Dragan Ga š evi ć @dgasevic 23 rd October 2018 NFETL Dublin, Ireland http://sheilaproject.eu/

Current state Understanding & supporting learning Moving away from deficit models

Our institution is in early days of adoption

ADOPTION CHALLENGES

Current state – Oz and Europe http://he-analytics.com http://sheilaproject.eu/

Adoption challenge Leadership for strategic implementation & monitoring Tsai, Y. S., & Ga š evi ć , D. (2017). Learning analytics in higher education – challenges and policies: a review of eight learning analytics policies. In Proceedings of the Seventh International Learning Analytics & Knowledge Conference (pp. 233-242).

Adoption challenge Equal engagement with different stakeholders Tsai, Y. S., & Ga š evi ć , D. (2017). Learning analytics in higher education – challenges and policies: a review of eight learning analytics policies. In Proceedings of the Seventh International Learning Analytics & Knowledge Conference (pp. 233-242).

Adoption challenge Policies for learning analytics practice Tsai, Y. S., & Ga š evi ć , D. (2017). Learning analytics in higher education – challenges and policies: a review of eight learning analytics policies. In Proceedings of the Seventh International Learning Analytics & Knowledge Conference (pp. 233-242).

DIRECTIONS

Inclusive adoption process

Inclusive adoption process http://sheilaproject.eu/

Inclusive adoption process Macfadyen, L., Dawson, S., Pardo, A., Gašević , D., (2014). The learning analytics imperative and the sociotechnical challenge: Policy for complex systems. Research & Practice in Assessment, 9 (Winter 2014), 17-28.

Methodology Other stakeh. Literature Students Academic staff Senior Experts - Workshops - Policy - Survey - Survey managers - Group concept - Committees - Adoption - Focus - Focus groups mapping - Survey groups - Interviews SHEILA framework Institutional policy/strateg y

SHEILA framework http://sheilaproject.eu/ Tsai, Y. S., Moreno-Marcos, P. M., Tammets, K., Kollom, K., & Gašević , D. (2018, March). SHEILA policy framework: informing institutional strategies and policy processes of learning analytics. In Proceedings of the 8th International Conference on Learning Analytics and Knowledge (pp. 320-329). ACM.

SHEILA framework http://sheilaproject.eu/ http://sheilaproject.eu/sheila-framework/

SHEILA framework http://sheilaproject.eu/

Map political context Internal and external drivers for learning analytics adoption

http://davideldon.typepad.com/eldononline/2013/05/index.html

SHEILA framework http://sheilaproject.eu/

Identify key stakeholders The project sponsor on the senior management team

Students’ perspective Students expect the use of their data provided ethics & privacy is assured

Students’ perspective Do we have legitimate interests for use? Cross-country differences in student expectations

Teaching staff’s perspective Concerned about their workload

Experts’ perspective Purpose, ethics, and privacy need to be addressed first

SHEILA framework http://sheilaproject.eu/

Identify desired behavior changes Identify areas where decisions will be informed by learning analytics

Define implications for primary users Dashboards can be harmful

Define implications for primary users Dashboards can be harmful Can lower GPA Chaturapruek, S., Dee, T. S., Johari, R., Kizilcec, R. F., & Stevens, M. L. (2018). How a Data- driven Course Planning Tool Affects College Students’ GPA: Evidence from Two Field Experiments. In Proceedings of the Fifth Annual ACM Conference on Learning at Scale (pp. 63:1 – 63:10). New York, NY, USA: ACM.

SHEILA framework http://sheilaproject.eu/

Develop engagement strategy Alignment of learning analytics with the wider institutional strategies

Develop engagement strategy How interventions will be triggered and who is responsible?

SHEILA framework http://sheilaproject.eu/

Analyze internal capacity Data storage, disposal, integration, and security evaluation

Analyze internal capacity Human, financial, and infrastructural capacity

Analyze internal capacity Evaluate institutional culture Trust in data Decision-making based on data Openness to changes and innovation Macfadyen, L. P., & Dawson, S. (2012). Numbers Are Not Enough. Why e-Learning Analytics Failed to Inform an Institutional Strategic Plan. Educational Technology & Society , 15 (3).

SHEILA framework http://sheilaproject.eu/

Establish monitoring & learning frameworks Highly immature and few actual examples

Establish monitoring & learning frameworks Establish quantitative and qualitative indicators of success Stage the process to recognize institutional development

Establish monitoring & learning frameworks Isolating the effect of learning analytics against other initiatives

EPILOGUE

Learning analytics principles Incomplete data and human involvement Transparency of data collection, use, and sharing Good governance as the core of the approach The University of Edinburgh (2017). Learning Analytics Policy, https://www.ed.ac.uk/academic-services/policies-regulations/learning-and- assessment/learning-analytics

Learning analytics principles Avoiding deficit models to address needs of all learners Algorithms can perpetuate bias Learning analytics not used to monitor staff performance The University of Edinburgh (2017). Learning Analytics Policy, https://www.ed.ac.uk/academic-services/policies-regulations/learning-and- assessment/learning-analytics

Learning analytics purposes Quality, equity, personalized feedback, coping with scale, student experience, skills, and efficiency The University of Edinburgh (2017). Learning Analytics Policy, http://www.ed.ac.uk/academic-services/projects/learning-analytics-policy

Scott, A.-M. (2018, June 22). Learning Analytics Policy Development. Retrieved June 27, 2018, from https://ammienoot.com/brain-fluff/learning-analytics-policy- development/

FINAL REMARKS

Embracing complexity of educational systems

Critical role of leadership for adoption of learning analytics

Development of culture for data-informed decision making

Sign up for the SHEILA MOOC Launch with edX planned in late November 2018

Learning Analytics in Higher Education is about Embracing Complexity Dragan Ga š evi ć @dgasevic 23 rd October 2018 NFETL Dublin, Ireland http://sheilaproject.eu/