Towards Tangible Trusted Learning Analytics Prof. Dr. Hendrik - - PowerPoint PPT Presentation

▶
towards tangible trusted learning analytics
SMART_READER_LITE
LIVE PREVIEW

Towards Tangible Trusted Learning Analytics Prof. Dr. Hendrik - - PowerPoint PPT Presentation

Mar 19, 2024 149 likes •675 views

Towards Tangible Trusted Learning Analytics Prof. Dr. Hendrik Drachsler @hdrachsler Learning Analytics Workshop, 08.04.2019, TU Delft, The Netherlands WhoAmI Hendrik Drachsler Professor of Educational Technologies Research topics

slide-1
SLIDE 1
  • Prof. Dr. Hendrik Drachsler @hdrachsler

Towards Tangible Trusted Learning Analytics

Learning Analytics Workshop, 08.04.2019, TU Delft, The Netherlands

slide-2
SLIDE 2

WhoAmI

Hendrik Drachsler Professor of Educational Technologies Research topics Recommender Systems Learning Analytics Multimodal Data for learning Computational Psychometrics Application domains Schools HEI Medical education

slide-3
SLIDE 3

Educational Technologies Team

Nicole Elker Function: Management Assistent Daniel Biedermann Function: PhD student Sambit Praharaj Function: PhD student George Ciordas- Hertel Function: PhD student Sebastian Wollny Function: PhD student

  • Dr. Jan

Schneider Function: PostDoc Atezaz Ahmad Function: PhD student Ioana Jivet Function: PhD student Daniele Dimitri Function: PhD student Marcel Schmitz Function: PhD student

  • Dr. Maren

Scheffel Function: PostDoc Hector Pijeira Diaz Function: PhD student

Superhero’s

slide-4
SLIDE 4
  • 1. Definition
  • f trust and

LA

  • 2. Fears of

Learning Analytics

  • 4. Approaches

towards Trusted Learning Analytics

  • 3. Human-

Centered Design

Lecture structure

slide-5
SLIDE 5

What are learning analytics for you?

slide-6
SLIDE 6

Greller, W. & Drachsler, H. (2012). Turning Learning into Numbers. Toward a Generic Framework for Learning Analytics. Journal of Educational Technology & Society.

http://ifets.info/journals/15_3/4.pdf

Learning Analytics

slide-7
SLIDE 7

Greller, W. & Drachsler, H. (2012). Turning Learning into Numbers. Toward a Generic Framework for Learning Analytics. Journal of Educational Technology & Society.

http://ifets.info/journals/15_3/4.pdf

Learning Analytics

slide-8
SLIDE 8

Sophistican model

Siemens, G., Dawson, S., & Lynch, G. (2014). Improving the Quality and Productivity of the Higher Education Sector – Policy and Strategy for Systems-Level Deployment of Learning Analytics. Canberra, Australia: Office of Learning and Teaching, Australian

  • Government. Retrieved from http://solaresearch.org/Policy_Strategy_Analytics.pdf

1 2 3 4 5

Learning Analytics

slide-9
SLIDE 9

9

slide-10
SLIDE 10

How do you define trust?

slide-11
SLIDE 11

Oxford Dictionary Trust is about a firm belief in the reliability, truth, or ability of someone or something. A trustful relation is mutually based on:

  • penness
  • truth
  • reliability
  • integrity
  • belief
  • faith
  • freedom of suspicion

Picture by Terry Johnston https://www.flickr.com/photos/powerbooktrance/466709245/

Multiple definitions of Trust

Trust = a multidimensional and multidisciplinary construct

Various Contexts

  • neself and others
  • rganizations
  • intelligent systems
  • automation
  • money or political power
slide-12
SLIDE 12

Niklas Luhmann. Trust and power. John Willey & Sons (1979).

A definition of Trust

Luhmann defined ‘TRUST’ as a way to cope with risk, complexity, and a lack of system understanding. For Luhmann the concept of trust compensates for insufficient capabilities for full understanding the complexity of the world.

Picture by: https://twitter.com/ niklasluhmann

slide-13
SLIDE 13

https://en.wikipedia.org/wiki/Ni klas_Luhmann

Trust in Learning Analytics

  • Data subjects face

uncertainty e.g. when receiving outcomes of learning analytics.

  • Data subjects can not fully

understand the complexity

  • f learning analytics.
  • Data subjects take a risk and

making oneself vulnerable by feeding learning analytics with personal data.

Following Luhmann, we define trust as a social phenomenon with the following characteristics: To gain trust from data subjects we need to demonstrate Transparency, Reliability, and

  • Integrity. As a return the data

subjects might ‘choose’ to trust us.

slide-14
SLIDE 14
  • 1. Definition of

trust and LA

  • 2. Fears of

Learning Analytics

  • 4. Approaches

towards Trusted Learning Analytics

  • 3. Human-

Centered Design

Lecture structure

slide-15
SLIDE 15

People are afraid of AI (in TEL)

Learning Analytics: Dystopia

slide-16
SLIDE 16

Examples why people don’t trust

slide-17
SLIDE 17

Keynote Neil Selwyn @ LAK 2018, Sydney, Australia

Learning Analytics has a trust problem …

Learning Analytics

slide-18
SLIDE 18

… because Learning Analytics has the potential

  • f becoming a high stakes

assessment.

Keynote Neil Selwyn @ LAK 2018, Sydney, Australia

Learning Analytics

slide-19
SLIDE 19

Being in the next industrial revolution means we are in an education system, where the norms, relationships and ways of teaching and learning are impacted.

  • Authority: Public -> Private

Influence and power are redistributed

  • New (AI) actors:

Feedback to students from machines

  • Data ownership:

Increased access for some may mean reduced access for others

  • False-truths:

Early products with simplistic reasoning don’t represent what learning is really about

https://commons.wikimedia.org/w iki/File:Coalbrookdale_loco.jpg

Education in the Industrial Revolution

slide-20
SLIDE 20

Open algorithms Transparent indicators No automated decisions Full access to data Knowing who accesses your data Feedback culture Unknown algorithms Unknown data collection Automated decisions No access to raw data No control who uses it Assessment culture

Black box vs. White box

slide-21
SLIDE 21
  • Right to be informed
  • Right of access
  • Right to rectification
  • Right to erasure
  • Right to restrict processing
  • Right to data portability
  • Right to object automated decision making

Do your Learning Technology systems support these rights? Hands-up!

GDPR 2018

slide-22
SLIDE 22
  • 1. Definition
  • f trust and

LA

  • 2. Fears of

Learning Analytics

  • 4. Approaches

towards Trusted Learning Analytics

  • 3. Human-

Centered Design

Lecture structure

slide-23
SLIDE 23

Some things are already on its way

http://www.open.ac.uk/students/charter/ess ential-documents/ethical-use-student-data- learning-analytics-policy# https://www.jisc.ac.uk/sites/default/file s/jd0040_code_of_practice_for_learni ng_analytics_190515_v1.pdf

slide-24
SLIDE 24

Drachsler, H. & Greller, W. (2016). Privacy and Analytics – it’s a DELICATE issue. A Checklist to establish trusted Learning Analytics. 6th Learning Analytics and Knowledge Conference 2016, April 25-29, 2016, Edinburgh, UK. Online at: http://www.laceproject.eu/ethics-privacy/

Some things are already on its way

slide-25
SLIDE 25

http://www.sheilaproject.eu Yi-Shan Tsai, Pedro Manuel Moreno-Marcos, Kairit Tammets, Kaire Kollom, and Dragan Gašević. 2018. SHEILA policy framework: informing institutional strategies and policy processes of learning

  • analytics. In Proceedings of

the 8th International Conference on Learning Analytics and Knowledge (LAK '18). ACM, New York, NY, USA, 320-329. DOI: https://doi.org/10.1145/31703 58.3170367

Some things are already on its way

There is no other Educational Technology discipline like Learning Analytics that critically works on social implications of their outcomes and addresses institutional development.

slide-26
SLIDE 26
  • 1. Definition
  • f trust and

LA

  • 2. Fears of

Learning Analytics

  • 4. Approaches

towards Trusted Learning Analytics

  • 3. Human-

Centered Design

Lecture structure

slide-27
SLIDE 27
  • Design-Based Research (DBR)
  • AB– testing

Barab, S. A. (2014). Design-based research: a methodological toolkit for engineering change. In K. Sawyer (ed.) Handbook of the Learning Sciences, Vol 2, (pp. 233-270), Cambridge, MA: Cambridge University Press.

Participatory Design-Process

slide-28
SLIDE 28

Moodle environment

slide-29
SLIDE 29
  • TLA is the first GDPR 2018

conform Big Data infrastructure followed a value-based design approach

  • Joined project with

GU, DIPF und OU

  • Among ‘traditional‘ learning

data we also aim to collect multimodal data.

Trusted Learning Analytics Infrastructure

slide-30
SLIDE 30

Trusted Learning Analytics Dashboard

slide-31
SLIDE 31

Declaration of Consent

slide-32
SLIDE 32

Self-Reflection Phase

SEREne Dashboard

Performance Phase Forethought Phase

slide-33
SLIDE 33

How to design your Trusted Learning Analytics

TACTIC Cube Trusted Analytics Cube to Teach Institutional Change

slide-34
SLIDE 34

35

Stakeholders

  • 1. Interviews with students (n=46)
  • 2. Survey on Learning Analytics (n=166)
  • 3. Group Concept Mapping Study (n=101, 46)
  • 4. Feedback from students, and teachers on dashboards
slide-35
SLIDE 35

Stakeholders

slide-36
SLIDE 36

37

Reflection Prediction

Objectives

slide-37
SLIDE 37

Objectives

slide-38
SLIDE 38

39

Scheffel, M., Ternier, S., & Drachsler, H. (2016a). The Dutch xAPI Specification for Learning Activities (DSLA) – Registry. Retrieved from http://bit.ly/DutchXAPIreg

http://www.laceproject.eu/blog/xapi-dsla/

Educational Data

slide-39
SLIDE 39

40

Educational Data

slide-40
SLIDE 40

41

Verbert, K., Duval, E., Klerkx, J., Govaerts, S., & Santos, J. L. (2013). Learning analytics dashboard applications. American Behavioral Scientist.

Technologies

slide-41
SLIDE 41

42

Technologies

slide-42
SLIDE 42

Drachsler, H. & Greller,

  • W. (2016). Privacy and

Analytics – it’s a DELICATE issue. A Checklist to establish trusted Learning

  • Analytics. LAK 2016,

April 25-29, Edinburgh, UK. Engelfriet, A., Jeunink, E., Manderveld, J. (2015). Learning analytics onder de Wet bescherming persoonsgegevens

Constraints

slide-43
SLIDE 43

Constraints

slide-44
SLIDE 44

45

Interpretation skills

Drachsler, H., Stoyanov, S., d'Aquin, M., Herder, E., Dietze, S., & Guy, M. (2014, 16-19 September). An Evaluation Framework for Data Competitions in TEL. 9th European Conference on Technology-Enhanced Learning (EC-TEL 2014), Graz, Austria.

slide-45
SLIDE 45

46

Interpretation skills

slide-46
SLIDE 46

47

Critical thinking

1.Data literacy 2.Agency 3.Privacy understanding

slide-47
SLIDE 47

48

Critical thinking

slide-48
SLIDE 48

Trusted Learning Analytics Cube

slide-49
SLIDE 49

Take home messages

  • 1. We need to actively develop and apply learning

analytics tools to have informed discussion what are the effects on the stakeholders.

  • 2. We need participatory design approaches to involve

all stakeholders in learning analytics and train their agency and data literacy skills.

  • 3. We have an opportunity through the GDPR and the

stakeholder discourse in point 2 to design more humanistic Trusted Learning Analytics.

slide-50
SLIDE 50

Learning Analytics & Knowledge Conference 2020

http://lak20.solaresearch.org

slide-51
SLIDE 51

52

Questions?

@hdrachsler

drachsler@dipf.de

https://www.linkedin.com/in/hendrikdrachsler