Identifying and understanding 21st Century Skills Professor Jeppe - - PowerPoint PPT Presentation

Professor Jeppe Bundsgaard

Professor Jeppe Bundsgaard

Danish School of Education Aarhus University

Identifying and understanding 21st Century Skills

Professor Jeppe Bundsgaard

Outline

• 21st Century Skills projects

– What is it? And what are the problems?

• An alternative approach to identifying 21st skills

– Prototypical situation oriented approach

• How to identify and understand 21st century

skills as part of assessment development

– The Danish 21st Skills assessment

Professor Jeppe Bundsgaard

P21 Partnership for 21st Century Skills ATC21s Assessment and teaching of 21st Century Skills

A number of 21st Century Skills projects

WEF Future of Jobs

Professor Jeppe Bundsgaard

Methods to identify and describe 21st Century Skills

• Brainstorming

– P21 – “P21 has identified and brought to the forefront a comprehensive set of skills that,

along with content mastery, are what all sectors can agree are essential for success. The P21 Framework for 21st Century Learning took several years to develop ...”

• Survey of frameworks

– ATC21s – “To arrive at this model framework we compared a number of available curriculum

and assessment frameworks for twenty-first century skills and skills that have been developed around the world. We analyzed these frameworks to determine not only the extent to which they differ but also the extent to which these frameworks provide descriptions of twenty-first century learning outcomes in measureable form.”

• Survey of experts

– World Economic Forum: The future of jobs – “survey of CHROs and other senior talent and strategy executives of leading global

employers”

Professor Jeppe Bundsgaard

Challenges

• Identification challenge

– How do we know that we have found the core skills?

• Ranking challenge

– How do we know that these skills are the most important ones?

• Description challenge

– How do we define, describe and subdivide each skill/competence?

• Teaching challenge

– How do we know which parts of the skills to focus on? What is hard,

what is easy?

• Organizational challenge

– How can the disciplines contribute (methods, knowledge, epistemic

frames)? How should the school be organized?

Professor Jeppe Bundsgaard

Solution?

A prototypical situation oriented approach

• A prototypical situation is a central member of a

category of situations (Cf. Eleanor Rosch)

• Recurrent situations, typified rhetorical action (Miller)
• E.g.

– Being a teacher means participating in

• Teaching situations
• Parent-teacher conferences
• Professional development
• Eating lunch
• Walking the hallways to class

– Not all these situations are prototypical of being a teacher.

Professor Jeppe Bundsgaard

The method

• Identify

– (first very general, later on

more specific) prototypical situations which we want students to be able to take part in

• Analyze challenges

– Describe and examine

empirical practices

• Identify competences (skills)

– Analyze the challenges and the

ways to address these challenges, i.e. the competences needed

• Politics
• Sociology
• Anthropology
• ‘Didactology’
• Psychology

Professor Jeppe Bundsgaard

How to identify prototypical situations

• Through sociological reasoning

– What are the main domains of a society?

• E.g. Habermas: The civil society

– What are the main types of situations in these domains?

• Through sociological imagination

– How will our societies develop? – What will be the main types of situations in this imagined

future?

• Through political dialogue

– What society do we want students to participate in developing? – Which situations should they therefore be able to take part in?

Professor Jeppe Bundsgaard

Six core identities

and their general prototypical situations

Professor Jeppe Bundsgaard

How to identify prototypical situations II, and identify challenges

• Through empirical studies of main types of

situations

– E.g. Studies of prototypical work practices (cf.

Wenger)

– Surveys, interviews, observations

• Through sociological and anthropological

reasoning and imagination

• Through production and study of fiction

(literature, movies, poetry etc.)

Professor Jeppe Bundsgaard

An example

Danish L1 Teacher’s prototypical situations

Professor Jeppe Bundsgaard

How to identify competences?

• Analyses of how challenges are met/could be

met

• Didactical and psychological reasoning and

imagination.

• Identification of approaches, methods,

knowledge etc. from the academic disciplines

– And how they can be brought into use.

Professor Jeppe Bundsgaard

What are the most important skills? (the ranking challenge)

• Different criteria:

– The situations that most people are in (or is expected

to be in), most of the time (sociology/anthropology)

– The situations we as a society find most important to

improve on (policy)

• As a citizen
• As a worker
• As a person, consumer, aesthete, etc.
• Which criteria are used in the 21st skills projects?

Professor Jeppe Bundsgaard

The description and teaching challenges

• An example:

– An assessment of 21st Century Skills

Professor Jeppe Bundsgaard

Design principles

• Design six scenarios and story lines
• Put the students in prototypical 21st century

situations in these scenarios

• Give the students tasks that are prototypical of

the situations

– With a focus on handling and production of

information, collaboration, project work

Professor Jeppe Bundsgaard

2 example modules

Professor Jeppe Bundsgaard

We are going to get a pet in the after-school club

Professor Jeppe Bundsgaard

Brainstorming ideas

Professor Jeppe Bundsgaard

Reflecting on other peoples suggestions

Professor Jeppe Bundsgaard

Researching the web

Professor Jeppe Bundsgaard

Planning how to build a cage

Professor Jeppe Bundsgaard

Choosing the design

Professor Jeppe Bundsgaard

Finding information

Professor Jeppe Bundsgaard

Evaluating information

Professor Jeppe Bundsgaard

Planning how to take care of the rabbit

Professor Jeppe Bundsgaard

Designing a poster

Professor Jeppe Bundsgaard

Charity dinner

Professor Jeppe Bundsgaard

Brainstorm tasks

Professor Jeppe Bundsgaard

Allocate tasks

Professor Jeppe Bundsgaard

Plan the preparations for the event

Professor Jeppe Bundsgaard

Interior design

Professor Jeppe Bundsgaard

Search information

Professor Jeppe Bundsgaard

Evaluate information

Professor Jeppe Bundsgaard

Evaluate information

Professor Jeppe Bundsgaard

Collaborate and solve conflicts

Professor Jeppe Bundsgaard

Collaborate with external business partners

Professor Jeppe Bundsgaard

Prepare a budget

Professor Jeppe Bundsgaard

Design a web page

Professor Jeppe Bundsgaard

Design

• Around 3000 students from 3rd through 8th grade

in 15 schools took the test

– Pre-test winter 2013 – Post-test spring 2015

• 6 modules
• Each student was assigned two modules at pre-

test and two other modules at post-test

Professor Jeppe Bundsgaard

Rotated design

Module Grades A B C D E F 3rd Pre Post Post Pre 4th – 6th Pre Post Pre Post Pre Post Post Pre Post Pre Pre Post Post Pre Post Pre 7th – 8th Pre Post Pre Post Pre Post Post Pre Post Pre Pre Post Post Pre Post Pre

Professor Jeppe Bundsgaard

Dimensions

• Scenario competence

– Imagine complex situations and envisage development and solutions to

challenges

• Process competence

– Planning and addressing issues that occur during a project

• Composition competence

– Compose multimodal messages that fits the situation

• Information competence

– Prepare and carry through Internet searches, and be critical towards the

information

• Collaboration competence

– Communicate and collaborate with peers, experts, costumers etc., identify

and solve conflicts

Professor Jeppe Bundsgaard

Short digression Rasch analysis helped my understanding

• Each task was described by a

dimension and a subdimension

• Scenario competence kept

having bad fit

• It turned out that most bad items

was from the planning subdimension

• I created a new dimension

mostly consisting of these items: the process dimension.

• An indication that management
• f time is not strongly connected

to the management of actors and space

• Scenario competence

– Brainstorm – Creating ideas – Planning – Consequences of

choices

– Interior design – Systematize

Professor Jeppe Bundsgaard

Scoring

• Multiple choice
• Drag and drop
• Human scored short text responses
• Human scored multimodal responses
• Polytomous and dichotomous items

Professor Jeppe Bundsgaard

Data

• 2604 cases (pre-test – post-test for 1302

students)

• 231 variables

– Collaboration: 18 – Information: 29 – Production: 51 – Scenario: 60 – Process: 73

Professor Jeppe Bundsgaard

Rasch analysis

• Polytomous facet model
• Using grade as a categorical variable to

account for the non-normal distribution of students

• Using gender to account for DIF

Professor Jeppe Bundsgaard

Reliability, FIT

• Fit: Weighted MNSQ between .75-1.21
• PV-reliability: 0.679, 0.621, 0.681, 0.816, 0.677

Professor Jeppe Bundsgaard

Gender and Grade differences

Professor Jeppe Bundsgaard

Wright Maps

Professor Jeppe Bundsgaard

Construction of Construct Maps

Professor Jeppe Bundsgaard

Scenario Competence

• The student is capable of imagining complex

situations and envisaging development and solutions to challenges

Professor Jeppe Bundsgaard

• 360

Can systematize concrete activities and things in categories. Can imagine a tangible situation and suggest important elements. Can design a room so that it can, to some extent, fulfill its purpose. Can imagine positive consequences of a concrete action. 360-445 Can imagine tasks directly related to a business activity. Can imagine a room and decorate it so that it is usable to some extent to the purpose. Can spot some of the consequences of one's own and other peoples' specific choices. 445-530 Can imagine negative consequences of a tangible action or situation. Can characterize a text with a key word. Can find ways to jointly choose a

• product. Can see the importance of having infrastructure in a city.

530-615 Can imagine secondary functions of common phenomena. Can imagine a hypothetical situation and ask the participants relevant questions about their wishes. Can draw a route on a map, but with shortcomings. Can design a room so an advanced activity can take place in a desirable way. 615- Can imagine the consequences of an expansion of existing practices (e.g., adding a new service to the portfolio of an institution). Can design and justify the layout of rooms and departments so that they largely fulfill their functions. Can draw complicated routes on a map. Can imagine a wide range of functions and activities of common phenomena.

Scenario Competence Construct Map

Professor Jeppe Bundsgaard

Process Competence

• Is capable of planning and addressing issues

that occur during a project

Professor Jeppe Bundsgaard

• 360

Can place concrete activities into plans, and to some extent relate them to each other. Can to some extent organize activities in types. 360-445 Can relate to the reciprocal temporal relationships of activities (put the first activity first). Can come up with a few tasks that need to be solved in a given situation. 445-530 Can to some extent put activities in a suitable timespan. Can come up with more concrete tasks in connection with an activity or service. 530-615 Can place a number of activities in the correct chronological order. Can come up with many concrete tasks in connection with an activity or service. 615- Can organize the subtasks in a complex activity appropriately, including ensuring an appropriate order, preventing that activities coincide or pile up.

Process Competence Construct Map

Professor Jeppe Bundsgaard

Collaboration competence

• Can communicate and collaborate with peers,

experts, costumers etc., identify and solve conflicts

Professor Jeppe Bundsgaard

Collaboration

• 360

Can put oneself in another person’s place and give reasons for the person’s point of view. 360- 445 Can see the consequences of a given distribution of tasks. Can take care of easily accommodated wishes in the redistribution of tasks. 445- 530 Can put oneself in other people’s place, e.g. by quoting reasons given for a certain sentiment, or by quoting reasonable reasons for the person's actions. Can contribute to resolving conflicts with support from suggestions or by contributing simple suggestions. Can take care of more persons' wishes in the planning of an activity. 530- 615 Can identify problems in a duty roster and come up with suggestions for

• solutions. Can put oneself in another person’s place by explaining why she or

he felt hurt. Can point to and give examples on unreasonable arguments for

• ther people’s point of view. Can come up with more suggestions of how to

jointly solve a conflict. 615- Can come up with a comprehensive suggestion to how a disagreement about a choice can be jointly resolved. Can come up with a pragmatic solution to a problem in a duty roster by involving all parties in the solution. Can respond to a conflict by giving reasons both for and against another person's actions. Can choose a suitable answer in a conflict and give more reasons.

Professor Jeppe Bundsgaard

Conclusion

Answers to the challenges

• Identification challenge

– Identify the prototypical situations

• Ranking challenge

– Sociology, anthropology, policy

• Description challenge

– Analyze challenges in situations. Develop measurements.

• Teaching challenge

– Construct/Wright maps

• Organization challenge

– N/A

Professor Jeppe Bundsgaard

Thank you!