1 st FLIP Event 18 th 19 th of June 2018 Paris INTRODUCTION - - PowerPoint PPT Presentation

1st FLIP Event 18th‐19th of June 2018 Paris

First FLIP Event 18th‐19th of June 2018, Paris

INTRODUCTION Thierry Rocher Office of Student Assessment, DEPP

• FLIP in brief
• Participants
• Agenda
• Housekeeping

First FLIP Event 18th‐19th of June 2018, Paris

Introduction: FLIP in brief

Context:

• Transition of national assessments from paper‐based

to e‐assessment

• Demand for very‐large‐scale assessments
• Interest in technology enhanced assessments
• Integration of assessment and learning

‘FLIP’ acronym: France‐Luxembourg‐Italy‐Portugal 1st meeting: January 2017, Rome New member: CAEd from Brazil Interest from: Sweden, Japan, Denmark, NY Dept of Education, Canada ‐ Québec, ...

First FLIP Event 18th‐19th of June 2018, Paris

Introduction: FLIP in brief

A collaborative framework

• Three axes (SHARING):

Sharing knowledge and experiences in e‐assessment Sharing IT development costs Sharing content

• Two principles (OPEN):

Open‐source Open standards

Event Goal: building collaboration and roadmap

First FLIP Event 18th‐19th of June 2018, Paris

Country Institution Brazil Caed/Ufjf ‐ Centro de Políticas Públicas e Avaliação da Educação da Universidade Federal de Juiz de Fora Denmark Aarhus University Aalborg University France CCIP ‐ Chambre de Commerce et d’Industrie de Paris Idf DSI Lille – Direction Service Informatique Lille DEPP ‐ Direction de l’évaluation, de la prospective et de la performance Germany IEA ‐ International Association for the Evaluation of Educational Achievement ‐ Data Processing Center Italy INVALSI ‐ Istituto nazionale per la valutazione del sistema educativo di istruzione e di formazione Japan NIER ‐ National Institute for Educational Policy Research NIAD‐QE ‐ National Institute for Academic Degree and Quality Enhancement of Higer Education

Introduction: participants

Participants

First FLIP Event 18th‐19th of June 2018, Paris

Country Institution Kyrgyz Republic Ministry of education and science UNICEF Lithuania National Examination Centre Luxembourg MENJE‐SCRIPT ‐ Service de Coordination de la Recherche et de l´Innovation pédagogiques et technologiques University of Luxembourg Morocco INE ‐ Institut national d’évaluation Norway Norwegian Directorate for Education and Training Portugal IAVE ‐ Instituto de avaliação educativa, I.P. UK NFER – National Foundation for Educational Research

Participants

Introduction: participants

First FLIP Event 18th‐19th of June 2018, Paris

Country Institution Egypt Ministry of education Poland Educational Research Institute Canada ‐ Québec Ministère de l'Éducation et de l'Enseignement supérieur Sweden Skolverket United states New‐York City Department of Education

Apologies

Introduction: participants

First FLIP Event 18th‐19th of June 2018, Paris

Company Country Cito Netherlands GERA solutions Lithuania IMS Sweden OAT Luxembourg Vretta Canada Wiqid France

Corporate representatives

Introduction: participants

First FLIP Event 18th‐19th of June 2018, Paris

Introduction: agenda

First FLIP Event 18th‐19th of June 2018, Paris

Introduction: agenda

First FLIP Event 18th‐19th of June 2018, Paris

Country report France

First FLIP Event 18th‐19th of June 2018, Paris

Summary

• Student assessment in France
• Transition to e‐assessment
• Very‐large‐scale assessment
• Innovative assessment
• Contributions to FLIP

French Ministry of Education

Primary & Secondary education

• 1.1 million employees

– Largest employer of France

• Organized as 30 “académies”

– 52,000 elementary schools – 11,000 secondary schools – 13 million students

• Total budget of 93 billion EUR

First FLIP Event 18th‐19th of June 2018, Paris

First FLIP Event 18th‐19th of June 2018, Paris

Classroom level Classroom level

to provide standardised tools to teachers in order to improve pedagogical interventions

Local level Local level

to provide indicators to educational managers in order to monitor pedagogical actions

National level National level

to provide national results, trends and international comparisons in

• rder to evaluate the education

system

DEPP and student assessment

DEPP is the Department of Statistics of the Ministry of Education DEPP leads many standardised assessment programmes 3 levels – 3 Objectives

National surveys

Assessment Programmes

CEDRE Subjects CEDRE Subjects

Sample‐based subject‐ based assessment programme Grades 5 and 9 ‐ French ‐ Foreign languages ‐ History & Geography ‐ Science ‐ Mathematics

Common Base Common Base

Sample‐based competences assessment programme Grades 3, 6 and 9 ‐ Common base competences, ‐ Socio‐cognitive competences, ‐ 21st century skills (4C)

Longitudinal Longitudinal

Longitudinal surveys: school pathways and family resources Grade 6 2007, Grade 1 2011, handicapped students (2005 & 2010) Subjects and competences

Policy evaluation Policy evaluation

2017: class size reduction policy in socio‐ economic disadvantaged schools, grade 1 to 3

PISA PISA

Programme for International Student Assessment 15 years‐old Reading Literacy Mathematical Literacy Scientific Literacy

PIRLS PIRLS

Progress in International Reading Literacy Survey Grade 4 Reading Comprehension

TIMSS TIMSS

Trends in International Mathematics and Science Study Grades 4, 8 and 12 Mathematics & Science

ICILS ICILS

International Computer and Information Literacy Study Grade 8 Digital Literacy Computational thinking

Grades 1 & 2 Grades 1 & 2

Diagnostic Test Early and mid Grade 1 Early Grade 2 French Maths

Grade 6 Grade 6

Diagnostic test & monitoring test Early grade 6 French Maths

Grade 10 Grade 10

Diagnostic Early Grade 10 French Maths

Army Day Army Day

Illiteracy diagnostic 17‐years old French girls and boys Reading

International surveys Nation‐wide assessments

First FLIP Event 18th‐19th of June 2018, Paris

First FLIP Event 18th‐19th of June 2018, Paris

Transition to e‐assessment

Since 2015, transition from paper‐based assessment to e‐ assessment:

• New opportunities: cost reduction, potential easiness to implement,

in line with digital school, more engaging, …

• New features: multimedia, interactivity, simulated situations, 21st

century skills, adaptive testing, log‐data, ...

• New challenges: comparability PBA/eA, usability, equipment,

security, confidentiality, …

2 axes as 2 added values of e‐assessment:

• 1. Very‐large‐scale assessment
• 2. Innovative assessment

First FLIP Event 18th‐19th of June 2018, Paris

E‐assessment in practice

• Dependency on school equipment
• Primary schools

– Offline tablet solution (mobile classes) – Sample‐based programmes

• Secondary schools

– Online solution (schools’ computers) – Sample‐based and nation‐wide programmes

• Particular case: international assessments (USB sticks)

First FLIP Event 18th‐19th of June 2018, Paris

Tablet‐based assessment

• Primary schools

– Mobile App (D‐Tab) – Adapted to young children – E.g. : 30,000 grade 1 and grade 2 students longitudinal survey

• Next step: nation‐wide?

– September 2018: 1.7 millions grade 1 + 2 students, paper‐based App version available – Use of school tablets?

Very‐large‐scale assessment

First FLIP Event 18th‐19th of June 2018, Paris

• Grade 6 Nation‐wide assessment (November 2017)

– 830,000 students / 7,100 schools – First “pilot” in 2015: 160,000 students – French and Maths, adaptive (MST) – TAO full web

• Feedback

– Participation rate: 98% – Scalability: up to 50 000 simultaneous connections – Offline version in 2018 for schools with low bandwidth

• A successful very‐large scale assessment thanks to e‐assessment
• Yearly programme + extension to grade 10 (September 2018)

Item samples

First FLIP Event 18th‐19th of June 2018, Paris

Innovative assessment

• Digital functionalities for better measurement

Multimedia, time data, accessibility, adaptive procedures, interactivity, …

• Illustration: Technology‐Enhanced Items (TEI)

From simple animations to interactive items 1) To enrich “traditional subjects” assessments 2) To explore “new” competences (4C)

• First results from sample‐based assessment programmes

First FLIP Event 18th‐19th of June 2018, Paris

Technology Enhanced Items

First FLIP Event 18th‐19th of June 2018, Paris

ChatBot Physics Maths (with Luxembourg) Computational thinking

Data Analytics

• Log‐data analysis :

strategies analysis + output automatic analysis Example: Math – May 2016 – 8,000 grade 9 students

• First insights

– Identification of students profiles by supervised classification

First FLIP Event 18th‐19th of June 2018, Paris

Perspectives

General context: a growing demand for e‐assessment Contributions to FLIP so far: ‐ Sharing experience ‐ Contributing to IT development (e.g. offline, portals, etc.) ‐ Buidling tools (Chatbot, etc.) ‐ Developing contents (Math items, MathemaTIC) To continue supporting collaborative efforts in FLIP in all aspects

First FLIP Event 18th‐19th of June 2018, Paris

Roadmap discussions

First FLIP Event 18th‐19th of June 2018, Paris

First FLIP Event 18th‐19th of June 2018, Paris

Roadmap discussions introduction

Reminder:

• Three axes (SHARING):

Sharing knowledge and experiences in e‐assessment Sharing IT developments Sharing content: toward an item‐store

• Update:

Meeting in Luxembourg 25/07/17 between FLIP & OAT: A focus on the needs for developments = first roadmap

→ Roadmap discussions objectives:

• Updating about the actual roadmap
• Identifying needs from countries
• Building a new roadmap

First FLIP Event 18th‐19th of June 2018, Paris

• 1. offline capability

1.1 item caching 1.2 local server support 1.3 offline device

• 2. math tools

2.1 geogebra item types (PCI) 2.2 geogebra student tools (e.g. calculator)

• 3. online marking

3.1 marking item response 3.2 marking logistics 3.3 external marking integration (API)

• 4. reporting tools

4.1 external reporting tool integration

• 5. content sharing

5.1 shared item banks 5.2 shared plugins and extensions

• 6. accessibility

6.1 dyslexia font support

• 7. item banking UX

7.1 improved test assembly 7.2 UX for staged adaptive 7.3 improved item workflow 7.4 simplified interface for teachers

• 8. PCI/student tools

8.1 computational thinking 8.2 scratch pad student tool 8.3 chat simulator

Current FLIP Requirements List

Roadmap discussions introduction

First FLIP Event 18th‐19th of June 2018, Paris

Monday 15:30 – 17:00 Part 1: Platform functionalities Tuesday 9:00 – 10:30 Part 2: Broad issues Tuesday 11:00 – 12:30 Part 3: Technology‐enhanced assessments Any time! Questions/Comments

Roadmap discussions introduction

First FLIP Event 18th‐19th of June 2018, Paris

Community‐building: parallel sessions 1. Partnership organisations 2. Geek hands‐on session

First FLIP Event 18th‐19th of June 2018, Paris

• 1. Partnership organisation

REMINDER FLIP collaborative framework

• Sharing knowledge and experiences
• Sharing costs
• Sharing content

Two principles

• Open standards
• Open‐source

First FLIP Event 18th‐19th of June 2018, Paris

• 1. Partnership organisation

Feedback, ideas, questions from countries

• How do we share knowledge and experience?
• How do we share ongoing experiences?
• Concretely: a platform? Annual meeting?

Webinar?

• Peer to peer, bilateral, multilateral

First FLIP Event 18th‐19th of June 2018, Paris

• 1. Partnership organisation

Feedback, ideas, questions from countries

• How do we structure FLIP?
• Formal: Association, Consortium?
• Governance: how to set priorities?
• Peer to peer, bilateral, multilateral?
• How do we share costs?
• Ensuring fairness between contributors:

membership levels?

• Managing finances: what rules?

First FLIP Event 18th‐19th of June 2018, Paris

• 1. Partnership organisation

Feedback, ideas, questions from countries

• How do we share developments and contents?
• IT development

 Encouraging GPL2

• PCI development:

 Tools: encouraged to be into GPL2 as well

• Items:

 Property rights? Peer‐to‐peer agreement?

• Concretely:

 An exchange platform?

First FLIP Event 18th‐19th of June 2018, Paris

• 1. Partnership organisation

Feedback, ideas, questions from countries

• Next steps
• Plan meetings

First FLIP Event 18th‐19th of June 2018, Paris