An Exploratory Data Analysis of OECDs 2012 PISA Surveys Di Cook - - PowerPoint PPT Presentation

▶
an exploratory data analysis of oecd s 2012 pisa surveys
SMART_READER_LITE
LIVE PREVIEW

An Exploratory Data Analysis of OECDs 2012 PISA Surveys Di Cook - - PowerPoint PPT Presentation

Feb 04, 2024 252 likes •1.15k views

An Exploratory Data Analysis of OECDs 2012 PISA Surveys Di Cook Iowa State University Joint exploration with Luke Fostveldt, Alex Shum, Ian Lyttle United Nations EDA DAY Nov 10 2014 " Ti e greatest value of a picture is when

slide-1
SLIDE 1

United Nations EDA DAY Nov 10 2014

  • An Exploratory Data Analysis
  • f OECD’s 2012 PISA Surveys

Di Cook Iowa State University

  • Joint exploration with

Luke Fostveldt, Alex Shum, Ian Lyttle

slide-2
SLIDE 2

United Nations EDA DAY Nov 10, 2014

  • 30

2

"Tie greatest value of a picture is when it forces us to notice what we never expected to see."

slide-3
SLIDE 3

United Nations EDA DAY Nov 10, 2014

  • 30

Gender gap: “boys are better at math” USA doesn’t perform well compared to other countries Would expect parents, time studying and socioeconomic status to be important.

  • Education - my expectations

3

slide-4
SLIDE 4

United Nations EDA DAY Nov 10, 2014

  • 30

Results from 2012 OECD PISA tests: “the world’s global metric for quality, equity and efficiency in school education”. 485,490 students math, science and reading test scores 65 countries, between 100-1500 schools in each Student questionnaires about their environment (635 vars) Parents surveyed on work, life, income (143 vars) Principals provide information about their schools (291 vars)

DATA

4

slide-5
SLIDE 5

United Nations EDA DAY Nov 10, 2014

  • 30

5

“Show the data."

slide-6
SLIDE 6

United Nations EDA DAY Nov 10, 2014

  • 30

6

Gender & Math

  • 1. Compute means by country

and by gender.

  • 2. Show mean difference by

country

slide-7
SLIDE 7

United Nations EDA DAY Nov 10, 2014

  • 30

7

Colombia Chile Costa Rica Luxembourg Liechtenstein Austria

Gender & Math

slide-8
SLIDE 8

United Nations EDA DAY Nov 10, 2014

  • 30

7

Gender & Math

slide-9
SLIDE 9

United Nations EDA DAY Nov 10, 2014

  • 30

7

Gender & Math

slide-10
SLIDE 10

United Nations EDA DAY Nov 10, 2014

  • 30

7

Gender & Math

slide-11
SLIDE 11

United Nations EDA DAY Nov 10, 2014

  • 30

7

USA

Gender & Math

slide-12
SLIDE 12

United Nations EDA DAY Nov 10, 2014

  • 30

Gender & Math

8

China Sweden

slide-13
SLIDE 13

United Nations EDA DAY Nov 10, 2014

  • 30

Gender & Math

9

United Arab Emirates Malaysia Tiailand Qatar Jordan

slide-14
SLIDE 14

United Nations EDA DAY Nov 10, 2014

  • 30

Gender & Math

9

United Arab Emirates Malaysia Tiailand Qatar Jordan

Surprised?

slide-15
SLIDE 15

United Nations EDA DAY Nov 10, 2014

  • 30

10

Gender & Reading

slide-16
SLIDE 16

United Nations EDA DAY Nov 10, 2014

  • 30

10

Gender & Reading Surprised again?

slide-17
SLIDE 17

United Nations EDA DAY Nov 10, 2014

  • 30

1 1

In reading scores, the world is PINK! In EVERY COUNTRY tested GIRLS score significantly BETTER THAN BOYS. Why don’t we talk about the gender gap in reading?

slide-18
SLIDE 18

United Nations EDA DAY Nov 10, 2014

  • 30

12

Individuals

  • Test scores range from 0-1000.
  • Gaps for math were at most 30

points.

  • For reading at most 80 points.
  • Differences are tiny.
slide-19
SLIDE 19

United Nations EDA DAY Nov 10, 2014

  • 30

12

Individuals

  • Test scores range from 0-1000.
  • Gaps for math were at most 30

points.

  • For reading at most 80 points.
  • Differences are tiny.
slide-20
SLIDE 20

United Nations EDA DAY Nov 10, 2014

  • 30

13

Individuals

  • Even countries with a math

gender gap, sometimes have a girl who scored the top mark.

slide-21
SLIDE 21

United Nations EDA DAY Nov 10, 2014

  • 30

13

Individuals

  • Even countries with a math

gender gap, sometimes have a girl who scored the top mark.

slide-22
SLIDE 22

United Nations EDA DAY Nov 10, 2014

  • 30

13

Individuals

  • Even countries with a math

gender gap, sometimes have a girl who scored the top mark.

Czech Republic

slide-23
SLIDE 23

United Nations EDA DAY Nov 10, 2014

  • 30

13

Individuals

  • Even countries with a math

gender gap, sometimes have a girl who scored the top mark.

Czech Republic

slide-24
SLIDE 24

United Nations EDA DAY Nov 10, 2014

  • 30

13

Individuals

  • Even countries with a math

gender gap, sometimes have a girl who scored the top mark.

Czech Republic USA

slide-25
SLIDE 25

United Nations EDA DAY Nov 10, 2014

  • 30

13

Individuals

  • Even countries with a math

gender gap, sometimes have a girl who scored the top mark.

Czech Republic USA

slide-26
SLIDE 26

United Nations EDA DAY Nov 10, 2014

  • 30

13

Individuals

  • Even countries with a math

gender gap, sometimes have a girl who scored the top mark.

Czech Republic USA Uruguay

slide-27
SLIDE 27

United Nations EDA DAY Nov 10, 2014

  • 30

14

Individuals

  • And although there is a gender

gap in reading in all countries, individually some top reading scores are obtained by boys.

slide-28
SLIDE 28

United Nations EDA DAY Nov 10, 2014

  • 30

14

Individuals

  • And although there is a gender

gap in reading in all countries, individually some top reading scores are obtained by boys.

slide-29
SLIDE 29

United Nations EDA DAY Nov 10, 2014

  • 30

14

Individuals

  • And although there is a gender

gap in reading in all countries, individually some top reading scores are obtained by boys.

France

slide-30
SLIDE 30

United Nations EDA DAY Nov 10, 2014

  • 30

14

Individuals

  • And although there is a gender

gap in reading in all countries, individually some top reading scores are obtained by boys.

France

slide-31
SLIDE 31

United Nations EDA DAY Nov 10, 2014

  • 30

14

Individuals

  • And although there is a gender

gap in reading in all countries, individually some top reading scores are obtained by boys.

France USA

slide-32
SLIDE 32

United Nations EDA DAY Nov 10, 2014

  • 30

14

Individuals

  • And although there is a gender

gap in reading in all countries, individually some top reading scores are obtained by boys.

France USA

slide-33
SLIDE 33

United Nations EDA DAY Nov 10, 2014

  • 30

14

Individuals

  • And although there is a gender

gap in reading in all countries, individually some top reading scores are obtained by boys.

France USA Uruguay

slide-34
SLIDE 34

United Nations EDA DAY Nov 10, 2014

  • 30

15

Hours spent out studying

  • ut of school time and

average math score by country

slide-35
SLIDE 35

United Nations EDA DAY Nov 10, 2014

  • 30

15

slide-36
SLIDE 36

United Nations EDA DAY Nov 10, 2014

  • 30

15

slide-37
SLIDE 37

United Nations EDA DAY Nov 10, 2014

  • 30

15

slide-38
SLIDE 38

United Nations EDA DAY Nov 10, 2014

  • 30

15

slide-39
SLIDE 39

United Nations EDA DAY Nov 10, 2014

  • 30

15

slide-40
SLIDE 40

United Nations EDA DAY Nov 10, 2014

  • 30

15

slide-41
SLIDE 41

United Nations EDA DAY Nov 10, 2014

  • 30

15

slide-42
SLIDE 42

United Nations EDA DAY Nov 10, 2014

  • 30

15

slide-43
SLIDE 43

United Nations EDA DAY Nov 10, 2014

  • 30

16

Reported truancy and average math score by country

slide-44
SLIDE 44

United Nations EDA DAY Nov 10, 2014

  • 30

16

slide-45
SLIDE 45

United Nations EDA DAY Nov 10, 2014

  • 30

16

slide-46
SLIDE 46

United Nations EDA DAY Nov 10, 2014

  • 30

16

slide-47
SLIDE 47

United Nations EDA DAY Nov 10, 2014

  • 30

16

slide-48
SLIDE 48

United Nations EDA DAY Nov 10, 2014

  • 30

16

slide-49
SLIDE 49

United Nations EDA DAY Nov 10, 2014

  • 30

16

slide-50
SLIDE 50

United Nations EDA DAY Nov 10, 2014

  • 30

16

slide-51
SLIDE 51

United Nations EDA DAY Nov 10, 2014

  • 30

16

slide-52
SLIDE 52

United Nations EDA DAY Nov 10, 2014

  • 30

17

Parents in the home and average math score by country

slide-53
SLIDE 53

United Nations EDA DAY Nov 10, 2014

  • 30

17

slide-54
SLIDE 54

United Nations EDA DAY Nov 10, 2014

  • 30

17

slide-55
SLIDE 55

United Nations EDA DAY Nov 10, 2014

  • 30

17

slide-56
SLIDE 56

United Nations EDA DAY Nov 10, 2014

  • 30

17

slide-57
SLIDE 57

United Nations EDA DAY Nov 10, 2014

  • 30

17

slide-58
SLIDE 58

United Nations EDA DAY Nov 10, 2014

  • 30

17

slide-59
SLIDE 59

United Nations EDA DAY Nov 10, 2014

  • 30

17

slide-60
SLIDE 60

United Nations EDA DAY Nov 10, 2014

  • 30

17

slide-61
SLIDE 61

United Nations EDA DAY Nov 10, 2014

  • 30

17

slide-62
SLIDE 62

United Nations EDA DAY Nov 10, 2014

  • 30

18

Possessions and average math score by country

slide-63
SLIDE 63

United Nations EDA DAY Nov 10, 2014

  • 30

18

slide-64
SLIDE 64

United Nations EDA DAY Nov 10, 2014

  • 30

18

slide-65
SLIDE 65

United Nations EDA DAY Nov 10, 2014

  • 30

18

slide-66
SLIDE 66

United Nations EDA DAY Nov 10, 2014

  • 30

18

slide-67
SLIDE 67

United Nations EDA DAY Nov 10, 2014

  • 30

18

slide-68
SLIDE 68

United Nations EDA DAY Nov 10, 2014

  • 30

18

slide-69
SLIDE 69

United Nations EDA DAY Nov 10, 2014

  • 30

18

slide-70
SLIDE 70

United Nations EDA DAY Nov 10, 2014

  • 30

19

http://bit.ly/pisa2012_explorer

TVs in household (blue) against study place by correlation with math score

slide-71
SLIDE 71

United Nations EDA DAY Nov 10, 2014

  • 30

19

http://bit.ly/pisa2012_explorer

slide-72
SLIDE 72

United Nations EDA DAY Nov 10, 2014

  • 30

19

http://bit.ly/pisa2012_explorer

slide-73
SLIDE 73

United Nations EDA DAY Nov 10, 2014

  • 30

19

http://bit.ly/pisa2012_explorer

slide-74
SLIDE 74

United Nations EDA DAY Nov 10, 2014

  • 30

19

http://bit.ly/pisa2012_explorer

slide-75
SLIDE 75

United Nations EDA DAY Nov 10, 2014

  • 30

19

http://bit.ly/pisa2012_explorer

slide-76
SLIDE 76

United Nations EDA DAY Nov 10, 2014

  • 30

EDA & Inference

I have just reported a huge number of conclusions based purely on exploratory methods. Do you believe them? Exploratory and inferential ARE NOT mutually exclusive. I could have showed you the exploratory statistics in the context of NOTHING HAPPENING. From this we could have obtained statistical significance

  • r lack of it.

20

slide-77
SLIDE 77

United Nations EDA DAY Nov 10, 2014

  • 30

21

Here is the math gap exploration placed in the CONTEXT of there being NO MATH GAP …

slide-78
SLIDE 78

United Nations EDA DAY Nov 10, 2014

  • 30

22

a b c d e f g h

Pick the one that is different from the others

slide-79
SLIDE 79

United Nations EDA DAY Nov 10, 2014

  • 30

22

a b c d e f g h

slide-80
SLIDE 80

United Nations EDA DAY Nov 10, 2014

  • 30

23

a b c d e f g h

Did you pick this one?

slide-81
SLIDE 81

United Nations EDA DAY Nov 10, 2014

  • 30

23

a b c d e f g h

slide-82
SLIDE 82

United Nations EDA DAY Nov 10, 2014

  • 30

EDA & Inference

If the plot that is picked is the plot of the real data, this is statistical significance, and a p-value can be placed on the discovery. Buja et al (2009) RSPT A (econ eg) Wickham et al (2010) InfoVis/TVCG Hofmann et al (2012) InfoVis/TVCG Majumder et al (2013) JASA Roy Chowdhury et al (2014) Comput. Stat. Zhao et al (2014) IJITAS

24

slide-83
SLIDE 83

United Nations EDA DAY Nov 10, 2014

  • 30

Nulls by permutation

Hold country fixed (subset by country) Permute the gender labels, so that the math scores are randomly assigned to a boy or girl Recalculate the difference between the means Plot the mean difference by country again

  • 25
slide-84
SLIDE 84

United Nations EDA DAY Nov 10, 2014

  • 30

Mapping data to plots

Making good plots of data is not straight forward! Tiere are more bad data plots than good. Learning how to effectively plot data should be in CORE statistics/data science curricula

26

slide-85
SLIDE 85

United Nations EDA DAY Nov 10, 2014

  • 30

Reproducibility

Being able to reproduce the analysis is critically important. I used RStudio, with knitr and Rmarkdown to embed analysis with documentation. Tiis is what I use in my statistics labs today, so that students learn to document their analysis, and enable others to see what they did to the data, and re-run the analysis themselves.

27

slide-86
SLIDE 86

United Nations EDA DAY Nov 10, 2014

  • 30

28

“Show the data."

slide-87
SLIDE 87

United Nations EDA DAY Nov 10, 2014

  • 30

What we learned

Gender gap is not universal in math. Gender gap is universal in reading. Individual differences trump everything else. Parents matter! Mothers more than fathers! Socioeconomic status matters. Turn the TV off, if you live in a developed country! Something’s funny about Albania’s data.

29

slide-88
SLIDE 88

United Nations EDA DAY Nov 10, 2014

  • 30

Tools

ggplot2, by Hadley Wickham and the RStudio team (look for their interactive web graphics ggvis soon!) knitr, by Yihui Xie and the RStudio team. So good and easy now that first year statistics students can use for their homeworks and labs! shiny, by RStudio, for the interactive web app.

30