Sampling and Representativeness Department of Government London - - PowerPoint PPT Presentation

▶
sampling and representativeness
SMART_READER_LITE
LIVE PREVIEW

Sampling and Representativeness Department of Government London - - PowerPoint PPT Presentation

Jan 10, 2023 336 likes •964 views

Representativeness Design-based (Statistical) Sampling Sampling and Representativeness Department of Government London School of Economics and Political Science Representativeness Design-based (Statistical) Sampling 1 Representativeness 2

slide-1
SLIDE 1

Representativeness Design-based (Statistical) Sampling

Sampling and Representativeness

Department of Government London School of Economics and Political Science

slide-2
SLIDE 2

Representativeness Design-based (Statistical) Sampling

1 Representativeness 2 Design-based (Statistical) Sampling

slide-3
SLIDE 3

Representativeness Design-based (Statistical) Sampling

1 Representativeness 2 Design-based (Statistical) Sampling

slide-4
SLIDE 4

Representativeness Design-based (Statistical) Sampling

Case selection

Our ambitions about what kind of inferences we want to derive from our descriptions influence how we select cases.

slide-5
SLIDE 5

Representativeness Design-based (Statistical) Sampling

Case selection

Our ambitions about what kind of inferences we want to derive from our descriptions influence how we select cases. Purposive

slide-6
SLIDE 6

Representativeness Design-based (Statistical) Sampling

Case selection

Our ambitions about what kind of inferences we want to derive from our descriptions influence how we select cases. Purposive Comparative

slide-7
SLIDE 7

Representativeness Design-based (Statistical) Sampling

Case selection

Our ambitions about what kind of inferences we want to derive from our descriptions influence how we select cases. Purposive Comparative Representative

slide-8
SLIDE 8

Representativeness Design-based (Statistical) Sampling

Case selection

Our ambitions about what kind of inferences we want to derive from our descriptions influence how we select cases. Purposive Comparative Representative

Unrepresentative

slide-9
SLIDE 9

Representativeness Design-based (Statistical) Sampling

Population

“The complete population of units (observations) we want to understand.”

slide-10
SLIDE 10

Representativeness Design-based (Statistical) Sampling

Population

“The complete population of units (observations) we want to understand.” We rarely observe all population units

slide-11
SLIDE 11

Representativeness Design-based (Statistical) Sampling

Population

“The complete population of units (observations) we want to understand.” We rarely observe all population units A “sample” is a set of units we actually

  • bserve
slide-12
SLIDE 12

Representativeness Design-based (Statistical) Sampling

Population

“The complete population of units (observations) we want to understand.” We rarely observe all population units A “sample” is a set of units we actually

  • bserve

Sometimes we aim to generalize from the sample to the population

slide-13
SLIDE 13

Representativeness Design-based (Statistical) Sampling

Discuss in Pairs! What does it mean for a “sample” (set of cases) to be representative of a population?

slide-14
SLIDE 14

Representativeness Design-based (Statistical) Sampling

Different conceptualizations

Design-based: A sample is representative because of how it was drawn (e.g., randomly) Model-based: A sample is representative because it resembles in the population with respect to certain variables (e.g., same proportion of women in sample and population, etc.) Expert judgement: A sample is representative as judged by an expert who deems it “fit for purpose”

slide-15
SLIDE 15

Representativeness Design-based (Statistical) Sampling

Obtaining Representativeness

slide-16
SLIDE 16

Representativeness Design-based (Statistical) Sampling

Obtaining Representativeness

Census

slide-17
SLIDE 17

Representativeness Design-based (Statistical) Sampling

Obtaining Representativeness

Census Convenience/Purposive samples

slide-18
SLIDE 18

Representativeness Design-based (Statistical) Sampling

Obtaining Representativeness

Census Convenience/Purposive samples Quota sampling (pre-1940s, post-2000s)

slide-19
SLIDE 19

Representativeness Design-based (Statistical) Sampling

Obtaining Representativeness

Census Convenience/Purposive samples Quota sampling (pre-1940s, post-2000s) Simple random sampling

slide-20
SLIDE 20

Representativeness Design-based (Statistical) Sampling

Obtaining Representativeness

Census Convenience/Purposive samples Quota sampling (pre-1940s, post-2000s) Simple random sampling Complex survey designs

slide-21
SLIDE 21

Representativeness Design-based (Statistical) Sampling

Obtaining Representativeness

Census Convenience/Purposive samples Quota sampling (pre-1940s, post-2000s) Simple random sampling Complex survey designs

slide-22
SLIDE 22

Representativeness Design-based (Statistical) Sampling

1 Representativeness 2 Design-based (Statistical) Sampling

slide-23
SLIDE 23

Representativeness Design-based (Statistical) Sampling

Inference from Sample to Population We want to know pop. parameter θ We only observe sample estimate ˆ θ We have a guess but are also uncertain

slide-24
SLIDE 24

Representativeness Design-based (Statistical) Sampling

Inference from Sample to Population We want to know pop. parameter θ We only observe sample estimate ˆ θ We have a guess but are also uncertain What range of values for θ does our ˆ θ imply?

slide-25
SLIDE 25

Representativeness Design-based (Statistical) Sampling

Simple Random Sampling

1 Define target population 2 Create “sampling frame” 3 Each unit in frame has equal probability

  • f selection

4 Collect data on each unit 5 Calculate sample statistic 6 Draw an inference to the population

slide-26
SLIDE 26

Representativeness Design-based (Statistical) Sampling

Population

slide-27
SLIDE 27

Representativeness Design-based (Statistical) Sampling

Population Sampling Frame

slide-28
SLIDE 28

Representativeness Design-based (Statistical) Sampling

Population Sampling Frame Sample

slide-29
SLIDE 29

Representativeness Design-based (Statistical) Sampling

Population Sampling Frame Sample Sample Sample Sample Sample Sample Sample Sample Sample

slide-30
SLIDE 30

Representativeness Design-based (Statistical) Sampling

Population Sampling Frame Sample Sample Sample Sample Sample Sample Sample Sample Sample Sample

slide-31
SLIDE 31

Representativeness Design-based (Statistical) Sampling

Simple Random Sampling

1 Define target population 2 Create “sampling frame” 3 Each unit in frame has equal probability

  • f selection

4 Collect data on each unit 5 Calculate sample statistic 6 Draw an inference to the population

slide-32
SLIDE 32

Representativeness Design-based (Statistical) Sampling

Statistical Inference I

To calculate a sample mean (or proportion): ¯ y = 1 n

n

  • i=1 yi

(1)

where yi = value for a unit, and n = sample size

slide-33
SLIDE 33

Representativeness Design-based (Statistical) Sampling

Statistical Inference II

If we calculate ¯ y in our sample, what does this tell us about the ¯ Y in the population?

slide-34
SLIDE 34

Representativeness Design-based (Statistical) Sampling

Statistical Inference II

If we calculate ¯ y in our sample, what does this tell us about the ¯ Y in the population? The sample estimate is our guess at the value of the population parameter within some degree of uncertainty

slide-35
SLIDE 35

Representativeness Design-based (Statistical) Sampling

Law of Large Numbers

Definition: The mean of the ˆ θ from each of a number of samples will converge on the population θ, as the number of samples increases

slide-36
SLIDE 36

Representativeness Design-based (Statistical) Sampling

Sampling Variance

The ˆ θ in any particular sample can differ from the population value θ This variation is calling “sampling variance” or “sampling error” The standard error describes the average amount of variation of the ˆ θ’s around θ

slide-37
SLIDE 37

Representativeness Design-based (Statistical) Sampling

How Uncertain Are We?

Our uncertainty depends on sampling procedures Most importantly, sample size

As n → ∞, uncertainty → 0

We typically summarize our uncertainty as the standard error

slide-38
SLIDE 38

Representativeness Design-based (Statistical) Sampling

Standard Errors (SEs)

Definition: “The standard error of a sample estimate is the average distance that a sample estimate (ˆ θ) would be from the population parameter (θ) if we drew many separate random samples and applied our estimator to each.”

slide-39
SLIDE 39

Representativeness Design-based (Statistical) Sampling

Standard Errors (SEs)

Definition: “The standard error of a sample estimate is the average distance that a sample estimate (ˆ θ) would be from the population parameter (θ) if we drew many separate random samples and applied our estimator to each.” Square root of the sampling variance

slide-40
SLIDE 40

Representativeness Design-based (Statistical) Sampling

Sample mean

¯ y = 1 n

n

  • i=1

yi (2) where yi = value for a unit, and n = sample size SE¯

y =

  • (1 − f )s2

n (3) where f = proportion of population sampled, s2 = sample (element) variance, and n = sample size

slide-41
SLIDE 41

Representativeness Design-based (Statistical) Sampling

Sample proportion

Pr(y = 1) = 1 n

n

  • i=1

yi (4) where yi = value for a unit, and n = sample size SEp =

  • (1 − f )p(1 − p)

n (5) where f = proportion of population sampled, p = sample proportion, and n = sample size

slide-42
SLIDE 42

Representativeness Design-based (Statistical) Sampling

Margin of Error

Uncertainty often stated in terms of a “margin of error” Standard MoE is twice the SE (x1.96) For estimated proportions, expressed as: “p ± MoE percentage points”

slide-43
SLIDE 43

Representativeness Design-based (Statistical) Sampling Source: http://www.abc.net.au/news/2016-01-17/ new-poll-show-widening-support-for-uk-to-leave-eu/7093730

slide-44
SLIDE 44

Representativeness Design-based (Statistical) Sampling Source: http://www.abc.net.au/news/2016-01-17/ new-poll-show-widening-support-for-uk-to-leave-eu/7093730

slide-45
SLIDE 45

Representativeness Design-based (Statistical) Sampling

Questions?

slide-46
SLIDE 46

Representativeness Design-based (Statistical) Sampling

Questions?

(There is an R lab activity about this.)

slide-47
SLIDE 47

Representativeness Design-based (Statistical) Sampling

Activity!

Source: ganeshaisis on Wikimedia

slide-48
SLIDE 48

Representativeness Design-based (Statistical) Sampling

What proportion of all Haribo Starmix gummies are ♥s?

slide-49
SLIDE 49

Representativeness Design-based (Statistical) Sampling

What proportion of all Haribo Starmix gummies are ♥s?

1 Everyone collect a random sample

slide-50
SLIDE 50

Representativeness Design-based (Statistical) Sampling

What proportion of all Haribo Starmix gummies are ♥s?

1 Everyone collect a random sample 2 Calculate ˆ

p =

♥

n

slide-51
SLIDE 51

Representativeness Design-based (Statistical) Sampling

What proportion of all Haribo Starmix gummies are ♥s?

1 Everyone collect a random sample 2 Calculate ˆ

p =

♥

n

3 Calculate element variance:

Var(x) = p(1 − p)

slide-52
SLIDE 52

Representativeness Design-based (Statistical) Sampling

What proportion of all Haribo Starmix gummies are ♥s?

1 Everyone collect a random sample 2 Calculate ˆ

p =

♥

n

3 Calculate element variance:

Var(x) = p(1 − p)

4 Calculate MoE: ˆ

p ±

   2 ∗

  • Var(x)

n

   

slide-53
SLIDE 53

Representativeness Design-based (Statistical) Sampling

slide-54
SLIDE 54

Representativeness Design-based (Statistical) Sampling

How large of a sample do we need?

1Population element variance is estimated by sample element variance.

slide-55
SLIDE 55

Representativeness Design-based (Statistical) Sampling

How large of a sample do we need? Uncertainty is influenced by:

Sample size Element variance1 Population size?

1Population element variance is estimated by sample element variance.

slide-56
SLIDE 56

Representativeness Design-based (Statistical) Sampling

How large of a sample do we need? Uncertainty is influenced by:

Sample size Element variance1 Population size?

So what do we do?

Decide on desired uncertainty Guess at element variance

1Population element variance is estimated by sample element variance.

slide-57
SLIDE 57

Representativeness Design-based (Statistical) Sampling

How large of a sample do we need? Uncertainty is influenced by:

Sample size Element variance1 Population size?

So what do we do?

Decide on desired uncertainty Guess at element variance Adjust sample size based on feasibility

1Population element variance is estimated by sample element variance.

slide-58
SLIDE 58

Representativeness Design-based (Statistical) Sampling

Estimating sample size

What precision (margin of error) do we want? ±2 percentage points: SE = 0.01 n = 0.25 0.012 = 0.25 0.0001 = 2500 (6)

slide-59
SLIDE 59

Representativeness Design-based (Statistical) Sampling

Estimating sample size

What precision (margin of error) do we want? ±2 percentage points: SE = 0.01 n = 0.25 0.012 = 0.25 0.0001 = 2500 (6) ±5 percentage points: SE = 0.025 n = 0.25 0.000625 = 400 (7)

slide-60
SLIDE 60

Representativeness Design-based (Statistical) Sampling

Estimating sample size

What precision (margin of error) do we want? ±2 percentage points: SE = 0.01 n = 0.25 0.012 = 0.25 0.0001 = 2500 (6) ±5 percentage points: SE = 0.025 n = 0.25 0.000625 = 400 (7) ±0.5 percentage points: SE = 0.0025 n = 0.25 0.00000625 = 40, 000 (8)

slide-61
SLIDE 61

Representativeness Design-based (Statistical) Sampling

Summary

Various ways to select cases Various notions of representativeness Case selection is one way of addressing questions of “external validity” but there are others, too

slide-62
SLIDE 62