The Effects of Raking and Cell Phone Integration on BRFSS Outcomes - - PDF document

▶
the effects of raking and cell phone integration on brfss
SMART_READER_LITE
LIVE PREVIEW

The Effects of Raking and Cell Phone Integration on BRFSS Outcomes - - PDF document

Sep 03, 2023 284 likes •725 views

4/24/2012 The Effects of Raking and Cell Phone Integration on BRFSS Outcomes Machell Town, M.S. Carol Pierannunzi, Ph.D. Division of Behavioral Surveillance Office of Surveillance, Epidemiology, and Laboratory Services Division of Behavioral

slide-1
SLIDE 1

4/24/2012 1

The Effects of Raking and Cell Phone Integration

  • n BRFSS Outcomes

Machell Town, M.S. Carol Pierannunzi, Ph.D.

Division of Behavioral Surveillance

Office of Surveillance, Epidemiology, and Laboratory Services Division of Behavioral Surveillance

slide-2
SLIDE 2

4/24/2012 2

Brief Agenda

 Weighting procedures

  • Design weights
  • Post stratification
  • Iterative proportional fitting

 Why change weighting procedures now?

  • Cell phone
  • Computer capacity

 Impact of changes on estimation

  • BRFSS
  • Examples of small and large impact
  • Changes when cell phones are incorporated

 Conclusions  Brief look at state level phone use data (preliminary)

slide-3
SLIDE 3

4/24/2012 3

WEIGHTING PROCEDURES

slide-4
SLIDE 4

4/24/2012 4

Design and GeoStrata Weighting

 Takes into account the geographic region/strata

  • f the sample.

 Design weight uses number of adults in

household and number of phones in household for landline sample.

 BRFSS landline sample is drawn using low/high

density strata within each of the regions (usually smaller than states)

 Stratum weight (_STRWT) = NRECSTR/ NRECSEL

slide-5
SLIDE 5

4/24/2012 5

Calculating the Design Weight

 Design Weight = _STRWT* (1/NUMPHON2) *

NUMADULT

  • NUMPHON2= number of phones within the household
  • NUMADULT = number of adults eligible for the survey within the

household

 Questions for the design weights are asked in

screening questions and in demographic sections of the survey

slide-6
SLIDE 6

4/24/2012 6

Data Weighting

 Data weights take the design weighting and

incorporate characteristics of the population and the sample

 Final Weights (_FINALWT) = Design Weight * some form

  • f data weighting
  • In past BRFSS used post stratification
  • In 2008 Iterative Proportional Fitting was first used
  • In 2011 Iterative Proportional Fitting will be only method of data

weighting for BRFSS

slide-7
SLIDE 7

4/24/2012 7

WEIGHTING

Post -Stratification

slide-8
SLIDE 8

4/24/2012 8

Where We Have Been--- Post Stratification

 Post Stratification is based on known demographics of

the population.

  • For BRFSS Post stratification included:

· Regions within states · Race/ Ethnicity (in detailed categories) · Gender · Age (in 7 categories)

 Post-stratification forces the sum of the weighted

frequencies to equal the population estimates for the region or state by race, age ,and gender.

 Post stratification weights are applied to the responses,

allowing for estimates of how groups of non- respondents would have answered survey questions.

slide-9
SLIDE 9

4/24/2012 9

Post-stratification

 Post-stratification Adjustment Factor is calculated for

each race/ethnicity, gender, and age group combination.

  • Requires knowledge of each subset of each factor at the

geographic level of interest –otherwise categories must be collapsed

  • Requires a minimum number of persons in each cell—otherwise

categories must be collapsed

 _POSTSTR = Population/Design weight within the

weighting class cell.

slide-10
SLIDE 10

4/24/2012 10

Weight Trimming

 Sometimes post-

stratification resulted in very small or disproportionately large weights within age/race/gender/region categories.

 Weight trimming or category

collapsing would be done if categories were disproportionately large or too small (< 50 responses).

slide-11
SLIDE 11

4/24/2012 11

WEIGHTING

Iterative Proportional Fitting (Raking)

slide-12
SLIDE 12

4/24/2012 12

Iterative Proportional Fitting

Rather than adjusting weights to categories, IPF adjusts for each dimension separately in an iterative process. The process will continue up to 75 times,

  • r until data

converges to Census estimates.

Region Age Race Gender Phone Type Home Ownership Education Marital Status Gender by Race Age by Gender Age by Race

slide-13
SLIDE 13

4/24/2012 13

New Variables Introduced as Controls With IPF

 Education  Marital status  Home ownership/renter  Telephone source (cell phone or

landline)

slide-14
SLIDE 14

4/24/2012 14

Post Stratification vs. Iterative Proportional Fitting

Post Stratification Iterative Proportional Fitting

Operates with less computer time Allows for incorporation of new variables. Allows for incorporation of cell phone data. Seems to more accurately represent population data (reduces bias).

slide-15
SLIDE 15

4/24/2012 15

Why Incorporate IPF Now?

 Computer capacity has increased.  Cell phones are becoming larger percentage of the

total number of calls.

 Noncoverage with declining response rates makes

weighting more important than ever.

slide-16
SLIDE 16

4/24/2012 16

Examples of IPF From 2010 Data

 Note that example may be slightly different from 2011

analyses because

  • We did not collect home ownership at that time
  • We still used phone interruption variable
  • Some of the iterations are different than will be conducted on

2011 dataset

slide-17
SLIDE 17

4/24/2012 17

Raking – Iteration 1

17

First Control Variable Output Weight Sum of Weights Target Total Sum of Weights Difference % of Output Weights Target % of Weights Difference in % Age 18-24,Male 87122.60 95468

  • 8345.40

6.533 7.159

  • 0.626

Age 18-24,Female 77180.40 90249

  • 13068.60

5.788 6.768

  • 0.980

Age 25-34,Male 109419.36 118670

  • 9250.64

8.206 8.899

  • 0.694

Age 25-34,Female 114395.17 112007 2388.17 8.579 8.400 0.179 Age 35-44,Male 121328.71 117184 4144.71 9.099 8.788 0.311 Age 35-44,Female 115609.98 113779 1830.98 8.670 8.533 0.137 Age 45-54,Male 138658.26 127077 11581.26 10.398 9.530 0.869 Age 45-54,Female 136904.33 127439 9465.33 10.267 9.557 0.710 Age 55-64,Male 90338.77 95032

  • 4693.23

6.775 7.127

  • 0.352

Age 55-64,Female 91693.43 97422

  • 5728.57

6.876 7.306

  • 0.430

Age 65-74,Male 57475.54 54171 3304.54 4.310 4.062 0.248 Age 65-74,Female 62709.50 61828 881.50 4.703 4.637 0.066 Age 75+,Male 49772.58 46515 3257.58 3.733 3.488 0.244 Age 75+,Female 80867.37 76635 4232.37 6.064 5.747 0.317

Should be │.025│

  • r less
slide-18
SLIDE 18

4/24/2012 18

Raking – Iteration 1

18

Second Control Variable Output Weight Sum

  • f Weights

Target Total Sum of Weights Difference % of Output Weights Target %

  • f

Weights Difference in % WH NH 1151321.16 1156947

  • 5625.84

86.340 86.762

  • 0.422

OT NH 15305.42 12036 3269.42 1.148 0.903 0.245 HISP 85300.51 84230 1070.51 6.397 6.317 0.080 BL NH,AS NH,AI NH 81548.91 80263 1285.91 6.116 6.019 0.096

slide-19
SLIDE 19

4/24/2012 19

Raking - Iteration 1

Third Control Variable Input Weight Sum

  • f Weights

Target Total Sum of Weights Difference % of Input Weights Target %

  • f Weights

Difference in % Less than HS 89962.05 143928

  • 53966.35

6.746 10.793

  • 4.047

HS Grad 412857.99 414505

  • 1646.81

30.961 31.085

  • 0.123

Some College 388163.96 448218

  • 60054.20

29.109 33.613

  • 4.504

College Grad 442492.00 326825 115667.37 33.183 24.509 8.674

19

slide-20
SLIDE 20

4/24/2012 20

Raking – Iteration 1

20

Fourth Control Variable Output Weight Sum of Weights Target Total Sum of Weights Difference % of Output Weights Target % of Weights Difference in % Married 816399.38 792326 24073.29 61.223 59.418 1.805 Never married, member unmarried couple 277180.73 300111

  • 22930.01

20.786 22.506

  • 1.720

Divorced, Widowed, Separated 239895.88 241039

  • 1143.29

17.990 18.076

  • 0.086

Fifth Control Variable Output Weight Sum

  • f Weights

Target Total Sum of Weights Difference % of Output Weights Target % of Weights Difference in % Phone interruption 78558.62 82944

  • 4385.49

5.891 6.220

  • 0.329

No Phone Interruption 1254917.38 1250532 4385.49 94.109 93.780 0.329

slide-21
SLIDE 21

4/24/2012 21

Raking – Iteration 1

21

Sixth Control Variable Output Weight Sum of Weights Target Total Sum of Weights Difference % of Output Weights Target % of Weights Difference in % Male, WH NH 553107.34 552171 936.34 41.479 41.408 0.070 Male, BL NH,AS NH,AI NH,OT NH,HISP 101008.49 101946

  • 937.51

7.575 7.645

  • 0.070

Female, WH NH 598213.82 604776

  • 6562.18

44.861 45.353

  • 0.492

Female, HISP 38304.69 32837 5467.69 2.873 2.463 0.410 Female, BL NH,AS NH,AI NH,OT NH 42841.66 41746 1095.66 3.213 3.131 0.082

slide-22
SLIDE 22

4/24/2012 22

Raking – Iteration 1

22

Seventh Control Variable Output Weight Sum of Weights Target Total Sum of Weights Difference % of Output Weights Target % of Weights Difference in % 18-34, WH NH 308020.95 332809

  • 24788.05

23.099 24.958

  • 1.859

18-34, BL NH,AS NH,AI NH,OT NH,HISP 80096.58 83585

  • 3488.42

6.007 6.268

  • 0.262

35-54, WH NH 442299.71 421539 20760.71 33.169 31.612 1.557 35-54, BL NH,AS NH,AI NH,OT NH,HISP 70201.57 63940 6261.57 5.265 4.795 0.470 55+, WH NH 401000.50 402599

  • 1598.50

30.072 30.192

  • 0.120

55+, BL NH,AS NH,AI NH,OT NH,HISP 31856.70 29004 2852.70 2.389 2.175 0.214

slide-23
SLIDE 23

4/24/2012 23

Raking – Iteration 1

23

Eighth Control Variable Output Weight Sum of Weights Target Total Sum of Weights Difference % of Output Weights Target % of Weights Difference in % Cell Phone Only 210390.11 197088 13302.35 15.778 14.780 0.998 Landline Only 270206.34 280297

  • 10090.31

20.263 21.020

  • 0.757

Landline and Cell Phone 852879.55 856092

  • 3212.04

63.959 64.200

  • 0.241
slide-24
SLIDE 24

4/24/2012 24

Raking – Iteration 2

24

First Control Variable Output Weight Sum of Weights Target Total % of Output Weights Target %

  • f Weights

Difference in % from Iteration1 Difference in % Age 18-24,Male 94727.80 95468 7.104 7.159

  • 0.626
  • 0.056

Age 18-24,Female 87222.36 90249 6.541 6.768

  • 0.980
  • 0.227

Age 25-34,Male 116312.81 118670 8.723 8.899

  • 0.694
  • 0.177

Age 25-34,Female 110348.83 112007 8.275 8.400 0.179

  • 0.124

Age 35-44,Male 118670.65 117184 8.899 8.788 0.311 0.111 Age 35-44,Female 113723.15 113779 8.528 8.533 0.137

  • 0.004

Age 45-54,Male 130207.90 127077 9.765 9.530 0.869 0.235 Age 45-54,Female 130419.01 127439 9.780 9.557 0.710 0.223 Age 55-64,Male 93001.49 95032 6.974 7.127

  • 0.352
  • 0.152

Age 55-64,Female 96092.37 97422 7.206 7.306

  • 0.430
  • 0.100

Age 65-74,Male 54156.67 54171 4.061 4.062 0.248

  • 0.001

Age 65-74,Female 62303.45 61828 4.672 4.637 0.066 0.036 Age 75+,Male 47039.67 46515 3.528 3.488 0.244 0.039 Age 75+,Female 79249.83 76635 5.943 5.747 0.317 0.196

slide-25
SLIDE 25

4/24/2012 25

Raking - Iteration 7

First Control Variable Output Weight Sum of Weights Target Total % of Output Weights Target % of Weights Difference in % from Iteration1 Difference in % Age 18-24,Male 95491.87 95468 7.161 7.159

  • 0.626

0.002 Age 18-24,Female 90265.83 90249 6.769 6.768

  • 0.980

0.001 Age 25-34,Male 118621.93 118670 8.896 8.899

  • 0.694
  • 0.004

Age 25-34,Female 111985.21 112007 8.398 8.400 0.179

  • 0.002

Age 35-44,Male 117205.13 117184 8.789 8.788 0.311 0.002 Age 35-44,Female 113769.71 113779 8.532 8.533 0.137

  • 0.001

Age 45-54,Male 127088.93 127077 9.531 9.530 0.869 0.001 Age 45-54,Female 127437.46 127439 9.557 9.557 0.710

  • 0.000

Age 55-64,Male 95037.18 95032 7.127 7.127

  • 0.352

0.000 Age 55-64,Female 97426.08 97422 7.306 7.306

  • 0.430

0.000 Age 65-74,Male 54168.73 54171 4.062 4.062 0.248

  • 0.000

Age 65-74,Female 61831.76 61828 4.637 4.637 0.066 0.000 Age 75+,Male 46503.23 46515 3.487 3.488 0.244

  • 0.001

Age 75+,Female 76642.96 76635 5.748 5.747 0.317 0.001

25 All less than │.025│

slide-26
SLIDE 26

4/24/2012 26

Raking - Iteration 7

Eighth Control Variable Output Weight Sum

  • f Weights

Target Total % of Output Weights Target %

  • f

Weights Difference in % at Iteration 1 Difference in % Cell Phone Only 197101.32 197088 14.781 14.780 0.998 0.001 Landline Only 280285.25 280297 21.019 21.020

  • 0.757
  • 0.001

Landline and Cell Phone 856089.43 856092 64.200 64.200

  • 0.241
  • 0.000

26

**** Program terminated at iteration 7 because all current percents differ from target percents by less than 0.025*****

slide-27
SLIDE 27

4/24/2012 27

IMPACT OF CHANGING TO RAKING (IPV) ON THE BRFSS

slide-28
SLIDE 28

4/24/2012 28

BRFSS 2010 Combined States a Data Difference In Weighted Percentages

A Excludes AK, DC, TN, SD

10 20 30 40 50 60 70 80 LL Post stratified LL Raking LLCP Raking

slide-29
SLIDE 29

4/24/2012 29

Marginal Changes Weighted Percentages for Demographic Characteristics, BRFSS 2010

  • 0.08
  • 0.06
  • 0.04
  • 0.02

0.02 0.04 0.06 0.08 0.1

slide-30
SLIDE 30

4/24/2012 30

BRFSS 2010 Combined States Data Difference In Weighted Percentages of Health Outcomes

5 10 15 20 25 30 35

LL Post stratified LL Raking LLCP Raking A Excludes AK, DC, TN, SD

slide-31
SLIDE 31

4/24/2012 31

Marginal Changes for in Weighted Percentage s Health Outcomes, BRFSS 2010

0.2 0.4 0.6 0.8 1 1.2

slide-32
SLIDE 32

4/24/2012 32

Weighted Prevalence Estimates for Current Smoker by Year, Weighting Method

5 10 15 20 25 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 Prevalence Estimate Year

Landline Post Stratification Landline Raking Weighting Landline/ Cell Phone Raking Weighting

NOTE: All US states and territories except SD and TN

slide-33
SLIDE 33

4/24/2012 33

STATE LEVEL OUTCOMES

slide-34
SLIDE 34

4/24/2012 34

In Some Cases, Small Changes (Landline Only)

Table 1 State-level Responses to Question: ―Has a doctor, nurse or other healthcare provider ever told you that you have diabetes?‖ By Type Of Weighting Procedure for Landline Data Response Landline Weighted frequency with Post- Stratification Landline Percent With Post- Stratification Landline Weighted frequency with Raking Landline Percent With Raking Differences in Landline Percentages (Post- Stratification- Raking) Yes

434,858 12.26 440,694 12.43

  • 0.17

Yes, but only during pregnancy

26,306 0.74 26,262 0.74 0.00

No

3,031,681 85.44 3,029,545 85.42 0.02

No, Pre-diabetes/ borderline diabetes

55,454 1.56 50,196 1.42 0.15

slide-35
SLIDE 35

4/24/2012 35

In Some Cases, Larger Differences– But Not Consistent Differences (Landline Only)

Table 2 State-level Responses to Question: ―Would you say that in general your health is excellent, very good, good, fair or poor?‖ By Type Of Weighting Procedure for Landline Data

Response

Landline Weighted Frequency With Post- Stratification Landline Percent With Post- Stratification Landline Weighted Frequency With Raking Landline Percent With Raking Differences In Landline Percentages (Post-Stratification

  • Raking)

Excellent 631,742 17.83 575,541 16.27 1.56 Very Good 1,037,345 29.27 963,330 27.23 2.04 Good 1,107,272 31.26 1,111,484 31.42

  • 0.16

Fair 519,248 14.65 591,716 16.73

  • 2.07

Poor 247,424 6.98 295,425 8.35

  • 1.37
slide-36
SLIDE 36

4/24/2012 36

In Some Cases, Consistent Differences (Landline Only)

Table 3 State-level Responses to Question: ―During the past month, other than your regular job, did you participate in any physical activities or exercises such as running, calisthenics, golf, gardening, or walking for exercise?‖ By Type Of Weighting Procedure for Landline Data

Response

Landline Weighted Frequency With Post- Stratification Landline Percent With Post- Stratification Landline Weighted Frequency With Raking Landline Percent With Raking Differences In Landline Percentages (Post-Stratification

  • Raking)

Yes 2,448,288 68.97 2,342,381 65.98 2.99 No 1,101,378 31.03 1,207,643 34.02

  • 2.99
slide-37
SLIDE 37

4/24/2012 37

But Differences Go Away Sometimes When Cell Phones Are Added

Table 4 State-level Responses to Question: ―During the past month, other than your regular job, did you participate in any physical activities or exercises such as running, calisthenics, golf, gardening, or walking for exercise?‖ By Type Of Weighting Procedure for Landline and Cell Phone Data

Response Landline Weighted Frequency With Post- Stratification Landline Percent With Post- Stratification Landline Weighted Frequency With Raking Landline Percent With Raking Differences In Landline Percentages (Post- Stratification - Raking) Landline And Cell Phone Weighted Frequency With Raking Landline And Cell Phone Percent Landline And Cell Phone Differences In Percentages (Post- Stratification - Raking)

Yes 2,448,288 68.97 2,342,381 65.98 2.99 2,447,823 68.96 0.02 No 1,101,378 31.03 1,207,643 34.02

  • 2.99 1,102,053

31.04

  • 0.02
slide-38
SLIDE 38

4/24/2012 38

Persistent Differences May Exist Even When Adding Cell Phone Responses

Table 5 State-level Responses to Question: ―Do you smoke cigarettes every day, some days or not at all?‖ By Type Of Weighting Procedure for Landline and Cell Phone Data

Response Landline Weighted Frequency With Post- Stratification Landline Percent With Post- Stratification Landline Weighted Frequency With Raking Landline Percent With Raking Differences In Landline Percentages (Post- Stratification - Raking) Landline And Cell Phone Weighted Frequency With Raking Landline And Cell Phone Percent Landline And Cell Phone Differences In Percentages (Post- Stratification - Raking)

Every day 581,967 36.32 704,831 40.95

  • 4.63

676,129 40.40

  • 4.08

Some Days 213,724 13.34 248,782 14.45

  • 1.12

199,278 11.91 1.43 Not At All 806,827 50.35 767,708 44.60 5.75 798,181 47.69 2.65

slide-39
SLIDE 39

4/24/2012 39

CONCLUSIONS

slide-40
SLIDE 40

4/24/2012 40

Conclusions (1)

 New weighting procedures are needed to keep pace

with changes in personal communications.

 The inclusion of new variables and more complex

weighting procedures for large scale survey data are now feasible, because of improvements in computer capacity.

 There will be some differences in estimates when

weighting procedures change and when new variables for weighting are introduced.

 Examples shown here are only depictions of potential

  • utcomes of changes at the BRFSS.
slide-41
SLIDE 41

4/24/2012 41

Conclusions (2)

 Good news: demographic characteristics adjusted to

more closely match Census data.

 Most health outcomes indicate increases in risk

behaviors (especially when state data are aggregated).

 Some increases in chronic conditions, but uneven

across states.

slide-42
SLIDE 42

4/24/2012 42

Thank You

For more information please contact Centers for Disease Control and Prevention 1600 Clifton Road NE, Atlanta, GA 30333 Telephone: 1-800-CDC-INFO (232-4636)/TTY: 1-888-232-6348 E-mail: cdcinfo@cdc.gov Web: http://www.cdc.gov

The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention.

Office of Surveillance, Epidemiology, and Laboratory Services Division of Behavioral Surveillance