How Interviewer Characteristics Differ Household Surveys?: An - - PDF document

1

How Interviewer Characteristics Differ Household Surveys?: An Analysis in 2013 Turkey Demographic and Health Survey

Melike Sarac (melikesarac@hacettepe.edu.tr) and A. Sinan Turkyilmaz (aturkyil@hacettepe.edu.tr) Hacettepe University, Institute of Population Studies, Ankara, Turkey

Abstract Information on data quality indicators such as non-response rates and sampling errors have been presented in Turkey Demographic and Health Surveys (TDHS) since 1993 as household sample surveys. Apart from these quality indicators, tables on data quality indicators regarding with the household population, eligible and interviewed women, missing information, and birth and death information are presented on main reports of TDHSs. However, lack of information about interviewer characteristics which is known as a major error source in terms of survey quality still exists. This study mainly emphasizes on interviewer characteristics covered in a special data set, called “Data Collection Staff Data Set”. Descriptive results which summarize recruitment process of TDHS-2013 and a multivariate analysis by the Poisson regression model that provides results on number of completed household interviews consist of main components of this study. An analysis on characteristics of interviewers who were employed in TDHS- 2013 fieldwork demonstrated significant effect on main component of the response rate: number of completed household interviews. As it is seen, number of completed interviews is assumed as an indicator that reflects the interviewer performance for this study in terms of response rate. Demographic characteristics of interviewers such as age and education have been interested in studies analyze interviewer effect on survey responses (Williams, 1964; Berk and Bernstein, 1988; Wilson and Olesen, 2002). Similarly, experience of interviewers has been discussed comprehensively within achieving higher response rates (Durbin and Stuart, 1951; Groves and Couper, 1998; Sala et. al., 2012). Our findings put forward that age, place of birth, experience and education of interviewers have an effect on number of completed household interviews under the control of other covariates.

2

Introduction and Objectives Main objective of a good survey design is to maximize survey quality under the survey

• constraints. This objective should be taken into account in all survey stages in order to

maximize survey quality. Understanding the survey process and considering the total survey error within the sampling and non-sampling errors are crucial in order to evaluate survey quality that is determinable with accuracy, timeliness, accessibility, and completeness. Controlling non-sampling errors, which are not easy to estimate as much as sampling errors, is possible with accurate planning and careful survey design, interactive with knowledge, experiences and theories of many disciplines (Biemer and Lyberg, 2003). Therefore, error types which creates non-sampling error such as measurement error and coverage error should be tried to keep in a minimum level during the survey process. Interviewer who plays a critical role on survey estimates has an effect on respondents and responses directly whereas variance is smaller due to assistance provided by an

• interviewer. This effect can be considered within the non-sampling error considering

the non-response behavior and social desirability. As Korbmacher (2014) mentioned in his study, interviewer has an impact on respondents’ willingness to participate. Initially, contact process with the sample unit and after that persuading respondent to survey participation and maintaining motivation of respondents during the interview are quite substantial issues in terms of gaining cooperation, having an interview with the respondent, and data quality. At the same time, interviewing approach that is provided by an interviewer affects data quality. Some researchers believe that questions to the respondent by interviewer should be asked very standardized manner whereas some researchers believe that the interviewing process is more interactive and flexible (Fowler and Mangione, 1990; Suchman and Jordan, 1990). It should be noted that, not

• nly interviewing techniques but also demographic and other characteristics of

interviewers have an influence on response behavior of respondents and survey estimates. This study mainly focuses on interviewer recruitment process of the TDHS-2013 based

• n the steps and interviewers’ demographic and other characteristics within the context
• f completed household interviews which can be considered as a reflection of

completion and response rates. In this regard, the main research question of the study is determined as: “Are there any impact of data collection staff characteristics on

3

number of completed household interviews in TDHS-2013?”. In this study, household interviews will be examined rather than women interviews due to the fact that the cooperation with respondent has been already taken in women interview thanks to the household interview. Data Source and Methodology In order to analyze the data collection staff characteristics and interview results, two different data sources were used. The first data set is TDHS-2013 Household Data Set, which is a nationally representative data on demographic and socio-economic characteristics of households in Turkey. And, secondly, a specially constructed data set, namely TDHS-2013 Data Collection Staff Data Set, constructed by aggregating ‘application’, ‘interview’, and ‘fieldwork preference’ forms filled by applicants involved in TDHS-2013 recruitment process. A Poisson regression model which is convenient for the number of completed household interviews, as a count variable, was used in order to identify and measure effect of interviewer characteristics on number of completed household interviews as well as estimable field characteristics. Average time of a household interview based

• n interviewer and number of days which spent for household interviews can be

considered within the varying field characteristics depending on the interviewers. The dependent variable, total number of completed interviews for each interviewer, seems to be convenient for Poisson regression analysis considering the non-negative

• integers. The model for the analysis on total number of completed interviews is based
• n the Poisson distribution as the following:

Pr(𝑍 = 𝑍|µ) = 𝑓−µµ𝑧 𝑧! , 𝑧 = 0,1,2, … where µ is the risk of a new occurrence of the interest during a specified time interval that can be assumed data collection process (in days) for this study. Additionally, descriptive results that focus on the recruitment process and household level completion and response rates will be presented based on the TDHS-2013 Household Data Set using the interview results.

4 𝐼𝑝𝑣𝑡𝑓ℎ𝑝𝑚𝑒 𝑆𝑓𝑡𝑞𝑝𝑜𝑡𝑓 𝑆𝑏𝑢𝑓 = 𝐷 𝐷 + 𝐼𝑄 + 𝑄 + 𝑆 + 𝐸𝑂𝐺 + 𝑄𝐷 𝐼𝑝𝑣𝑡𝑓ℎ𝑝𝑚𝑒 𝐷𝑝𝑛𝑞𝑚𝑓𝑢𝑗𝑝𝑜 𝑆𝑏𝑢𝑓 = 𝐷 𝐷 + 𝐼𝑄 + 𝐼𝐵 + 𝑄 + 𝑆 + 𝐸𝑊 + 𝐸𝐸 + 𝐸𝑂𝐺 + 𝑄𝐷 + 𝑃

where categories “completed” (C), “no household member/no competent member at home” (HP), “entire household absent for extended period of time” (HA), “postponed” (P), “refused” (R), “dwelling vacant or address not a dwelling” (DV), “dwelling destroyed” (DD), “dwelling not found” (DNF), “partially completed” (PC), and “other” (O). Firstly, recruitment process of the TDHS-2013 was reviewed and after that the impact

• f the interviewer characteristics such as age, place of birth, educational status,

working status, interested in social science, survey experience, language abilities and field characteristics such as having an interview in metropolitan cities of Turkey, mean number of household members, and average time of a household interview were examined in Poisson regression model. Findings Recruitment steps and number of candidates who take part in each step are demonstrated in Table 1. As it is seen in Table 1 multi-stage recruitment process was conducted in TDHS-2013 recruitment process. As we have mentioned previously, interviewer specific completion / response rates, which is calculated with the total number of completed interviews, can be considered as one of the performance indicators of data collection staff. In total, interviews with 11749 out of 14489 selected households were completed. Completion rates should be interpreted carefully as they are affected by several factors such as operational difficulties, and logistic problems depending on field work regions. Therefore, for each team, tables associated with completion rates are presented along with their field work provinces (see Table 2). Furthermore, findings put forward that age, place of birth (5 regions of Turkey), educational status, survey experience, and average time of a household interview based on data collection staff have an impact on the incident rate of “completed household interviews” under the control of other variables by looking at the incidence ratios (Table 3). Interestingly, considering survey experience, staff who has never

5

participated in survey, have less incidence risk on the number of completed household

• interviews. The percent change in the incidence rate of the number of completed

household interviews is an increase of approximately 1.38 percent in MA/PhD students compared to university students who are in preparatory, first or second classes. As

• pposed to common belief, being interested in social science is not an advantage for

social research, specifically TDHS, based on the results. Table 1. Data Collection Staff Candidates by Recruitment Steps, TDHS-2013 Data Collection Staff Candidate Status Percentage Number Eliminated during the application process 20.7 79 Did not attend to personal interview even though being invited 35.1 134 Eliminated at the end of the personal interview 5.2 20 Did not attend to the training process even though being invited 0.3 1 Eliminated during training 1.8 7 Did not attend to the main fieldwork even though being invited 1.3 5 Main data collection staff of TDHS-20131 35.6 136 Total 100.0 382

1Data collection staff do not include five project assistants of TDHS-2013 and one

research assistant from HUIPS.

6

Table 2. Household level response and completion rates, TDHS-2013 Provinces Household response rates Household completion rates Düzce-Bartın-Zonguldak-Karabük 92.5 76.4 Erzurum-Gümüşhane-Bayburt 95.9 86.5 Gaziantep-Kilis 93.9 86.8 Çorum-Amasya-Tokat-Sivas 96.3 84.0 Edirne-Kırklareli-Tekirdağ 92.9 81.6 Tunceli-Bingöl-Bitlis-Muş-Van 97.4 85.8 Ağrı-Kars-Ardahan-Iğdır 96.8 90.5 Sinop-Samsun 97.3 87.6 Kırşehir-Nevşehir-Niğde-Aksaray-Kırıkkale 94.6 84.4 Urfa-Hatay 97.2 88.9 Rize-Giresun 89.9 72.3 Kayseri-Yozgat 94.1 81.9 Kahramanmaraş-Elazığ-Adıyaman 94.5 79.8 Kastamonu-Ordu 97.4 86.5 Malatya-Erzincan-Osmaniye 96.0 87.2 Artvin-Trabzon 95.0 76.7 İstanbul 85.4 73.4 Çanakkale-Balıkesir 96.1 81.6 İzmir 91.3 77.7 Aydın-Denizli-Muğla-Afyon 92.7 80.1 Uşak-Kütahya-Manisa-Bilecik-Eskişehir 94.9 85.7 Bursa-Yalova 87.4 73.9 Bolu-Sakarya-Kocaeli-Çankırı 96.0 80.9 Konya-Karaman 94.3 83.3 Burdur-Isparta-Antalya 91.2 78.9 Hatay-Mersin 94.4 82.5 Adana 94.3 85.6 Şırnak-Siirt-Mardin-Diyarbakır-Hakkari-Batman 92.9 84.2 Ankara 92.8 80.8 Total 93.3 81.4

7

Table 3. Results of Poisson Regression Analysis on Number of Completed Household Interviews Variables Significance Exp(B) Age (ref: 25-29) 20-24 0.002 1.081 Place of births (5 regions) (ref: East) West 0.087 0.917 South 0.154 0.932 Central 0.000 1.126 North 0.213 1.052 Educational status (ref: university students in class prep., 1-2) MA/PhD student 0.000 1.384 Graduated from university 0.000 1.309 University student in class 3-4 0.082 1.120 Survey experience (ref: yes) No 0.011 0.938 Average time of a household interview (in minute) 0.000 1.024 *under the interviewer based control variables: working status, interested in social science, language abilities, having an interview in metropolitan cities, mean number of household members, average time of a household interview Conclusion Results confirm that age, experience, and educational status should be taken into consideration in data collection staff recruitment process of social surveys, specifically TDHS, due to it has an effect on number of completed interviews and response rates. Similarly, Groves and McGonagle (2001) and Campanelli et al. (1997) stated that there is an association between getting high response rates and experienced interviewers. For all of these results, considering interviewer characteristics when evaluating candidates is an important step in terms of data quality as well as achieving high response rates and data quality. Strategies, especially when applied in staff recruitment processes, are crucial in terms of data quality and reducing interviewer

8

bias on survey estimates. Interpersonal skills, ability of persuasion and making contact with another person, matching characteristics of interviewers and sample units should be considered in the process. Applicants who have high level of confidence and communication skills should be preferred with the thought of getting higher response

• rates. Additionally, doorstep interaction strategies in terms of gaining cooperation with

the sample unit should be emphasized during the training. Necessity of maintaining respondent motivation should be explained as part of training. To the best of our knowledge, this study can be considered as the first study in Turkey that aims to emphasize interviewer characteristics and examine the characteristics in terms of achieving higher response rates. References Berk, M.L. and A. B. Bernstein (1988) Interviewer Characteristics and Performance on a Complex Health Survey, Social Science Research 17(3), 239-251. Biemer, P. P. and L. E. Lyberg. (2003) Introduction to Survey Quality, John Wiley & Sons, Inc., Hoboken, New Jersey. Campanelli, P., P. Sturgis and S. Purdon (1997) “Can you hear me knocking? and investigation into the impact of interviewers on survey response rates.” National Centre for Social Research. Durbins, J. and A. Stuart (1951). “Differences in response rates of experienced and inexperienced interviewers.” Journal of the Royal Statistical Society. Series A (General), 114(2), 163-206. Fowler Jr, F. J., & Mangione, T. W. (1990). Standardized survey interviewing: Minimizing interviewer-related error (Vol. 18). Sage. Groves, R. M. and M. Couper (1998) Nonresponse in Household Interview Surveys, Wiley-Interscience, New York. Groves, R. M. and K. A. McGonagle (2001). A Theory-Guided Interviewer Training Protocol Regarding’ Survey Participation. Journal of Official Statistics, 17(2), 249. Korbmacher, J. M. (2014). Interviewer Effects on Respondents’ Willingness to Provide Blood Samples in SHARE, SHARE Working Paper Series, 20-2014, 2014.

9

Sala, E., J. Burton, and G. Knies (2012) Correlates of Obtaining Informed Consent to Data Linkage Respondent, Interview, and Interviewer Characteristics. Sociological Methods and Research, 41(3), 414-439. Schuman, H., & Converse, J. M. (1971). The effects of black and white interviewers on black responses in 1968. Public opinion quarterly, 35(1), 44-68. Williams JR, J. A. (1964) Interviewer-respondent interaction: A study of bias in the information interview.” Sociometry, 338-352. Wilson, D.C.and E.P. Olesen (2002). Perceived race of interviewer effects in telephone interviews, in 57th AAPOR/WAPOR Conference, St. Pete Beach.