timing is everything what we can learn from survey
play

Timing is Everything: What we can learn from survey procrastinators - PowerPoint PPT Presentation

Mar 03, 2024 •160 likes •535 views

Timing is Everything: What we can learn from survey procrastinators Presented by: Lauren M. Conoscenti, Ph.D. Tufts University Office of Institutional Research & Evaluation (OIR&E) NEAIR Annual Conference, Newport, Rhode Island

  1. Timing is Everything: What we can learn from “survey procrastinators” Presented by: Lauren M. Conoscenti, Ph.D. Tufts University Office of Institutional Research & Evaluation (OIR&E) NEAIR Annual Conference, Newport, Rhode Island November 9-12, 2013

  2. Web-Based Surveys Tufts, like many institutions, relies heavily on online survey platforms to collect data. Online survey platforms have many advantages over paper-and-pencil surveys. Inexpensive Easy to use Improved distribution Bells-and-whistles Office of Institutional Research, November 13

  3. Web-Based Surveys A major advantage: busy respondents can complete surveys at their convenience. More time = better data? Comfy students = better data? Tech-savvy = better data? A major disadvantage: Low response rates. Office of Institutional Research, November 13

  4. Response Rates Response rates are going down… everywhere. (e.g. Jans & Roman, 2007) Low response rates are ok if sample is representative … …but most aren’t. Nonresponse Error: When survey responders differ from nonresponders in key ways, leading us to draw erroneous conclusions. Office of Institutional Research, November 13

  5. Why Don’t They Participate? Have we traded data quality in favor of ease and price? Why don’t students take web surveys? (Stay tuned, Tuesday at 10:30am) Office of Institutional Research, November 13

  6. Why Don’t They Participate? Online non- response might be because… Unread email routed to spam folder Student temporarily too busy (e.g. exams) Student forgot Office of Institutional Research, November 13

  7. Why Don’t They Participate? Online non- response might be because… Unread email routed to spam folder Student temporarily too busy (e.g. exams) Student forgot …Passive nonresponders. We assume they want to take the survey. Office of Institutional Research, November 13

  8. Why Don’t They Participate? Online non- response might be because… Student doesn’t want to participate …Active nonresponders We hope they are a small fraction of our sample! Office of Institutional Research, November 13

  9. Survey Reminders We send reminders to intervene in passive nonresponding … …and maybe a little active nonresponding, too. We assume people who complete a survey after the reminder are similar to those who completed it before the reminder. Is that a valid assumption? Office of Institutional Research, November 13

  10. Late Responders Non-responders are different from responders Male (McCabe, et al., 2002; Porter & Whitcomb, 2005b) Nonwhite (Couper, Traugott, & Lamias, 2001) Lower GPA (Porter, et al., 2004; Woosley, 2005) Could late responders – or “procrastinators” be different, too? Office of Institutional Research, November 13

  11. Late Responders Past research on this topic has found differences… But most is based on mail surveys… …and findings are mixed. Some evidence that late responders display more problem behavior. Office of Institutional Research, November 13

  12. Late Responders Who is a survey procrastinator? Most survey responses are submitted within 12-24 hours of an email. Survey A: 89.5% within 24 hours of email Survey B: 82.8% Survey C: 87.6% Survey D: 87% Office of Institutional Research, November 13

  13. Late Responders We send reminders to help the passive nonresponders Procrastinators: respondents who wait until a reminder is issued to start a survey. Office of Institutional Research, November 13

  14. Research Questions Are procrastinators different from “regular” responders? Demographic, academic, and survey variables How does the survey incentive impact procrastination (and the procrastinators)? Office of Institutional Research, November 13

  15. Research Method Analyzed data from two undergraduate surveys at Tufts Tufts University… Private Research University – Very high activity Entering class size ~1300 Competitive admissions 4-year Liberal Arts & Engineering undergraduate schools Office of Institutional Research, November 13

  16. The Surveys Sophomore Survey Administered sophomore spring 2012 Survey: Highly incentivized Initial email & 2 reminders Advising, majors, student life, services, civic engagement 2013 Survey Low incentiveInitial email & 6 reminders Added “flourishing scale”, removed advising questions Office of Institutional Research, November 13

  17. The Students 2012: 1073 Liberal Arts sophomores 93.3% Response Rate (LA class size: 1151) 54.5% female Not different from population GPA not significantly different 2013: 718 Liberal Arts sophomores 64% Response Rate (LA class size: 1123) 61% female Different from population Mean GPA higher than nonresponders Office of Institutional Research, November 13

  18. Percent of Class Nonresponding 100% 80% 2013 (Low Incentive) 60% 36.1% 40% 2012 (High Incentive) 20% 6.7% 0% Start R1 R2 R3 R4 R5 R6 End Office of Institutional Research, November 13

  19. Percent of Procrastinators 100% 80% 60% 40% 2013 (Low Incentive) 20% 2012 (High Incentive) 0% Start R1 R2 R3 R4 R5 R6 End Office of Institutional Research, November 13

  20. Results 2012 (High Incentive) Started before a reminder issued: 58.7% Procrastinators (41.3%): More males ( t = 4.52, p <.01) More nonwhite students ( t = 3.80, p <.01) Lower GPA ( t = 7.35, p <.001) Office of Institutional Research, November 13

  21. Results 2012 (High Incentive) Procrastinators: Reported more difficulty choosing a major ( t = 2.06, p <.05) Less likely to make an appointment to see academic advisor… ( t = 2.22, p <.05) …and more likely to drop in at the last minute. ( t = 6.81, p <.01) …and more likely to feel the time spent with their advisor was inadequate. ( t = 2.21, p <.05) Office of Institutional Research, November 13

  22. Results 2012 (High Incentive) Procrastinators: Less likely to participate in community service ( t = 2.99, p <.01) Less likely to feel comfortable asking faculty for help Rec letters ( t = 3.13, p <.01) Academic advising ( t = 2.13, p <.01) Office of Institutional Research, November 13

  23. Results 2012 (High Incentive) Procrastinators: Less likely to choose Tufts again ( t = 2.02, p <.05) Took less time to complete survey ( t = 2.53, p <.01) Proxy of how engaged they were? Office of Institutional Research, November 13

  24. Results 2013 (Low Incentive) Started before first reminder issued: 48.7% Six reminders to get the other 51.3%... Started before second reminder issued: 67% Explore procrastination in two ways: After one reminder or after two …but they were the same. (Stats presented will reflect one reminder) Office of Institutional Research, November 13

  25. Results 2013 (Low Incentive) Procrastinators: Less likely to engage in academic discussions outside of class ( t = 2.11, p <.05) Less likely to feel comfortable asking faculty for help Rec letter ( t = 2.17, p <.05) Additional educational opportunities ( t = 2.14, p <.05) Advising ( t = 1.94, p <.05) Office of Institutional Research, November 13

  26. Results 2013 (Low Incentive) Procrastinators: Less likely to participate in student organizations and community service ( t = 2.05, p <.05; t = 1.99, p <.05) Feel less able to contribute to the campus community ( t = 2.07, p <.05) Office of Institutional Research, November 13

  27. Results 2013 (Low Incentive) Procrastinators: Less satisfied with sense of community on campus ( t = 2.89, p <.01) Perceived prejudice against students from different socioeconomic backgrounds as a campus problem ( t = 3.07, p <.01) Office of Institutional Research, November 13

  28. Results 2013 (Low Incentive) Procrastinators: Less time to complete survey ( t = 1.91, p <.05) Less likely to have left a final comment ( t = 2.11, p <.05) (“Please use this space below to provide any additional comments about your Tufts experience.”) Office of Institutional Research, November 13

  29. Results 2013 (Low Incentive) No differences: Likelihood of choosing Tufts again GPA % Male % nonwhite Flourishing scale Office of Institutional Research, November 13

  30. 2012 vs. 2013 Early Late Early Late Non- Category Responders Responders Responders Responders responders 2012 2012 2013 2013 2013 GPA 3.48 3.31 3.45 3.43 3.35 % Male 39% 52% 37% 41% 57% % 24% 35% 34% 32% 31% Nonwhite Office of Institutional Research, November 13

  31. Percent of Class Nonresponding 100% 80% 2013 (Low Incentive) 60% 36.1% 40% 20% 6.7% 2012 (High Incentive) 0% Start R1 R2 R3 R4 R5 R6 End Office of Institutional Research, November 13

  32. Take Home Points Procrastinators were different from non- procrastinators More academically adrift Less engaged on campus More dissatisfied Those who waited for a reminder spent less time on the survey Office of Institutional Research, November 13

  33. Take Home Points Procrastinators different from non- responders Students who procrastinate in high- incentive situations participate because they have to. They otherwise wouldn’t be doing the survey. Office of Institutional Research, November 13

  34. Take Home Points Reminders are good! Send them. Dissatisfied and less engaged students procrastinate – wait for them! Plan your survey administration carefully. Incentives? When to send reminders? How many? Office of Institutional Research, November 13

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Tim ing is Everything: W hat w e can learn from survey procrastinators Presented by:

Tim ing is Everything: W hat w e can learn from survey procrastinators Presented by:

Tim ing is Everything: W hat w e can learn from survey procrastinators Presented by: Lauren M. Conoscenti, Ph.D. Tufts University Office of Institutional Research &amp; Evaluation (OIR&amp;E) AIR Annual Forum, Long Beach, California May

680 views • 30 slides
New methods for controlling timing channels Andrew Myers Cornell University (with Danfeng

New methods for controlling timing channels Andrew Myers Cornell University (with Danfeng

New methods for controlling timing channels Andrew Myers Cornell University (with Danfeng Zhang, Aslan Askarov) Timing channels e adversary can learn (a lot) from timing measurements. Known to exist Hard to detect Hard to prevent

639 views • 40 slides
2011 Citizen Survey Survey Administration Review of Feedback Future Action Plans February 27,

2011 Citizen Survey Survey Administration Review of Feedback Future Action Plans February 27,

2011 Citizen Survey Survey Administration Review of Feedback Future Action Plans February 27, 2012 www.cityofws.org 1 Prior Survey Methods Telephone Surveys Conducted 2006, 2007 and 2008 Structure and timing of survey limited

489 views • 14 slides
Origins of the project building condition survey Plan for funding and timing of the

Origins of the project building condition survey Plan for funding and timing of the

Address elements of the project over next several Board of Education workshops Origins of the project building condition survey Plan for funding and timing of the project Nov 1st Dec 6th High school and middle school maintenance

560 views • 36 slides
Surveying the Survey. What can we Learn about the Effect of Monetary Policy on Inflation

Surveying the Survey. What can we Learn about the Effect of Monetary Policy on Inflation

Surveying the Survey. What can we Learn about the Effect of Monetary Policy on Inflation Expectations? Michael Pedersen Central Bank of Chile XXV Reunin de la Red de Investigadores de Bancos Centrales CEMLA, virtual, 28-30 October 2020 * The

1.42k views • 66 slides
Learn Blackboard Learn Learn with others Learn in your own time, pace, space Learn through

Learn Blackboard Learn Learn with others Learn in your own time, pace, space Learn through

Learn Blackboard Learn Learn with others Learn in your own time, pace, space Learn through designing for teaching Learn with others 1. Black Swan Conferences Hear The bigger picture Higher Education Blackboard UWA Experience

478 views • 16 slides
Timing Analysis Timing Path Groups and Types Timing paths are grouped into path groups

Timing Analysis Timing Path Groups and Types Timing paths are grouped into path groups

Introduction to Digital VLSI Logic Synthesis Timing Analysis Timing Path Groups and Types Timing paths are grouped into path groups according to the clock associated with the endpoint of the path. There is a default path group that

540 views • 18 slides
FORMAL MODELING AND VERIFICATION FOR TIMING PREDICTABILITY Mathieu Jan, Mihail Asavoae, Belgacem

FORMAL MODELING AND VERIFICATION FOR TIMING PREDICTABILITY Mathieu Jan, Mihail Asavoae, Belgacem

FORMAL MODELING AND VERIFICATION FOR TIMING PREDICTABILITY Mathieu Jan, Mihail Asavoae, Belgacem Ben Hedia ERTS 2020 Toulouse Outline of the talk Motivations and goals Timing anomalies in Worst-Case Timing Reasoning An example of a timing

286 views • 16 slides
Top 5 Timing Closure Techniques Greg Daughtry Correct Timing Constraints Analyze Before

Top 5 Timing Closure Techniques Greg Daughtry Correct Timing Constraints Analyze Before

Top 5 Timing Closure Techniques Greg Daughtry Correct Timing Constraints Analyze Before Doing Implementation Strategies and Directives Congestion and Complexity Advanced Physical Optimization Create Good Timing Constraints

658 views • 31 slides
X-ray ray Timing and Timing and Polarizati Polarization on mi missi ssion &amp; &amp; inst

X-ray ray Timing and Timing and Polarizati Polarization on mi missi ssion &amp; &amp; inst

X-ray ray Timing and Timing and Polarizati Polarization on mi missi ssion &amp; &amp; inst strumen mentation DONG Y Yongw ngwei Center fo r for r Parti rticle A Astroph trophysics Ins nstitute o of High Ene igh Energy P gy

437 views • 15 slides
Timing and Coordination Essential Knowledge 2.E.2 and 2.E.3 Timing and Coordination Timing

Timing and Coordination Essential Knowledge 2.E.2 and 2.E.3 Timing and Coordination Timing

Timing and Coordination Essential Knowledge 2.E.2 and 2.E.3 Timing and Coordination Timing and coordination of physiological events are regulated by multiple mechanisms ( 2.E.2 ) Timing and coordination of physiological events are

598 views • 37 slides
A Highly Compressed Timing Macro-modeling Algorithm for Hierarchical and Incremental Timing

A Highly Compressed Timing Macro-modeling Algorithm for Hierarchical and Incremental Timing

A Highly Compressed Timing Macro-modeling Algorithm for Hierarchical and Incremental Timing Analysis Tin-Yin Lai, and Martin D. F. Wong March. 16, 2018 Outline Introduction Timing Macro-modeling Problem Formulation Previous

397 views • 23 slides
Digital Design Discussion: RTL Storage Components Shift Register Timing Register File Timing

Digital Design Discussion: RTL Storage Components Shift Register Timing Register File Timing

Principles Of Digital Design Discussion: RTL Storage Components Shift Register Timing Register File Timing RAM Up/Down Counter FIFO Queue Shift Register Timing Problem: Complete the timing diagram for the shift register shown. Assume

302 views • 12 slides
Timing Library Format (TLF) Advanced VLSI Design CMPE 414 Timing Library Format (TLF) TLF is an

Timing Library Format (TLF) Advanced VLSI Design CMPE 414 Timing Library Format (TLF) TLF is an

Timing Library Format (TLF) Advanced VLSI Design CMPE 414 Timing Library Format (TLF) TLF is an ASCII representation of the timing and power parameters associated with any cell in a particular semiconductor technology The timing and power

308 views • 15 slides
Agenda Compare 4 alternative timing mechanisms (long and short the S&amp;P 500)

Agenda Compare 4 alternative timing mechanisms (long and short the S&amp;P 500)

Agenda Compare 4 alternative timing mechanisms (long and short the S&amp;P 500) VectorVest confirmed ups and downs EMA / RT combo Timing Cube Ride the Waves Show the results of using the best timing method with a pair

804 views • 44 slides
1 Clock skew optimization Another approach for sequential timing optimization

1 Clock skew optimization Another approach for sequential timing optimization

Sequential Timing Optimization Long path timing constraints Data must not reach destination FF too late d max (i,j) s i + d(i,j) + T setup s j + P i j s s i i sj d(i,j) T setup Short path timing constraints FF should not

85 views • 7 slides
High dynamic range imaging Digital Visual Effects, Spring 2006 Yung-Yu Chuang 2006/3/8 with

High dynamic range imaging Digital Visual Effects, Spring 2006 Yung-Yu Chuang 2006/3/8 with

Announcements Room change to 102? Assignment #1 is online (due on 3/ 25 midnight) High dynamic range imaging Digital Visual Effects, Spring 2006 Yung-Yu Chuang 2006/3/8 with slides by Fedro Durand, Brian Curless, Steve Seitz and

359 views • 12 slides
Discussion of Monetary Policy Drivers of Bond and Equity Returns by Campbell, Pflueger and

Discussion of Monetary Policy Drivers of Bond and Equity Returns by Campbell, Pflueger and

Discussion of Monetary Policy Drivers of Bond and Equity Returns by Campbell, Pflueger and Viceira Martin Lettau UC Berkeley Monetary Policy and Financial Markets Federal Reserve Bank of San Francisco, March 28, 2014 Martin Lettau

491 views • 15 slides
Effec%ve Technical Presenta%ons Lawrence O. Hall Department of

Effec%ve Technical Presenta%ons Lawrence O. Hall Department of

Effec%ve Technical Presenta%ons Lawrence O. Hall Department of Computer Science and Engineering, ENB118 University of South Florida 4202 E. Fowler

563 views • 28 slides
PRESENTATION When used tastefully, visual props are an effective way to complement your

PRESENTATION When used tastefully, visual props are an effective way to complement your

By Terri Williams, eHow Contributor updated July 07, 2011 POINTS THAT CREATE A GREAT ORAL PRESENTATION When used tastefully, visual props are an effective way to complement your presentation. The devil is in the details, and you should

490 views • 13 slides
PHYSICIAN WELLNESS AND ENGAGEMENT POTENTIAL SOLUTIONS FOR AN ORGANIZATIONAL CHALLENGE

PHYSICIAN WELLNESS AND ENGAGEMENT POTENTIAL SOLUTIONS FOR AN ORGANIZATIONAL CHALLENGE

PHYSICIAN WELLNESS AND ENGAGEMENT POTENTIAL SOLUTIONS FOR AN ORGANIZATIONAL CHALLENGE DEFINITION OF BURNOUT A work-related syndrome involving emotional exhaustion, depersonalization and a sense of reduced personal accomplishment The

479 views • 26 slides
ORGANIZATION Whether youre a Panster or a Plotter some planning and organization is helpful

ORGANIZATION Whether youre a Panster or a Plotter some planning and organization is helpful

ORGANIZATION Whether youre a Panster or a Plotter some planning and organization is helpful Panster Plotter Set a goal Set a goal Set aside time to write Set aside time to write Develop a plot maybe Develop

296 views • 12 slides
Spatial Modeling, Regional Science, and UCSB Arthur Getis Emeritus, San Diego State University

Spatial Modeling, Regional Science, and UCSB Arthur Getis Emeritus, San Diego State University

Spatial Modeling, Regional Science, and UCSB Arthur Getis Emeritus, San Diego State University March 1, 2016 My Link to UCSB The 1980s at UCSB (summers and sabbatical) Problems within Geography: The Quantitative Revolution

255 views • 23 slides
Distributed Optimization of CNNs and RNNs GTC 2015 William Chan williamchan.ca

Distributed Optimization of CNNs and RNNs GTC 2015 William Chan williamchan.ca

Distributed Optimization of CNNs and RNNs GTC 2015 William Chan williamchan.ca williamchan@cmu.edu March 19, 2015 Outline 1. Motivation 2. Distributed ASGD 3. CNNs 4. RNNs 5. Conclusion Carnegie Mellon University 2 Motivation Why

357 views • 22 slides

More recommend