Conducting survey methods research in an official statistics production environment: Why is it hard? How do you make it work? Jaki S. McCarthy USDA’s National Agricultural Statistics Service B Based on a poster presented at d d the Journal Of Official Statistics 30 th Anniversary Conference June 2015
Research in Survey Methods is Key to Improving Statistics y p g • Survey methods research can be conducted outside the statistical production environment – Stand alone experiments – Analysis of existing data Analysis of existing data • Pros – No impact on on-going data N i t i d t – May be easier to manipulate conditions • Cons – Doesn’t mimic real survey conditions – Results often out of context, infeasible or do not translate to operations
Research embedded in production provides unique benefits, but…… • Research in production is HARD! h i d i i ! • What can we do to address that? • What can we do to address that?
COMMUNICATION! • COMMUNICATION • COMMUNICATION • COMMUNICATION
Wh i it h Why is it hard? d? Wh t What can you do to address that? d t dd th t? “The two words 'information' and 'communication' are often used interchangeably, but they signify quite different things. Information is giving out; communication is getting through ” Information is giving out; communication is getting through. ---Sydney J. Harris
No benefit is seen in doing research • Choose your problems wisely! y p y – Understand what problems are important to management or operational staff ti l t ff – Vet research with operational staff – Enlist program management support for research – Clearly state how your research outcomes can be beneficial
Or…..clear benefits are perceived so operational staff want to jump right to production j p g p • Explain how research can demonstrate benefits (or costs) p ( ) – Be clear when answers are not known – Understand potential negative impacts that should be evaluated – Explain how research can MEASURE benefits and costs – Or be used to evaluate what NOT to do Or be used to evaluate what NOT to do
Research staff do not understand production environment • Do your homework! y – Researchers must fully understand how surveys work, what is required, survey timelines and dependencies – Know who in your organization is responsible for what Know who in your organization is responsible for what – Build relationships with operational staff – go grab a beer!
Concern about negative impacts on survey operations on survey operations • Understand what is required BOTH to conduct research and q to implement findings – Do as much as you can to reduce effort needed from operational staff staff – Find operational partners to help you understand how to get research embedded in operations – Make sure any findings CAN be implemented – Work with operational staff to understand HOW findings would be implemented – Be willing to do development work beyond your research
Operational staff are busy getting their jobs done • Share your results and then share them again y g – Bring research findings to operational staff – Document results and archive them where people can find them – Communicate more than you think you should have to (with C i h hi k h ld h ( i h everyone)
Concern about disrupting time series, even if improvements can be made p • Plan for phased implementation of research, if necessary p p , y – Remember research results are not research implementation – With your results, develop a realistic plan for phased implementation – Create a plan for how to handle disruption in time series C l f h h dl di i i i i
You can’t do anything without good communication! • You can do the best research and be making the strongest g g intellectual argument, but if readers don't get past the third paragraph you've wasted your energy and valuable ink. Carl Hiaasen Carl Hiaasen
Recommend
More recommend
