People are fed up with taking surveys

And who is surprised by this piece of news? But it may matter for the quality of data collected by stats agencies such as StatsNZ. And that matters for policy determination.

In a recent article in the Journal of Economic Perspectives, "Household Surveys in Crisis" by Bruce D. Meyer, Wallace K. C. Mok and James X. Sullivan, it is argued that declining cooperation by survey respondents is adversely affecting the quality of data being collected.

The abstract reads,

Household surveys, one of the main innovations in social science research of the last century, are threatened by declining accuracy due to reduced cooperation of respondents. While many indicators of survey quality have steadily declined in recent decades, the literature has largely emphasized rising nonresponse rates rather than other potentially more important dimensions to the problem. We divide the problem into rising rates of nonresponse, imputation, and measurement error, documenting the rise in each of these threats to survey quality over the past three decades. A fundamental problem in assessing biases due to these problems in surveys is the lack of a benchmark or measure of truth, leading us to focus on the accuracy of the reporting of government transfers. We provide evidence from aggregate measures of transfer reporting as well as linked microdata. We discuss the relative importance of misreporting of program receipt and conditional amounts of benefits received, as well as some of the conjectured reasons for declining cooperation and for survey errors. We end by discussing ways to reduce the impact of the problem including the increased use of administrative data and the possibilities for combining administrative and survey data.

Given the claims we hear about evidence based policy being the big thing these days these issues matter. There is little point in basing policy on evidence if that evidence is crap.

Ref.:

• Meyer, Bruce D., Wallace K. C. Mok and James X. Sullivan. 2015. "Household Surveys in Crisis." Journal of Economic Perspectives, 29(4):199-226.